2
votes
4answers
62 views

Not enough disk space: remove first n lines of text file without copying the file [duplicate]

I want to remove the first $rmv lines of a huge text file called $filename. This text file is so big that I cannot fit two copies of it on my hard drive. The following leaves me with a blank file ...
76
votes
7answers
16k views

dd vs cat — is dd still relevant these days?

I recently realized we can use cat as much as dd, and it's actually faster than dd I know that dd was useful in dealing with tapes where block size actually mattered in correctness, not just ...
48
votes
3answers
10k views

What are the shell's control and redirection operators?

I often see tutorials online that connect various commands with different symbols. For example: command1 | command2 command1 & command2 command1 || command2 command1 && command2 ...
37
votes
5answers
24k views

How can I make iconv replace the input file with the converted output?

I have a bash script which enumerates through every *.php file in a directory and applies iconv to it. This gets output in STDOUT. Since adding the -o parameter ( in my experience ) actually writes ...
35
votes
8answers
19k views

How to obtain inverse behavior for `tail` and `head`?

Is there a way to head/tail a document and get the reverse output; because you don't know how many lines there are in a document? I.e. I just want to get everything but the first 2 lines of foo.txt ...
19
votes
4answers
18k views

How do I remove certain lines (using line numbers) in a file?

There are specific lines that I want to remove from a file. Let's say it's line 20-37 and then line 45. How would I do that without specifying the content of those lines?
16
votes
4answers
15k views

Delete the first n bytes of files

I've got an extreme problem, and all of the solutions I can imagine are complicated. According to my UNIX/Linux experience there must be an easy way. I want to delete the first 31 bytes of each file ...
10
votes
8answers
950 views

sed in-place line deletion on full filesystem?

Due to an application bug as yet undiagnosed, I have several hundred servers with a full disk. There is one file that has been filled up with duplicate lines—not a log file, but a user environment ...
5
votes
9answers
4k views

Is there a faster way to remove a line (given a line number) from a file?

A related question is here. I often have to edit a large file by removing a few lines from the middle of it. I know which lines I wish to remove and I typically do the following: sed "linenum1,...
3
votes
4answers
2k views

Is is possible to modify a file in a read only directory? [duplicate]

I have already read: Is there a way to modify a file in-place? I am curious if there is a way to modify a file in place using a command that will not create temporary files. Let's say I create a ...
10
votes
3answers
429 views

bash: Some issue when using read <<<“$VARIABLE” on a read-only root partition. Any known workarounds?

Just by coincidence I had to use my ATA-ID-to-device-name script (found here: http://serverfault.com/questions/244944/linux-ata-errors-translating-to-a-device-name/426561#426561) on a read-only / ...
2
votes
1answer
2k views

remove carriage return character from large binary file

I have a large binary file ( nearly binary file ! ) which has many \r\n in it [ gets Generated in windows env and doesnt have controle over generating application ]. I want to remove \r from this ...
4
votes
2answers
589 views

How to delete part of a binary file without copying

I need to delete the first 2 bytes of a 6MB file. However, this is an embedded Linux with only 32 Mbytes RAM and less than 1 MB free flash memory. I tried using dd, as: 1 - # dd bs=1 skip=2 count=...
1
vote
4answers
154 views

“Multipass” scripted modification of large file in-place (file-system level)?

I have just came to the problem of having to cut some lines from a large (gigabyte) sized file, and being aware of potential CPU hog trying to read it in memory, I wanted to edit it in-place instead......
1
vote
1answer
78 views

Can I use tail with a file both as input and destination?

I have a log file that gets big fast. I tried using tail with this syntax but it didn't work. tail logfile.log -n 100000 > logfile.log The output file is 0 bytes and blank. What am I doing wrong?...

15 30 50 per page