Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

There are specific lines that I want to remove from a file. Let's say it's line 20-37 and then line 45. How would I do that without specifying the content of those lines?

share|improve this question
    
How big is your file? Could it be loaded into memory? –  Faheem Mitha Apr 13 '11 at 16:41
    
A few kilobytes. –  Tshepang Apr 13 '11 at 22:40

2 Answers 2

up vote 23 down vote accepted

With sed, like so:

sed '20,37d; 45d' < input.txt > output.txt

If you wanted to do this in-place:

sed --in-place '20,37d; 45d' file.txt
share|improve this answer
    
Is there a way of doing it in-place? –  Tshepang Apr 12 '11 at 21:02
    
I suggest sed -i file –  enzotib Apr 12 '11 at 21:24
1  
@Tshepang: Use ed, or GNU sed -i, or sponge, or a large file method. –  Gilles Apr 12 '11 at 22:00
2  
I've often wondered about the possibly misleading term in-place, when referring to 'sed', so I looked it up in 'man sed': --in-place[=SUFFIX] This option specifies that files are to be edited in-place. GNU sed' does this by creating a temporary file and sending output to this file rather than to the standard output.` ... I don't know about any other 'sed' but the logistics of updating "in place" with a stream editor don't "compute" :) –  Peter.O Apr 13 '11 at 14:36
1  
Most "in-place" methods use a temporary file, in my experience. –  Faheem Mitha Apr 14 '11 at 10:08

Just read it into memory, alter it, then write it back. You can do something like

filename = "foo"
f = open(filename, 'r+')                                                                                                                                 
linenums = [1, 3]                                                                                                                                            
s = [y for x, y in enumerate(f) if x not in [line-1 for line in linenums]]                                                                                                                                          
f.seek(0)
f.write(''.join(s))
f.truncate(f.tell())
f.close()

Tested with a 5 line file. Credits to http://pleac.sourceforge.net/pleac_python/fileaccess.html, see section "Modifying a File in Place Without a Temporary File". See also http://stackoverflow.com/questions/125703/how-do-i-modify-a-text-file-in-python

Some notes:

  1. One could first truncate the file, then write to it, rather than write, then truncate, as above. However, I don't know of a Python flag that allows one to read, and then do a truncated write. But maybe I'm missing something, as the document isn't all that clear. Which brings me to

  2. Sometimes the Python docs really suck. See http://docs.python.org/library/functions.html#open

    Modes 'r+', 'w+' and 'a+' open the file for updating (note that 'w+' truncates the file).

    Does this mean anything to you? What the hell is "open for updating"?

  3. I don't know if doing this in python as opposed to something unixy like the stream editor is better. It might be more portable, but I don't know how portable sed is. I just wrote it like that because I'm more comfortable with low level programming than using classic unix tools, which are good if they do exactly what you want, but (I think) are generally less flexible.

  4. This approach (manipulating the file in memory) trades memory for disk space. It should work Ok on machines with a few Gb of memory for files up to a few hundred Mb. Python doesn't handle strings very efficiently, so switching to C/C++ for example would slightly increase performance and greatly reduce memory usage.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.