Due to an application bug as yet undiagnosed, I have several hundred servers with a full disk. There is one file that has been filled up with duplicate lines—not a log file, but a user environment file with variable definitions (so I can't just delete the file).
I wrote a simple sed
command to check for the erroneously added lines and delete them, and tested it on a local copy of the file. It worked as intended.
However, when I tried it on the server with the full disk, I got approximately the following error (it's from memory, not copy and paste):
sed: couldn't flush /path/to/file/sed8923ABC: No space left on deviceServerHostname
Of course, I know there's no space left. That's why I'm trying to delete stuff! (The sed
command I'm using will reduce a 4000+ line file to about 90 lines.)
My sed
command is just sed -i '/myregex/d' /path/to/file/filename
Is there a way I can apply this command despite the full disk?
(It must be automated, since I need to apply it to several hundred servers as a quick-fix.)
(Obviously the application bug needs to be diagnosed, but in the meantime the servers aren't working correctly....)
Update: The situation I faced was resolved by deleting something else that I found out I could delete, but I'd still like the answer to this question, which would be helpful in the future and for other people.
/tmp
is a no-go; it's on the same filesystem.
Before I freed up disk space, I did test and find out that I could delete the lines in vi
by opening the file and running :g/myregex/d
and then successfully save the changes with :wq
. It seems it should be possible to automate this, without resorting to a separate filesystem to hold a temp file.... (?)
sed -i
creates a temporary copy to operate on. I suspect thated
would be better for this, though I'm not familiar enough to proscribe an actual solutioned
you'd run:printf %s\\n g/myregex/d w q | ed -s infile
but keep in mind some implementations also use temporary files just likesed
(you could try busybox ed - afaik it doesn't create a temporary file)echo
. useprintf
. and makesed
append some char you drop at the last line so you can avoid losing trailing blanks. also, your shell needs to be able to handle the whole file in a single command-line. that's your risk - test first.bash
is especially bad at that (i think its to do w/ stack space?) and may sick up on you at any time. the twosed
's i recommended would at least use the kernel's pipe buffer to good effect between them, but the method is fairly similar. your command sub thing will also truncatefile
whether or not the sed w/in is successful.sed '/regex/!H;$!d;x' <file|{ read v && cat >file;}
and if it works read the rest of my answer.'