Possible Duplicate:
How can I make iconv replace the input file with the converted output?

I frequently connect to amazon ec2 using their public DNS names
(ec2-12-34-56-78.compute-1.amazonaws.com) and because of this my known_hosts file gets overwhelmed with a lot of ec2 entries I will never use again.

I know I could probably use sed -i to edit in place but I wanted to use grep and so I did this:

grep -v ec2 ~/.ssh/known_hosts > ~/.ssh/known_hosts

That leaves known_hosts as an empty file. If I do:

grep -v ec2 ~/.ssh/known_hosts > ~/.ssh/tmp
mv ~/.ssh/tmp ~/.ssh/known_hosts

then things are fine, but I am confused why reading and writing to the same file leaves it blank, and if there is any way around this when using grep, cat, etc.

  • 1
    Even if the program were doing the redirection itself, opening the file for read and then for write before actually reading from it would result in reading nothing (unless the language's standard library speculatively prebuffered, but that means you can't configure the buffer first). What were you expecting to happen?
    – geekosaur
    Apr 11, 2012 at 19:33
  • 1
    Because it's actually a related issue, not the original problem (the shell is doing the file create, so grep would not have a chance to do anything about it anyway). I'm just confused as to how it would be expected to work in the first place.
    – geekosaur
    Apr 11, 2012 at 20:00
  • 1
    This is not the first time I've experienced this, so I don't "expect" to to work anymore, but initially I assumed that input flows from left to right and you operate on it on the left side and then would output it to the file on the right side. Guess part of the problem is that years ago I was a windows user and something similar would work in DOS.
    – cwd
    Apr 11, 2012 at 20:34
  • 1
    Even DOS (2 and up) would do "the wrong thing" in most circumstances, although it at least would do the speculative prebuffer in many cases so you would see it appeart to work if the file was small enough; moreover, DOS 2 and later had shell redirection in COMMAND.COM and so had the overwrite happen before the command even ran.
    – geekosaur
    Apr 11, 2012 at 20:38
  • 3
    @cwd Hence the value of having this one closed as a duplicate, with a highly-visible link to the other question. Apr 11, 2012 at 23:06

2 Answers 2


There is a utility called sponge that is a part of the moreutils suite. It was made for this exact purpose.

grep -v ec2 ~/.ssh/known_hosts | sponge ~/.ssh/known_hosts
  • Why not just piping to cat? Like: ... | cat > ~/.ssh/known_hosts ?
    – hek2mgl
    Jul 3, 2014 at 13:47
  • 4
    cat can write to its output at any time; if it writes past the point at whatever is on the other side of the pipe has read, then you will read in what you wrote instead of the original file contents. That is what makes sponge special. It buffers its input until stdin closes, then writes. Jul 3, 2014 at 13:54
  • 1
    This might be interesting: If sponge isn't available, you could emulate it using sed: ... | sed ':a;N;ba'
    – hek2mgl
    Jul 4, 2014 at 11:49
  • 1
    sponge is also part of sbase
    – Sebastian
    Oct 16, 2014 at 13:23

Redirections are done by the shell, before the command runs. This means that the shell is told to truncate the file before grep gets a chance to read it. There is no way around this if you are using shell redirection.

Not the answer you're looking for? Browse other questions tagged or ask your own question.