33 
> brew install moreutils                                                          
==> Downloading https://homebrew.bintray.com/bottles/moreutils-0.55.yosemite.bottle.tar.gz    
######################################################################## 100.0%               
==> Pouring moreutils0.55.yosemite.bottle.tar.gz       
🍺  /usr/local/Cellar/moreutils/0.55: 67 files, 740K

sponge reads standard input and writes it out to the specified file. Unlike a shell redirect, sponge soaks up all its input before writing the output file. This allows constructing pipelines that read from and write to the same file.

I don't understand. Please give me some useful examples.

What does soaks up mean?

Improve this question
3

3 Answers 3

Reset to default
47

Assume that you have a file named input, you want to remove all line start with # in input. You can get all lines don't start with # using:

grep -v '^#' input

But how do you make changes to input? With standard POSIX toolchest, you need to use a temporary file, some thing like:

grep -v '^#' input >/tmp/input.tmp
mv /tmp/input.tmp ./input

With shell redirection:

grep -v '^#' input >input

will truncate input before you reading from it.

With sponge, you can:

grep -v '^#' input | sponge input
Improve this answer
11
  • 5
    You actually can read and write a file at the same time safely as long as the bytes are only being transformed, using the <> operator.
    – Chris Down
    Commented Jun 6, 2015 at 9:14
  • @ChrisDown: Yes, I mean without making it corroupt
    – cuonglm
    Commented Jun 6, 2015 at 9:19
  • 1
    @ChrisDown: Let me remove that sentence to avoid confusing. I actually mean when using <>file, you open file for reading and writing but you actually don't write anything to file.
    – cuonglm
    Commented Jun 7, 2015 at 7:12
  • Did you try it? :-) <> has no problem writing to a file, try it for yourself: printf foobar >q; printf bar 1<>q results in barbar.
    – Chris Down
    Commented Jun 7, 2015 at 7:46
  • 8
    I think the point that @ChrisDown is trying to make is that <> doesn't truncate a file, but merely replaces its existing bytes with the new output. If the new output is too short, you'll have leftover garbage at the end of the file. But if the new output is long enough, there's no risk.
    – BallpointBen
    Commented Aug 2, 2018 at 13:56
20

The moreutils home page itself documents a typical use case:

sed "s/root/toor/" /etc/passwd | grep -v joey | sponge /etc/passwd

Here, /etc/passwd is both being written to and read to, and is being modified. Without sponging up stdin before writing, /etc/passwd might be corrupted (as the file changed during reading).

Improve this answer
1
  • 4
    And that would be a good example on the moreutils page, if it had explained the way you did :-)
    – Br.Bill
    Commented Jan 24, 2020 at 22:55
0

The easiest example of sponge I've ever read:

$ cat file1 
I
You
Me
We
Us

To sort it, you can simply

$ sort file1 > file1_sorted

However, with sponge, you can read from and write to the same file.

$ sort file1 | sponge file1

without requiring the temporary file file1_sorted. This is because sponge first read from STDIN, and then write it out to STDOUT, in separate steps, thus avoiding corrupting the file as it is changed during the reading.

Improve this answer
1
  • 1
    Using sort as an example is a bit awkward, as you would rather use sort -o file1 file1 to sort the file in place (i.e., sort already has this "built-in", so to speak).
    – Kusalananda ♦
    Commented Mar 12 at 9:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .