Is there a standard alternative to sponge to pipe a file into itself?

Question

I frequently want to do something like this:

 cat file | command > file

(which obviously doesn't work). The only solution I've seen for this is sponge, i.e.

 cat file | command | sponge file

Unfortunately, sponge is not available to me (nor can I install it or any other package).

Is there a more standard quick way to do this without having to break it up every time into multiple commands (pipe to temp file, pipe back to original, delete temp file)? I tried tee for example, and it seems to work, but is it a consistent/safe solution?

Answer 1

A shell function replacing sponge:

mysponge () (
    append=false

    while getopts 'a' opt; do
        case $opt in
            a) append=true ;;
            *) echo error; exit 1
        esac
    done
    shift "$(( OPTIND - 1 ))"

    outfile=$1

    tmpfile=$(mktemp "$(dirname "$outfile")/tmp-sponge.XXXXXXXX") &&
    cat >"$tmpfile" &&
    if "$append"; then
        cat "$tmpfile" >>"$outfile"
    else
        if [ -f "$outfile" ]; then
            chmod --reference="$outfile" "$tmpfile"
        fi
        if [ -f "$outfile" ]; then
            mv "$tmpfile" "$outfile"
        elif [ -n "$outfile" ] && [ ! -e "$outfile" ]; then
            cat "$tmpfile" >"$outfile"
        else
            cat "$tmpfile"
        fi
    fi &&
    rm -f "$tmpfile"
)

This mysponge shell function passes all data available on standard input on to a temporary file.

When all data has been redirected to the temporary file, the collected data is copied to the file named by the function's argument. If data is not to be appended to the file (i.e -a is not used), and if the given output filename refers to an existing regular file, if it does not exist, then this is done with mv (in the case that the file is an existing regular file, an attempt is made to transfer the file modes to the temporary file using GNU chmod first). If the output is to something that is not a regular file (a named pipe, standard output etc.), the data is outputted with cat.

If no file was given on the command line, the collected data is sent to standard output.

At the end, the temporary file is removed.

Each step in the function relies on the successful completion of the previous step. No attempt is made to remove the temporary file if one command fails (it may contain important data).

If the named file does not exist, then it will be created with the user's default permissions etc., and the data arriving from standard input will be written to it.

The mktemp utility is not standard, but it is commonly available.

The above function mimics the behaviour described in the manual for sponge from the moreutils package on Debian.

---

Using tee in place of sponge would not be a viable option. You say that you've tried it and it seemed to work for you. It may work and it may not. It relies on the timing of when the commands in the pipeline are started (they are started pretty much concurrently), and the size of the input data file.

The following is an example showing a situation where using tee would not work.

The original file is 200000 bytes, but after the pipeline, it's truncated to 32 KiB (which could well correspond to some buffer size on my system).

$ yes | head -n 100000 >hello
$ ls -l hello
-rw-r--r--  1 kk  wheel  200000 Jan 10 09:45 hello

$ cat hello | tee hello >/dev/null
$ ls -l hello
-rw-r--r--  1 kk  wheel  32768 Jan 10 09:46 hello

Answer 2

There's this short bash script, which requires Perl
https://github.com/ildar-shaimordanov/perl-utils#sponge

The second script should be a drop-in replacement for the version in moreutils

There's also a version that is a stand-alone perl script.

Answer 3

function wf() {
    #create a temporary file
    local tmpf="${1}_$(< /dev/urandom tr -dc A-Za-z0-9 | head -c16)"
    #redirect the result
    cat > $tmpf
    #replace the original file
    mv -f $tmpf "${1}"
}

next we use the function

grep "error" messages.log | wf messages.log

Answer 4

Why take cannons to shoot flies? A possible solution is this:

stdin-to-file () {
local function_name="${FUNCNAME[0]}"
local tmp_file
local append=false
local exit_code=0
for (( i=1; i<=$#; i++ )); do
    if [[ ${!i} = -- ]]; then
        set -- "${@:1:i-1}" "${@:i+1}"
        break
    fi
    if [[ ${!i} = -a || ( --append = ${!i}* && $(expr length "${!i}") -ge 3 ) ]]; then
        append=true
        set -- "${@:1:i-1}" "${@:i+1}"
        ((i--))
        continue
    fi
done
if [[ $# -ne 1 || -t 0 ]]; then
    echo "$function_name: Wrong number of arguments or missing stdin." >&2
    return 1
fi
tmp_file="$(mktemp "/tmp/$(basename -- "$1")-XXXXXXXXXXXX")" &&
cat > "$tmp_file" &&
if $append; then
    cat "$tmp_file" >> "$1"
else
    cat "$tmp_file" > "$1"
fi ||
exit_code=$?
rm -f -- "$tmp_file"
if [[ $exit_code != 0 ]]; then
    echo "$function_name: An error has occurred." >&2
fi
return $exit_code
}

Then:

cat file | command | stdin-to-file file

For append:

cat file | command | stdin-to-file -a file

Or:

cat file | command | stdin-to-file --append file