"Make sure not to read and write the same file in the same pipeline" [duplicate]

Question

shellcheck reports:

echo -e "blah/blah\n$(cat "$tmpdir"/"$filename".jpdf)" > "$tmpdir"/"$filename".jpdf
                          ^-- SC2094: Make sure not to read and write the same file in the same pipeline.
                                                         ^-- SC2094: Make sure not to read and write the same file in the same pipeline.

The command is intended to insert a line blah/blah at the beginning of a file "$tmpdir"/"$filename".jpdf.

What does "Make sure not to read and write the same file in the same pipeline" mean?

What shall I do instead?

Thanks.

Answer 1

As the other answers pointed out, it is warning you that reading from and writing to the same file will erase the file. The reason is that > causes the file to be truncated before input is finished.

You have two options,

1. Use sponge from the moreutils package. sponge ensures that the input is fully consumed before it is piped to the next command.

   Instead of

   command "$file" > "$file"
   

   do

   command "$file" | sponge "$file"
   

2. Redirect output to another file and rename afterwards.

   Like this

   command "$file" > "$file2" && mv "$file2" "$file"
   

Regarding your specific question: prepending 'blah/blah' to the file, you can use sed if it supports the -i option. Take GNU sed as an example:

sed '1i\
blah/blah' -i "$file"

Answer 2

It means exactly what it says in the most literal sense possible. What this command is actually doing is overwriting the file $tmpdir/$filename.jpdf with only the single line being echo'ed. The shell will see the redirect, then truncate that file and open it for writing, and only then will it begin to perform the commands to generate its (new) contents. What you should do instead is write the new contents to a different file, then copy the new file over to the old name in a second command.

Answer 3

Lets type an equivalent simpler command line:

cat file > file

If file contains data it will be erased, lets try:

$ echo "content of file" > file
$ cat file
content of file

$ cat file > file

$ cat file
                     # empty file.

The shell process redirections before executing the command.
On processing >file, it opens file for writing truncating it.
After that, the file is empty.
The command cat does the cat of an empty file, nothing gets written.