cat package.yaml > package.yaml

The above will truncate package.yaml, and hence the file will be empty - cat will have empty output.

How can I avoid truncating the file? Is there an alternative to using a temporary file? A variable maybe?

  • 1
    What are you trying to accomplish?
    – jesse_b
    Feb 26 '18 at 18:17
  • Sorry, the command is a simplified useless case. Essentially I'm modifying the file and would like to 'update' it. Feb 26 '18 at 18:18
  • 2
  • In this case it would help to know how you are modifying it because some text processing programs (like sed) can modify files in place natively.
    – jesse_b
    Feb 26 '18 at 18:19
  • A variable potentially could be used but is certainly not going to be the best or safest option: myvar=$(sed 's/my_string/new_string/' input) && echo "$myvar" > input
    – jesse_b
    Feb 26 '18 at 18:22

All utilities that allows for in-place editing of files does so by internally writing the result to a temporary file and then replacing the original file by this temporary file once the operation is complete.

Files can be modified in place, but you would only be able to overwrite existing data and/or extend the length of the file. This may be done using the dd utility, for example:

$ cat file.txt
hello world
abc abc 123 123

$ cat insert.txt

$ dd if=insert.txt of=file.txt bs=1 seek=6 conv=notrunc
7+0 records in
7+0 records out 
7 bytes transferred in 0.000 secs (30918 bytes/sec)

$ cat file.txt
hello hello!
bc abc 123 123

Here, we insert what's in insert.txt into file.txt by first seeking 6 bytes forward in the file (stepping past hello and the space) and then modifying it. The conv=notrunc stops the output file from being truncated at the end of the writing operation.

If if=insert.txt was left out, it would have been possible to insert any text from the keyboard. Note that "inserting" is the wrong term here really. "Overwriting" may better describe what's happening (see the first character of the second line being overwritten by the newline at the end of insert.txt).

You wouldn't want to edit a file this way though.

  • Regarding your first sentence: there are other strategies. What you describes is the best strategy in terms of safety: if something fails, the original is untouched. But many programs do backup-then-overwrite, which results in a truncated file if interrupted but has the advantage of preserving permissins. And some dangerous programs do read-then-overwrite, which saves disk space but your data disappears if anything fails. Feb 27 '18 at 7:22
  • Different text editors have different strategies to edit files (without having to store the whole content in memory for some), but few if any take the approach you describe. What you describe sounds more like perl -pi (also sed -i, gawk -v inplace) but is not the norm. See also the <> redirection operator, $mapfile in zsh, moreutils sponge, mmap()... Mar 4 '18 at 20:08
  • Some systems do support inserting data into file (see fallocate(FALLOC_FL_INSERT_RANGE) on Linux for instance, though that's only in multiples of the filesystem block size). Mar 4 '18 at 20:10

Not the answer you're looking for? Browse other questions tagged or ask your own question.