>> in a POSIX shell guarantees that the file will be opened with
The Open Group Base Specifications Issue 7 states:
O_APPEND flag of the file status flags is set, the file offset shall be set to the end of the file prior to each write and no intervening file modification operation shall occur between changing the file offset and the write operation.
POSIX defines the mimimum number of bytes that can be requested to be written in a single
write(2) call (
SSIZE_MAX = 32,767). The return value is the number of bytes actually written (guaranteed atomic).
Not all filesystems comply. Appending to a File from Multiple Processes says:
The caveat is that not all filesystems are POSIX-compatible. Two famous examples are NFS and the Hadoop Distributed File System (HDFS). On these networked filesystems, appends are simulated and subject to race conditions.
Not everyone plays nice
While you might open using
O_APPEND, the other processes writing to the file may not. You can check for any given file:
lsof +fg <file>
Worryingly, when I run
lsof +fg ~/.xsession-errors, I see no
AP (append) flag, suggesting that the 22 processes in my listing (Manjaro Linux based on Arch) are not opening the file safely.
Only when I run
cat >> ~/.xsession-errors in another shell does the final output line include the
cat 3099 ravi 1w REG W,AP,LG 0,48 963 1926479 .xsession-errors
If anyone knows where this issue should be raised upstream, please comment.
If all processes open the file locally with either:
open(2) and flag
fopen(3) and the
Then no data should be overwritten via race conditions.
Thanks to @Gilles answer for pointing me in the right direction.