In bash, how is it that a file redirected for input can be written to, and can it be prevented?

Question

I thought I had a good handle on bash file redirection, and generally I try to avoid "useless use of cat", but I experienced some unexpected behaviour with a script and I would like to understand why it occurs.

Within a bash script, I execute:

somecommand < file1 > file2

My expectation was that file1 is safe and opened in a read-only manner. In practice, I found that file1 can be overwritten. How/why does this happen, and is there a way to prevent it without resorting to a cat?

If it's working how I imagine (the process ends up with a direct rw file descriptor?), it seems like it should be considered dangerous to redirect files this way, yet I've never seen this behaviour mentioned before.

To add some specifics from my case: the command in question is sops, which in the background is doing some GPG stuff. The GPG password prompt is sometimes^† being written to the file used for input, overwriting it. The complete command I used is:

sops --input-type json --output-type json -d /dev/stdin < ./secrets/file.json > ./secrets/file-decrypted.json

I have since switched to cat file1 | sops.. > file2 and everything works as expected. I would have said this was a "useless use of cat" - but it doesn't seem so useless anymore!

---

_{^†It seems to be when gpg-agent is not running and prompts for the first time.}

Answer 1

That's due to the way /dev/stdin (actually /proc/self/fd/0) is implemented on Linux (and Cygwin, but generally not other systems).

On Linux opening /dev/stdin is not like doing a dup(0), it just reopens the same file as open on fd 0 anew. It doesn't share the open file description that fd 0 refers to (with the readonly mode), but gets a completely unrelated new open file description, with the mode as specified in open().

So if sops -d /dev/stdin opens /dev/stdin in read+write mode and fd 0 was open in read-only on /some/file, /some/file will be open in read+write.

Effectively, cmd /dev/stdin < file there is the same as cmd file < file. You'll find that /dev/stdin is just a symlink¹ to file:

/tmp$ namei -l /dev/stdin < file
f: /dev/stdin
drwxr-xr-x root     root     /
drwxr-xr-x root     root     dev
lrwxrwxrwx root     root     stdin -> /proc/self/fd/0
drwxr-xr-x root     root       /
dr-xr-xr-x root     root       proc
lrwxrwxrwx root     root       self -> 73569
dr-xr-xr-x stephane stephane     73569
dr-x------ stephane stephane   fd
lr-x------ stephane stephane   0 -> /tmp/file
drwxr-xr-x root     root         /
drwxrwxrwt root     root         tmp
-rw-r--r-- stephane stephane     file

It can get worse. If it was opening with O_TRUNC, the file would be truncated. If fd 0 was pointing to the reading end of a pipe and /dev/stdin was open in write-only mode, you'd get the other end of the pipe.

But using:

cat file | cmd /dev/stdin

Would guard against cmd overwriting file as all cmd would see would be the pipe. And even if it did open in write-only mode, it couldn't get back to the file, it would just get to the writing end of the pipe and the only file descriptor on the reading end would be cmd's stdin.

Other OSes don't have the problem as opening /dev/stdin there is like doing a dup(0), so you get the same open file description and if you open with an incompatible mode, the open() system call just fails.

---

^{¹ technically, as noted by @user414777 in comments, /proc/<pid>/fd/<fd> are magic symlinks in that for instance they can reach into places that normal symlinks could not, but when it comes to opening them, past the path resolution stage, they act like normal symlinks in that you just open the target file}