Bash program not executed if redirection would fail

Question

In bash, I notice that if a command using redirection would fail, any programs which run prior to that are not run.

For example, this program opens the file "a" and writes 50 bytes to file "a". However, running this command with redirection to a file with insufficient permissions (~root/log), yields no change in file size of "a".

$ ./write_file.py >> ~root/log
-bash: /var/root/log: Permission denied
cdal at Mac in ~/experimental/unix_write
$ ls -lt
total 16
-rw-rw-r--  1 cdal  staff  0 Apr 27 08:54 a <-- SHOULD BE 50 BYTES

One would think the program would run, capture any output (but also write to the file "a"), and then fail to write any output to ~root/log. Instead the program is never run.

Why is this, and how does bash choose the order of the "checks" it performs prior to executing a program? Are other checks performed as well?

p.s. I'm trying to determine whether a program run under cron actually ran when redirected to a "permission denied" file.

Answer 1

It's not really a question of ordering checks, simply the order in which the shell sets things up. Redirections are set up before the command is run; so in your example, the shell tries to open ~root/log for appending before trying to do anything involving ./write_file.py. Since the log file can't be opened, the redirection fails and the shell stops processing the command line at that point.

One way to demonstrate this is to take a non-executable file and attempt to run it:

$ touch demo
$ ./demo
zsh: permission denied: ./demo
$ ./demo > ~root/log
zsh: permission denied: /root/log

This shows that the shell doesn't even look at ./demo when the redirection can't be set up.

Answer 2

From the bash man page, section REDIRECTION (emphasis by me):

Before a command is executed, its input and output may be redirected using a special notation interpreted by the shell.

...

A failure to open or create a file causes the redirection to fail.

So the shell tries to open the target file for stdout, which fails, and the command isn't executed at all.

Answer 3

It's worth observing that the shell must establish redirections before starting the program.

Consider your example:

./write_file.py >> ~root/log

What happens in the shell is:

1. We (the shell) fork(); the child process inherits the open file descriptors from its parent (the shell).
2. In the child process, we fopen() (the expansion of) "~root/log", and dup2() it to fd 1 (and close() the temporary fd). If the fopen() fails, call exit() to report the error to the parent.
3. Still in the child, we exec() "./write_file.py". This process is now no longer running any of our code (unless we failed to execute, in which case we exit() to report the error to the parent).
4. The parent will wait() for the child to terminate, and handle its exit code (by copying it into $?, at least).

So the redirection has to happen in the child between fork() and exec(): it can't happen before fork() because it must not change the shell's stdout, and it can't happen after exec() because the filename and the shell's executable code have now been replaced by the Python program. The parent has no access to the file descriptors of the child (and even if it did, it couldn't guarantee to redirect between exec() and the first write to stdout).

Answer 4

I'm sorry to inform you that it is quite the opposite. The shell needs to open it's I/O first and then passes control to the program.

tee might prove helpful in this case: ./write_file.py | tee -a ~root/log > /dev/null