9

In bash, I notice that if a command using redirection would fail, any programs which run prior to that are not run.

For example, this program opens the file "a" and writes 50 bytes to file "a". However, running this command with redirection to a file with insufficient permissions (~root/log), yields no change in file size of "a".

$ ./write_file.py >> ~root/log
-bash: /var/root/log: Permission denied
cdal at Mac in ~/experimental/unix_write
$ ls -lt
total 16
-rw-rw-r--  1 cdal  staff  0 Apr 27 08:54 a <-- SHOULD BE 50 BYTES

One would think the program would run, capture any output (but also write to the file "a"), and then fail to write any output to ~root/log. Instead the program is never run.

Why is this, and how does bash choose the order of the "checks" it performs prior to executing a program? Are other checks performed as well?

p.s. I'm trying to determine whether a program run under cron actually ran when redirected to a "permission denied" file.

  • Everything is in good working order (i.e. ownerships and permissions of your .py file that is) your program executes fine. Your problems comes from the redirect. You do not have permission to write a filein /root directory. And you have redirected your stdout to do exactly that. So, you will not see any output, even though your program ran. – MelBurslan Apr 27 '16 at 13:47
  • 2
    Mel, that's not true, the program never actually ran. See answers below. – Charlie Dalsass Apr 27 '16 at 13:53
  • You: "Run the write_file.py program and send its output to ~root/log bash: "Sorry, but you're not allowed to write to that file!" The shell is doing exactly what it should do. If it can't do what you asked it to do, it immediately informs you why there's a problem, giving you the opportunity to decide how to deal with it. For all the bash maintainers know, Very Bad Things could happen if you run that command and don't get to save the output. If it was important enough you designated a place to save it, it would be wrong to ASS|U|ME it was OK to run without saving stdout. – Monty Harder Apr 27 '16 at 20:14
18

It's not really a question of ordering checks, simply the order in which the shell sets things up. Redirections are set up before the command is run; so in your example, the shell tries to open ~root/log for appending before trying to do anything involving ./write_file.py. Since the log file can't be opened, the redirection fails and the shell stops processing the command line at that point.

One way to demonstrate this is to take a non-executable file and attempt to run it:

$ touch demo
$ ./demo
zsh: permission denied: ./demo
$ ./demo > ~root/log
zsh: permission denied: /root/log

This shows that the shell doesn't even look at ./demo when the redirection can't be set up.

  • Wow, it's that simple? I didn't realize that redirections were done first. Thanks for this answer and for other great answers as well. – Charlie Dalsass Apr 27 '16 at 13:49
  • 6
    If they weren't done first, where would the output be written to? – Charles Duffy Apr 27 '16 at 18:38
  • And if the output can't be written, how do we know it's safe to run the command? Maybe the command outputs information that is being deleted from a data store, and it's absolutely necessary that the output be captured. Good thing bash won't let it run until you fix those permissions, eh? – Monty Harder Apr 27 '16 at 20:17
11

From the bash man page, section REDIRECTION (emphasis by me):

Before a command is executed, its input and output may be redirected using a special notation interpreted by the shell.

...

A failure to open or create a file causes the redirection to fail.

So the shell tries to open the target file for stdout, which fails, and the command isn't executed at all.

  • Thanks so much. I wish the man page clarified a bit with "... if the output cannot be redirected, the program will not be executed". – Charlie Dalsass Apr 27 '16 at 13:58
  • Updated; it's rather hidden some paragraphs below. – Murphy Apr 27 '16 at 14:04
  • Actually, it's pretty clear. "A failure to open or create a file causes the redirection to fail." There it is. Thanks again. – Charlie Dalsass Apr 27 '16 at 14:13
3

It's worth observing that the shell must establish redirections before starting the program.

Consider your example:

./write_file.py >> ~root/log

What happens in the shell is:

  1. We (the shell) fork(); the child process inherits the open file descriptors from its parent (the shell).
  2. In the child process, we fopen() (the expansion of) "~root/log", and dup2() it to fd 1 (and close() the temporary fd). If the fopen() fails, call exit() to report the error to the parent.
  3. Still in the child, we exec() "./write_file.py". This process is now no longer running any of our code (unless we failed to execute, in which case we exit() to report the error to the parent).
  4. The parent will wait() for the child to terminate, and handle its exit code (by copying it into $?, at least).

So the redirection has to happen in the child between fork() and exec(): it can't happen before fork() because it must not change the shell's stdout, and it can't happen after exec() because the filename and the shell's executable code have now been replaced by the Python program. The parent has no access to the file descriptors of the child (and even if it did, it couldn't guarantee to redirect between exec() and the first write to stdout).

0

I'm sorry to inform you that it is quite the opposite. The shell needs to open it's I/O first and then passes control to the program.

tee might prove helpful in this case: ./write_file.py | tee -a ~root/log > /dev/null

  • Won't the Python script just die on SIGPIPE after tee fails? – Kevin Apr 27 '16 at 19:36
  • Not according to the test I did but I do suggest you try it. – Julie Pelletier Apr 27 '16 at 19:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.