5

My bash knowledge is a bit rusty (and hasn't been very solid before as well), so I seem to be unable to find an answer to the following question:

As the heading says, I'd like to know how I can determine whether a non-zero exit code after command execution has been set by bash (meaning a real error) or by the command (possibly indicating an error, dependent on the command and my purpose).

For example, let's have a look at the following very simple script:

#!/bin/bash

string='abc'
grep 'd' <<< "$string"
echo $?

This outputs 1, which is expected after having read grep's manual (excerpt, shortening mine):

EXIT STATUS
Normally the exit status is 0 if a line is selected, 1 if no lines were selected, and 2 if an error occurred. [...]

After having read the respective section of bash's manual, I am having a problem (excerpt, shortening and emphasis mine):

EXIT STATUS

[...]

If a command is not found, the child process created to execute it returns a status of 127. If a command is found but is not executable, the return status is 126.

If a command fails because of an error during expansion or redirection, the exit status is greater than zero.

Shell builtin commands return a status of 0 (true) if successful, and non-zero (false) if an error occurs while they execute. All builtins return an exit status of 2 to indicate incorrect usage, generally invalid options or missing arguments.

[...]

My problem is the emphasized statement.

My scripts generally need to treat real errors (like lack of permissions, needed programs not being available, resource exhaustion etc.) specially, but in my example above, it is not an error in the above-mentioned sense when grep does not select a line; instead, it just means that its input did not contain a matching character sequence.

However, if I take the section from bash's manual literally, it could be bash itself which may have set the exit status of 1. From that section, we know what happens if a command could not be found (exit status 127) or is not executable (exit status 126).

The next statement in that section, as I understand it, means that every other error can be mapped to any exit status in the inclusive range [1, 255] by bash. Notably, it can be mapped to exit status 1. I am considering that a major problem because I believe that there is a vast number of errors besides "command not found" or "command not executable". For example, command execution could be prevented by memory exhaustion, file handle exhaustion, timeouts due to disk read errors, and so on.

In contrast to the "grep could not find a matching line" error, these are real severe errors which mostly must cause an email being sent to the administrator for immediate action.

But now it seems that I can't differentiate between the two sorts of errors (non-zero exit status set by executed command vs. non-zero exit status set by bash after having tried to execute the command).

Could anybody point me to a reasonable solution?

Similar questions

During my research, I have come across a lot of similar questions. However, to my best understanding, nobody had the exact same problem.

Instead, most people just wanted to suppress a non-zero exit code returned by a command (applied to my example, they would have wanted to have exit status 0 instead of 1 when grep did not select a line), and were given a solution similar to command || true.

While this might be acceptable for them, it is not a solution to me, because it also would suppress the real errors mentioned above. For example, consider the following:

root@cerberus:~/scripts# { ThisProgramDoesNotExist 2>/dev/null || true; } && { echo "Gotcha!"; }
Gotcha!
root@cerberus:~/scripts#

This demonstrates how that solution suppresses not only non-zero exit statuses (or is it "stati"?) from an executed command, but also severe errors reported by bash when failing to start a command. This is a no-go in most of my scripts.

2
  • I find your question somewhat hard to understand because the example you give is SO artificial and contrived, I don’t know what your real objective is.  Specifically, I can’t think of any way for grep 'd' <<< "$string" to fail at the bash level.  cp f1 f2; grep Cheshire < f2 might be a better example, where a command failure in the cp command can cause a bash failure that prevents the grep from running. And Gilles’s answer seems to be trying to give you ways to run the grep even if the redirection fails.  I don’t think you want that, but I can’t tell from reading your question. – G-Man Says 'Reinstate Monica' Sep 20 '20 at 5:42
  • (Cont’d) …  See also How is the return status of a variable assignment determined? Disclosure: I am the author of the accepted answer to that question. It doesn’t answer your question per se, but it contains a lot of information about how $? is calculated in commands that run multiple programs, like foo $(bar) and FOO=$(bar) baz. It was a great exercise in collaboration — I got a couple of helpful edits and many intelligent, constructive comments. – G-Man Says 'Reinstate Monica' Sep 20 '20 at 5:42
1

You can't tell. All you get is a single value between 0 and 255, which is 0 if everything went well and nonzero otherwise.

If you want to treat some nonzero statuses as successes, be sure that the command in question can't fail for other reasons such as a redirection. Break up the command so that different kinds of failures happen in different commands or result in different statuses.

For example, if you need to know whether an error comes from a redirection, either do the redirection in a separate command, or do it separately over a block.

Combined status:

mycommand <foo
status=$?
if [ $status -ne 0 ]; then echo "Either mycommand failed or <foo failed"; fi

Separate statuses, but no way to avoid running the command if the redirection fails:

{
  mycommand
  command_status=$?
} <foo
redirection_status=$?
if [ $command_status -ne 0 ]; then echo "mycommand failed"; fi
if [ $redirection_status -ne 0 ]; then echo "<foo failed"; fi

Do the redirection first. Note that being able to react to the failure of the redirection in this way is a bash feature. POSIX shells, including bash in POSIX mode, exit if a redirection on the exec builtin fails.

exec 3<&1         # Save stdin to file descriptor 3
exec <foo         # bash keeps going if the redirection fails
redirection_status=$?
mycommand
command_status=$?
exec <&3          # Restore stdin
if [ $command_status -ne 0 ]; then echo "mycommand failed"; fi
if [ $redirection_status -ne 0 ]; then echo "<foo failed"; fi

Do the redirection fails, in a subshell to contain the failure of the redirection and not have to restore the file descriptor afterwards.

(
  exec <foo || exit $?     # In POSIX sh, "|| exit $?" is redundant.
  mycommand
  command_status=$?
  if [ $command_status -ne 0 ]; then echo "mycommand failed"; fi
)
redirection_status=$?
if [ $redirection_status -ne 0 ]; then echo "<foo failed and mycommand didn't run"; fi

If you need to know whether an error comes from some other expansion, do the expansion separately and save its result.

Saving one argument: instead of `mycommand "$(…)", save the result of the expansion first.

foo=$(…) && mycommand "$foo"

More generally:

foo=$(…)
command_substitution_status=$?
mycommand "$foo"
mycommand_status=$?

Note that if an assignment contains multiple command substitutions, its status is the status of the last substitution: the status is 0 if the last substitution succeeds, even if earlier ones failed.

foo=$(…)
foo_status=$?
bar=$(…)
bar_status=$?
mycommand "$foo" "$bar"
mycommand_status=$?

To save multiple arguments, use an array, or the positional parameters inside a function.

args=()
foo=$(…)
foo_status=$?
args+=(-x "$foo")
bar=$(…)
bar_status=$?
args+=(-y "$bar")
mycommand "${args[@]}"
3
  • Brilliant, as usual, but I’m confused.  I really like your { / mycommand / command_status=$? / } <foo trick, but I’m puzzled by the statement “no way to avoid running the command if the redirection fails.”  It seems to me that, if < foo fails, then mycommand doesn’t run and command_status isn’t set.  … (Cont’d) – G-Man Says 'Reinstate Monica' Sep 20 '20 at 5:44
  • (Cont’d) …  And therefore, you should set command_status to some impossible value (like -0 or null) before the {. If that hasn’t changed, then you know that the redirection failed and you didn’t get inside the block. If command_status is changed from its ‘impossible’ value, then you know that the redirection succeeded, you got inside the block, and command_status is the exit status from mycommand. … … P.S. In the foo=$(…) case, ISTM that (perhaps) the user might want to consider running mycommand only if foo_status is 0 (and so you should give an example of that).  … (Cont’d) – G-Man Says 'Reinstate Monica' Sep 20 '20 at 5:44
  • (Cont’d) …  Also, your answer does not seem to differentiate between Case E: grep calls exit(127), and Case F: exec("/bin/grep",…) fails, and the shell sets the status to 127 — and this seems to be precisely the sort of thing that the OP is concerned with.  Admittedly, this is a hard test to do in a thorough, reliable and portable manner.  OK, I guess [ -x "$(type -p "grep")" ] is a good start, but that doesn’t handle the case where the “executable” file has a shebang that points to an invalid interpreter, or it contains binary data that don’t look like a valid binary executable. – G-Man Says 'Reinstate Monica' Sep 20 '20 at 5:44
1

I am not aware of any approach for this differentiation after running the command. But if you accept a race condition then there may be a sufficient option for you.

Instead of

cmd with "$params" and </re/di/rections

you do

if : with "$params" and </re/di/rections; then
   # expansion and redirections are OK
   # let's hope no redirection-relevant paths are deleted or created in the meantime
   # and, of course, this does not work well with noclobber
   cmd with "$params" and </re/di/rections
else
   : error "outside" the command
fi
2
  • You mention race conditions, which are problems arising from improper sequencing of asynchronous, concurrent events.  I suggest that side effects are more important here.  For example: (1) A parameter of $(date | tee -a log) will modify the log file, so you shouldn’t evaluate it more than once.  (2) Similarly, $(tar cvf feathers.tar *) will create the feathers.tar file even if the directory is empty.  Run it again and * will expand to feathers.tar.  … (Cont’d) – G-Man Says 'Reinstate Monica' Sep 20 '20 at 5:47
  • (Cont’d) …  (3) Trivially, redirections have side effects.  > can create a file, so echo $(stat -c%s quux) > quux will get different results the first and second times you run it.  (OK, I just noticed that you allude to this issue in a comment in your code block.)  (4) More subtly, opening a named pipe or a file-system socket may wake up another process, even if you don’t do any I/O. – G-Man Says 'Reinstate Monica' Sep 20 '20 at 5:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.