2

errexit (set -e) is often suggested as a way to make simple scripts more robust. However, it is generally regarded as a horrible trap by anyone who has seen how it behaves in more complex scripts, in particular with functions. And a very similar problem can be seen with subshells. Take the following example, adapted from https://stackoverflow.com/questions/29926013/exit-subshell-on-error

(
set -o errexit
false
true
) && echo "OK" || echo "FAILED";

The trap here is that shells show "OK", and not "FAILED".[*]

While the behaviour of set -e in specific cases has historically varied in unfortunate ways, shells available on a modern distribution claim to follow the POSIX shell standard, and testing showed the same behaviour ("OK") for all of the following:

  • bash-4.4.19-2.fc28.x86_64 (Fedora)
  • busybox-1.26.2-3.fc27.x86_64 (Fedora)
  • dash 0.5.8-2.4 (Debian 9)
  • zsh 5.3.1-4+b2 (Debian 9)
  • posh 0.12.6+b1 (Debian 9)
  • ksh 93u+20120801-3.1 (Debian 9).

Question

  1. Is there any technical reason why you can't write a shell with similar features to POSIX sh, except that it prints FAILED for the above code?

    I hope that it would also change set -e so that top-level && expressions which return false are considered fatal. https://serverfault.com/a/847016/133475 And hopefully it would have some nicer idiom to use with grep.

    set -e
    
    # print matching lines, but it's not an error if there are no matches in the file
    (
    set +e
    grep pattern file.txt
    RET=$?
    [[ $RET == 0 || $RET == 1 ]] || exit $RET
    )
    

    I guess I also assume arithmetic statements (let builtin) would be redefined in some way that avoids implicitly coercing their numeric value to a true/false exit status.

  2. Is there an example of a shell which does any of this?

    I don't mind looking at something with a different syntax from Bourne. I'm interested in something which is compact, when writing short "glue" scripts, but which also has a compact strategy for detecting errors. I don't like using && as a statement separator, if it means accidentally omitting a statement separator will cause errors to be silently ignored.


[*] EDIT. This is perhaps not a perfect example to use. Discussion suggests that a better example would be to move the set -o errexit above the subshell.

AFAICT this is very similar to the test case (false; echo foo) || echo bar; echo \ $? from the table here, which says that it would show "foo" (and then "0") on the original Bourne shell and most of its descendents. The exceptions are "hist.ash" and bash 1.14.7.

When you add set -e inside the subshell - (set -e; false; echo foo) || echo bar; echo \ $? - there are two additional exceptions, "SVR4 sh sun5.10" and dash-0.3.4.

For the spirit of my question, I suppose having set -e inside the subshell was a distraction. My main interest is in an idiom where you use set -e at the top of the script, and it also applies to subshells (which is already the POSIX behaviour).

Jörg Schilling suggests that the Bourne shell from original Unix would print "FAILED" for the example I used in this question, that he has ported this shell to POSIX as "osh" in schilytools, and that this result has been verified as of release 2018-06-11. Perhaps this is based on 1) set -e being inside the subshell and 2) "SVR4 sh sun5.10". "osh" is "based on the OpenSolaris sources and thus based on SVR4 and SVID3". Or perhaps there is some horrifying additional variance caused by adding && echo "OK" in between the subshell and || echo "FAILED".

I don't think the schily shell answers my question. You could use subshells for functions (use ( in place of { to run the function in a subshell), and start every subshell with set -e. However, using subshells introduces the limitation that you cannot modify global variables. At least, I have yet to see anyone defend this as a general purpose coding style.

  • 1
    My interpretation of the POSIX standard is that the first script should print FAILED. ksh93 however prints OK. – schily Jun 14 '18 at 16:35
  • @schily based on my reading of stackexchange posts, I think you would be in the minority there :). I've edited to include tested versions of bash and busybox. ksh93 doesn't look like a specific individual version to my modern ears. I'm interested in any indications for specific shell versions, whether explicitly bourne-compatible or not, it would be relevant background to the question in my mind. – sourcejedi Jun 14 '18 at 17:47
  • 2
    An impression from the gut does not help here and opinions from others don't do as well. You would need to reason your statements. I would not take bash as an example here since all versions of bash including bash-3.x handled set -e completely wrong. It took me and David Korn two months to convince the bash maintainer to delay bash-4.0 and to fix it before. Bash was not usable bymake that calls sh -ce cmd. At the same time, we tried to make the POSIX standard text more obvious. As a rule of thumb: POSIX does not make historic UNIX programs wrong unless there is a really good reason. – schily Jun 14 '18 at 18:25
  • 1
    Since I know that I did not change this behavior, I grant this for all releases including 2018-06-11. Since you mention that while false; do ...; done should not exit at false this just what the Bourne Shell did since 1978. The problem with your first example is that the false command is placed in a way that it's exit code is not evaluated. This results in an exit of the related subshell if set -e is in effect for that shell. – schily Jun 15 '18 at 12:21
  • 1
    I wouldn't call them POSIX shells. There are two shells that passed certification: 1) the ksh88 that was modified by Sun, HP and IBM for compliance. This shell now is in conflict with some modifications in the standard that appeared after 2008. 2) a bash-3.x that was hacked by Apple for passing the tests. This bash e.g. has a corrected echo (like the one on Solaris) and it passed the set -e tests because these tests did not check for the known bash-3.0 deviations. A similar problem in the POSIX test suite let Mac OS slip through even though it comes with a non-compliant waitid()syscall. – schily Jun 15 '18 at 14:17
1

The POSIX standard (c.f. -e) is clearer than the Bash manual about the errexit shell option.

When this option is on, when any command fails (for any of the reasons listed in Consequences of Shell Errors or by returning an exit status greater than zero), the shell immediately shall exit, as if by executing the exit special built-in utility with no arguments, with the following exceptions:

  1. The failure of any individual command in a multi-command pipeline shall not cause the shell to exit. Only the failure of the pipeline itself shall be considered.

  2. The -e setting shall be ignored when executing the compound list following the while, until, if, or elif reserved word, a pipeline beginning with the ! reserved word, or any command of an AND-OR list other than the last.

  3. If the exit status of a compound command other than a subshell command was the result of a failure while -e was being ignored, then -e shall not apply to this command.

This requirement applies to the shell environment and each subshell environment separately. For example, in:

set -e; (false; echo one) | cat; echo two

the false command causes the subshell to exit without executing echo one; however, echo two is executed because the exit status of the pipeline (false; echo one) | cat is zero.

Clearly, the sample code is equivalent to the following pseudocode.

( LIST; ) && COMMAND || COMMAND

The exit status of the list is zero because:

  • the errexit shell option is ignored.

  • the return status is the exit status of the last command specified in the list, here true.

Therefore, the second part of the AND list is executed: echo "OK".

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.