18

I want my shell scripts to fail whenever a command executed with them fails.

Typically I do that with:

set -e
set -o pipefail

(typically I add set -u also)

The thing is that none of the above works with process substitution. This code prints "ok" and exit with return code = 0, while I would like it to fail:

#!/bin/bash -e
set -o pipefail
cat <(false) <(echo ok)

Is there anything equivalent to "pipefail" but for process substitution? Any other way to passing to a command the output of commands as it they were files, but raising an error whenever any of those programs fails?

A poor's man solution would be detecting if those commands write to stderr (but some commands write to stderr in sucessful scenarios).

Another more posix compliant solution would be using named pipes, but I need to lauch those commands-that-use-process-substitution as oneliners built on the fly from compiled code, and creating named pipes would complicate things (extra commands, trapping error for deleting them, etc.)

  • You don't need to trap errors to delete named pipes. In fact, that's not a good way at all. Consider mkfifo pipe; { rm pipe; cat <file; } >pipe. That command will hang until a reader opens pipe because it is the shell which does the open() and so as soon as there is a reader on pipe the fs link for pipe is rm'd and then cat copies infile out to the shell's descriptor for that pipe. And anyway, if you want to propagate an error out of a process sub do : <( ! : || kill -2 "$$") – mikeserv Jul 22 '15 at 12:36
  • Thanks for the tipe on deleting named pipes. Unfortunately, the $$ substitution does not work for me, since ths command substitution is not done as command that uses process substitution is done inside a command pipeline that is spawned from a "non shell" code (python). Probably I should create subprocess in python and pipe them programatically. – juanleon Jul 22 '15 at 14:19
  • So use kill -2 0. – mikeserv Jul 22 '15 at 14:19
  • Unfortunately "kill -2 0" would kill (signal) python. Writing signal handlers for performing business logic within a multithread application is not something I am looking forward to :-) – juanleon Jul 22 '15 at 15:01
  • If you don't want to handle signals then why are you trying to receive them? Anyway, I expect named pipes would be the most simple solution in the end, if only because doing it right requires laying out your own framework and setting up your own tailored dispatcher. Get over that hurdle and it all comes together. – mikeserv Jul 22 '15 at 15:13
6

You could only work around that issue with that for example:

cat <(false || kill $$) <(echo ok)
other_command

The subshell of the script is SIGTERMd before the second command can be executed (other_command). The echo ok command is executed "sometimes": The problem is that process substitutions are asynchronous. There's no guarantee that the kill $$ command is executed before or after the echo ok command. It's a matter of the operating systems scheduling.

Consider a bash script like this:

#!/bin/bash
set -e
set -o pipefail
cat <(echo pre) <(false || kill $$) <(echo post)
echo "you will never see this"

The output of that script can be:

$ ./script
Terminated
$ echo $?
143           # it's 128 + 15 (signal number of SIGTERM)

Or:

$ ./script
Terminated
$ pre
post

$ echo $?
143

You can try it and after a few tries, you will see the two different orders in the output. In the first one the script was terminated before the other two echo commands could write to the file descriptor. In the second one the false or the kill command were probably scheduled after the echo commands.

Or to be more precisely: The system call signal() of the kill utillity that sends the the SIGTERM signal to the shells process was scheduled (or was delivered) later or earlier than the echo write() syscalls.

But however, the script stops and the exit code is not 0. It should therefore solve your issue.

Another solution is, of course, to use named pipes for this. But, it depends on your script how complex it would be to implement named pipes or the workaround above.

References:

2

For the record, and even if the answers and comments were good and helpful, I ended implementing something a little different (I had some restrictions about receiving signals in the parent process that I did not mention in the question)

Basically, I ended doing something like this:

command <(subcomand 2>error_file && rm error_file) <(....) ...

Then I check the error file. If it exists, I know what subcommand failed (and the contents of the error_file can be useful). More verbose and hackish that I originally wanted, but less cumbersome than creating named pipes in a one-liner bash commmand.

  • Whatever works for you is usually the best way. Thanks especially for coming back and answering - selfies are my favorites. – mikeserv Jul 23 '15 at 10:35
1

In a similar way as you would implement $PIPESTATUS/$pipestatus with a POSIX shell that doesn't support it, you can obtain the exit status of the commands by passing them around via a pipe:

unset -v false_status echo_status
{ code=$(
    exec 3>&1 >&4 4>&-
    cat 3>&- <(false 3>&-; echo >&3 "false_status=$?") \
             <(echo ok 3>&-; echo >&3 "echo_status=$?")
);} 4>&1
cat_status=$?
eval "$code"
printf '%s_code=%d\n' cat   "$cat_status" \
                      false "$false_status" \
                      echo  "$echo_status"

Which gives:

ok
cat_code=0
false_code=1
echo_code=0

Or you could use pipefail and implement process substitution by hand like you would with shells that don't support it:

set -o pipefail
{
  false <&5 5<&- | {
    echo OK <&5 5<&- | {
      cat /dev/fd/3 /dev/fd/4
    } 4<&0 <&5 5<&-
  } 3<&0
} 5<&0
1

This example shows how to use kill together with trap.

#! /bin/bash
failure ()
{
  echo 'sub process failed' >&2
  exit 1
}
trap failure SIGUSR1
cat < <( false || kill -SIGUSR1 $$ )

But kill can not pass a return code from your sub process to the parent process.

  • I like this variation on the accepted answer – sehe Apr 1 at 11:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.