3 another typo fix
source | link
  • A tempfile is created with mktemp. This usually immediately creates a file in /tmp
  • This tempfile is then redirected to FD 9 for write and FD 8 for read
  • Then the tempfile is immediately deleted. It stays open, though, until both FDs go out of existence.
  • Now the pipe is started. Each step adds to FD42FD 9 only, if there was an error.
  • The wait is needed for ksh, because ksh else does not wait for all pipe commands to finish. However please note that there are unwanted sideffects if some background tasks are present, so I commented it out by default. If the wait does not hurt, you can comment it in.
  • Afterwards the file's contents are read. If it is empty (because all worked) read returns false, so true indicates an error
  • ksh and similar shells which only wait for the last pipe command need the wait uncommented

  • The last example uses printf "%1s" "$?" instead of echo -n "$?" because this is more portable. Not every platform interprets -n correctly.

  • printf "%"$?" would do it as well, however printf "%1s" catches some corner cases in case you run the script on some really broken platform. (Read: if you happen to program in paranoia_mode=extreme.)

  • FD 8 and FD 9 can be higher on platforms which support multiple digits. AFAIR a POSIX conformant shell does only need to support single digits.

  • Was tested with Debian 8.2 sh, bash, ksh, ash, sash and even csh

  • A tempfile is created with mktemp. This usually immediately creates a file in /tmp
  • This tempfile is then redirected to FD 9 for write and FD 8 for read
  • Then the tempfile is immediately deleted. It stays open, though, until both FDs go out of existence.
  • Now the pipe is started. Each step adds to FD42 only, if there was an error.
  • The wait is needed for ksh, because ksh else does not wait for all pipe commands to finish. However please note that there are unwanted sideffects if some background tasks are present, so I commented it out by default. If the wait does not hurt, you can comment it in.
  • Afterwards the file's contents are read. If it is empty (because all worked) read returns false, so true indicates an error
  • ksh and similar shells which only wait for the last pipe command need the wait uncommented

  • The last example uses printf "%1s" "$?" instead of echo -n "$?" because this is more portable. Not every platform interprets -n correctly.

  • printf "%?" would do it as well, however printf "%1s" catches some corner cases in case you run the script on some really broken platform. (Read: if you happen to program in paranoia_mode=extreme.)

  • FD 8 and FD 9 can be higher on platforms which support multiple digits. AFAIR a POSIX conformant shell does only need to support single digits.

  • Was tested with Debian 8.2 sh, bash, ksh, ash, sash and even csh

  • A tempfile is created with mktemp. This usually immediately creates a file in /tmp
  • This tempfile is then redirected to FD 9 for write and FD 8 for read
  • Then the tempfile is immediately deleted. It stays open, though, until both FDs go out of existence.
  • Now the pipe is started. Each step adds to FD 9 only, if there was an error.
  • The wait is needed for ksh, because ksh else does not wait for all pipe commands to finish. However please note that there are unwanted sideffects if some background tasks are present, so I commented it out by default. If the wait does not hurt, you can comment it in.
  • Afterwards the file's contents are read. If it is empty (because all worked) read returns false, so true indicates an error
  • ksh and similar shells which only wait for the last pipe command need the wait uncommented

  • The last example uses printf "%1s" "$?" instead of echo -n "$?" because this is more portable. Not every platform interprets -n correctly.

  • printf "$?" would do it as well, however printf "%1s" catches some corner cases in case you run the script on some really broken platform. (Read: if you happen to program in paranoia_mode=extreme.)

  • FD 8 and FD 9 can be higher on platforms which support multiple digits. AFAIR a POSIX conformant shell does only need to support single digits.

  • Was tested with Debian 8.2 sh, bash, ksh, ash, sash and even csh

2 csh added
source | link
  • ksh and similar shells which only wait for the last pipe command need the wait uncommented

  • The last example uses printf "%1s" "$?" instead of echo -n "$?" because this is more portable. Not every platform interprets -n correctly.

  • printf "%?" would do it as well, however printf "%1s" catches some corner cases in case you run the script on some really broken platform. (Read: if you happen to program in paranoia_mode=extreme.)

  • FD 8 and FD 9 can be higher on platforms which support multiple digits. AFAIR a POSIX conformant shell does only need to support single digits.

  • Was tested with Debian 8.2 sh, bash, ksh, ash and, sash and even csh

  • ksh and similar shells which only wait for the last pipe command need the wait uncommented

  • The last example uses printf "%1s" "$?" instead of echo -n "$?" because this is more portable. Not every platform interprets -n correctly.

  • printf "%?" would do it as well, however printf "%1s" catches some corner cases in case you run the script on some really broken platform. (Read: if you happen to program in paranoia_mode=extreme.)

  • FD 8 and FD 9 can be higher on platforms which support multiple digits. AFAIR a POSIX conformant shell does only need to support single digits.

  • Was tested with Debian 8.2 sh, bash, ksh, ash and sash

  • ksh and similar shells which only wait for the last pipe command need the wait uncommented

  • The last example uses printf "%1s" "$?" instead of echo -n "$?" because this is more portable. Not every platform interprets -n correctly.

  • printf "%?" would do it as well, however printf "%1s" catches some corner cases in case you run the script on some really broken platform. (Read: if you happen to program in paranoia_mode=extreme.)

  • FD 8 and FD 9 can be higher on platforms which support multiple digits. AFAIR a POSIX conformant shell does only need to support single digits.

  • Was tested with Debian 8.2 sh, bash, ksh, ash, sash and even csh

1
source | link

Following is meant as an addon to the answer of @Patrik, in case you are not able to use one of the common solutions.

This answer assumes following:

  • You have a shell which does not know of $PIPESTATUS nor set -o pipefail
  • You want to use a pipe for parallel execution, so no temporary files.
  • You do not want to have additional clutter around if you interrupt the script, possibly by a sudden power outage.
  • This solution should be relatively easy to follow and clean to read.
  • You do not want to introduce additional subshells.
  • You cannot fiddle with the existing file descriptors, so stdin/out/err must not be touched (however you can introduce some new one temporarily)

Additional assumptions. You can get rid of all, but this clobbers the recipe too much, so it is not covered here:

  • All you want to know is that all commands in the PIPE have exit code 0.
  • You do not need additional side band information.
  • Your shell does wait for all pipe commands to return.

Before: foo | bar | baz, however this only returns the exit code of the last command (baz)

Wanted: $? must not be 0 (true), if any of the commands in the pipe failed

After:

TMPRESULTS="`mktemp`"
{
rm -f "$TMPRESULTS"

{ foo || echo $? >&9; } |
{ bar || echo $? >&9; } |
{ baz || echo $? >&9; }
#wait
! read TMPRESULTS <&8
} 9>>"$TMPRESULTS" 8<"$TMPRESULTS"

# $? now is 0 only if all commands had exit code 0

Explained:

  • A tempfile is created with mktemp. This usually immediately creates a file in /tmp
  • This tempfile is then redirected to FD 9 for write and FD 8 for read
  • Then the tempfile is immediately deleted. It stays open, though, until both FDs go out of existence.
  • Now the pipe is started. Each step adds to FD42 only, if there was an error.
  • The wait is needed for ksh, because ksh else does not wait for all pipe commands to finish. However please note that there are unwanted sideffects if some background tasks are present, so I commented it out by default. If the wait does not hurt, you can comment it in.
  • Afterwards the file's contents are read. If it is empty (because all worked) read returns false, so true indicates an error

This can be used as a plugin replacement for a single command and only needs following:

  • Unused FDs 9 and 8
  • A single environment variable to hold the name of the tempfile
  • And this recipe can be adapted to fairly any shell out there which allows IO redirection
  • Also it is fairly platform agnostic and does not need things like /proc/fd/N

BUGs:

This script has a bug in case /tmp runs out of space. If you need protection against this artificial case, too, you can do it as follows, however this has the disadvantage, that the number of 0 in 000 depends on the number of commands in the pipe, so it is slightly more complicated:

TMPRESULTS="`mktemp`"
{
rm -f "$TMPRESULTS"

{ foo; printf "%1s" "$?" >&9; } |
{ bar; printf "%1s" "$?" >&9; } |
{ baz; printf "%1s" "$?" >&9; }
#wait
read TMPRESULTS <&8
[ 000 = "$TMPRESULTS" ]
} 9>>"$TMPRESULTS" 8<"$TMPRESULTS"

Portablility notes:

  • ksh and similar shells which only wait for the last pipe command need the wait uncommented

  • The last example uses printf "%1s" "$?" instead of echo -n "$?" because this is more portable. Not every platform interprets -n correctly.

  • printf "%?" would do it as well, however printf "%1s" catches some corner cases in case you run the script on some really broken platform. (Read: if you happen to program in paranoia_mode=extreme.)

  • FD 8 and FD 9 can be higher on platforms which support multiple digits. AFAIR a POSIX conformant shell does only need to support single digits.

  • Was tested with Debian 8.2 sh, bash, ksh, ash and sash