Can we use $PIPESTATUS with the tee (or pee) command?

Question

In my bash scripts, I often use pipes and would like to know which stage of the pipe was causing the problem in case of errors. The basic structure of such snippets is:

#!/bin/bash

ProduceCommand 2>/dev/null | ConsumeCommand >/dev/null 2>&1
PipeErrors=("${PIPESTATUS[@]}")
[[ "${PipeErrors[0]}" -eq '0' ]] || { HandleErrorInProduceCommand; }
[[ "${PipeErrors[1]}" -eq '0' ]] || { HandleErrorInConsumeCommand; }

Now (interestingly enough for the first time) I am in a situation where it would be great if I could use either tee or pee. But what happens to $PIPESTATUS when using these commands? For example:

#!/bin/bash

ProduceCommand 2>/dev/null | tee >(ConsumeCommand1) >(ConsumeCommand2) >/dev/null 2>&1
PipeErrors=("${PIPESTATUS[@]}")

or

#!/bin/bash

ProduceCommand 2>/dev/null | pee ConsumeCommand1 ConsumeCommand2 2>/dev/null
PipeErrors=("${PIPESTATUS[@]}")

I believe that in both cases ${PipeErrors[0]} reflects the error status of ProduceCommand. Further, it would be logical to assume that ${PipeErrors[1]} reflects the error status of tee or pee itself, respectively.

But this leads me into at least two understanding problems:

1. What is the error status (return value) of tee or pee? I didn't find precise statements about that in the man pages. Do they return a hard-coded error status if one of the consume commands fails, or do they relay the error status of the consume commands somehow (as ssh does, for example)? If the former is the case, how can we find out which of the consume commands is the culprit? If the latter is the case, which error status is relayed? Is it simply that of the command which fails first?

2. AFAIK, bash or the tee or pee command itself, respectively, internally use pipes (fifos) to get ProduceCommand's output to the consume commands. This means that we have a pipe whose (first and in this case, only) receiving side is a pipe itself. This should not influence $PipeErrors in the sample code above, but I am really unsure.

Could somebody shed some light on this?

Answer 1

What is the error status (return value) of tee

It's 0 when it was able to copy all the data to all the output files, and >0 if not. See the spec. The GNU coreutils implementation of tee has extra options to ignore errors when writing to pipes (as those used to implement >(...)):

$ seq 1024 | tee >(false) >/dev/null; echo $?
141
$ seq 1024 | tee -p >(false) >/dev/null; echo $?
0

There are no options to know which of the outputs has failed, if any [1].

---

But your question seems to be rather about whether the exit status of the commands run in >(..) process substitutions is reflected in any way in PIPESTATUS, or if it could be in any way reflected in PIPESTATUS.

The answer to that is No.

First of all, notice that >(...) are more like ... & background commands than like ...|... pipeline commands. In a snippet like:

... | tee >(cmd ...) | ...; echo ${PIPESTATUS[@]}

there's no guarantee that cmd has finished at all by the time you run echo ${PIPESTATUS[@]}.

But they're not exactly like ...& either as you cannot wait for them and cannot obtain their status from $! -- except in some limited cases which do not include their use with tee or other external commands:

$ bash -c 'echo 1 | tee >(sleep 2; sed s/1/2/); wait; echo DONE'
1
DONE
$
<after two seconds>
2

As you see, both tee and the main shell have finished long before the command from >(...).

[1] A command like pee which is running the output "subcommands" itself (and is waiting for them to finish) could be smarter and reflect in its exit status which of them have failed (e.g. by setting bit 1 for the first, bit 2 for the second, and so on, up to 8 subcommands), but does no such thing either.

Answer 2

You can always do:

{
  {
    ProduceCommand 2>/dev/null 3>&- ||
      HandleErrorInProduceCommand >&3 3>&-
  } |
    tee >(
      ConsumeCommand1 3>&- ||
        HandleErrorInConsumer1 >&3 3>&-
    ) >(
      ConsumeCommand2 3>&- ||
        HandleErrorInConsumer2 >&3 3>&-
    ) > /dev/null
} 3>&1

That handles the error in each of the subshells where the producer and consumers are started.

We duplicate stdout onto fd 3, so can restore the original stdout for the error handlers as we don't want their output if any to go through the pipes.

If you'd rather the error handlers be run within the main shell process (so it can exit it for instance), you could get those subshells to pass the exit status to the parent shell via some command substitution pipe:

 producer_status=-1
consumer1_status=-1
consumer2_status=-1
{
  eval "$(
    {
      {
        ProduceCommand 2>/dev/null 4>&-
        echo "producer_status=$?" >&4
      } | tee >(
        ConsumeCommand1 4>&-
        echo "consumer1_status=$?" >&4
      ) >(
        ConsumeCommand2 4>&-
        echo "consumer2_status=$?" >&4
      )
    } 4>&1 >&3 3>&-
  )"
} 3>&1

[ "$producer_status"  -eq 0 ] || HandleErrorInProduceCommand
[ "$consumer1_status" -eq 0 ] || HandleErrorInConsumer1
[ "$consumer2_status" -eq 0 ] || HandleErrorInConsumer2

That avoids the $PIPESTATUS bashism or you could avoid the >(...) kshism and replace them with a normal pipeline:

{
  ProduceCommand 2>/dev/null |
    {
      tee /dev/fd/4 |
        ConsumeCommand1 4>&-
    } 4>&1 >&3 3>&- |
      ConsumeCommand2 3>&-
} 3>&1
 producer_status=${PIPESTATUS[0]}
consumer1_status=${PIPESTATUS[1]}
consumer2_status=${PIPESTATUS[2]}

[ "$producer_status"  -eq 0 ] || HandleErrorInProduceCommand
[ "$consumer1_status" -eq 0 ] || HandleErrorInConsumer1
[ "$consumer2_status" -eq 0 ] || HandleErrorInConsumer2

Or combine the two approaches and then get standard sh syntax and also get access to tee's exit status as a bonus.

 producer_status=-1
      tee_status=-1
consumer1_status=-1
consumer2_status=-1

{
  eval "$(
    {
      {
        ProduceCommand 2>/dev/null 4>&-
        echo "producer_status=$?" >&4
      } 3>&- |
        {
          {
            tee /dev/fd/5 4>&-
            echo "tee_status=$?" >&4
          } |
            ConsumeCommand1 4>&-
          echo "consumer1_status=$?" >&4
        } 5>&1 >&3 3>&- |
        ConsumeCommand2 >&3 3>&- 4>&- 
        echo "consumer2_status=$?" >&4
    } 4>&1
  )"
} 3>&1

[ "$producer_status"  -eq 0 ] || HandleErrorInProduceCommand
[ "$tee_status"       -eq 0 ] || HandleErrorInTee
[ "$consumer1_status" -eq 0 ] || HandleErrorInConsumer1
[ "$consumer2_status" -eq 0 ] || HandleErrorInConsumer2

Note that tee may die of a SIGPIPE if one of the processes exits without reading all the input, which means the other process may also miss some input. So it may be important to also check its exit status.

As already noted by @UncleBilly, with the GNU implementation of tee, that can be worked around with the -p option (where tee ignores SIGPIPE signals and just stops trying to write more data to a pipe when it becomes broken).

With other implementations, you can replace tee ... with (trap '' PIPE; exec tee ...) to get a similar behaviour (though possibly some error messages about the broken pipes even if tee is not aborted).