5

An example should clarify my question. This behavior makes sense to me:

$ echo hi | cat
hi
$ echo hi | tee >(cat)
hi
hi

The first case is obvious. In the 2nd case, we pipe "hi" into tee using command substitution, and one "hi" is printed by the tee'd cat, while another is printed by tee's pass through pipe. So far, so good...

But what happens to the first "hi" in this case:

$ echo hi | tee >(echo yo)
yo

The return code is 141, a pipe fail. What could be causing this?

I'm running Mac OSX El Capitain, bash in default terminal app

  • 2
    What system are you running? I am getting the correct output (both "yo" and "hi") on my systems (Arch, Cygwin, CentOS). – nehcsivart Apr 4 '16 at 3:18
  • 3
    can't reproduce.. – heemayl Apr 4 '16 at 3:19
  • @tchen Mac OSX El Capitain, bash, running in default terminal app. – Jonah Apr 4 '16 at 3:20
  • @Jonah: What's the return code? – cuonglm Apr 4 '16 at 3:44
  • @heemayl, i verified on another system that the output is "yo hi". any idea what could be going on in my system to cause this? Mac uses BSD tee if that matters... – Jonah Apr 4 '16 at 3:46
7

I think I’ve figured out how to tweak your experience to turn it into something other people will be able to reproduce:

$ (echo hello; sleep 1; echo world) | tee >(cat)
hello
hello                                                       … and, after a brief delay,
world
world

$ echo "$?"
0

$ (echo hello; sleep 1; echo world) | tee >(echo yo)
yo
hello

$ echo "$?"
141

As you hopefully understand, >(command) creates a pipe to a process running command.  The standard input of command is connected to a pathname that other commands on the command line (in this case, tee) can open and write to.  When command is cat, the process sits there and reads from stdin until it gets an EOF.  In this case, tee has no problem writing all the data it reads from its stdin to the pipe.

But, when command is echo yo, the process writes yo to the stdout and immediately exits.  This causes a problem for tee; when it writes to a pipe with no process at the other end, it gets a SIGPIPE signal.

Apparently OS X’s version of tee writes to the file(s) on the command line first, and then its stdout.  So, in your example (echo hi | tee >(echo yo)), tee gets the pipe failure on its very first write.  Whereas, the version of tee on Linux and Cygwin writes to the stdout first, so it manages to write hi to the screen before it dies.  In my enhanced example, tee dies when it writes hello to the pipe, so it doesn’t get a chance to read and write world.

  • 1
    I don't think the problem is what you said. Try the command on linux, you will get 141 return code too. Try strace, and you see bash write to the file first. I guess the problem is the lack of /dev/fd, so bash used named pipe for process substitution, some thing went wrong in this case – cuonglm Apr 4 '16 at 18:47
2

To visualize what's going on, compare the following two variations:

bash -c 'echo hi | tee >(sleep 1; echo yo); echo $?'

bash -c 'wait_and_tee () { sleep 1; tee "$@"; };
         echo hi | wait_and_tee >(echo yo); echo $?'

Notice what's happening in the first variation?

$ bash -c 'echo hi | tee >(sleep 1; echo yo); echo $?'     
hi
0
$ yo

The command in the process substitution sleep 1; echo yo is executed in parallel with the commands outside, and bash doesn't wait for it to finish. So the sequence of events is:

  • The three commands echo hi, sleep 1; echo yo and tee are started in parallel.
  • echo hi writes hi to its output (the | pipe).
  • tee reads from the pipe and writes to both its standard output and its command line argument, which is another pipe created by >(…). This results in one copy of hi printed to the terminal, and one copy in the buffer of the >(…) pipe.
  • In parallel with the previous bullet point, echo hi exits, which closes the write end of the | pipe.
  • tee notices that it has reached he end of its input file. It has written all of its data out, so it exits.
  • From bash's perspective, both sides of the pipe have exited, so the command is over. Since tee returned 0, the status of the pipeline is 0.
  • One second later, sleep 1 finishes, and echo yo is executed.

In the second variation, I force echo yo to terminate before tee, by forcing it to terminate before tee starts. This time, the sequence of events is:

  • The three commands echo hi, echo yo and sleep 1 are started in parallel.
  • echo hi writes hi to its output (the | pipe).
  • In parallel with the previous bullet point, echo yo prints yo and exits.
  • One second later, sleep 1 exits and tee starts.
  • tee reads hi from its input, and attempts to write it both to the terminal (its standard output) and the pipe resulting from >(…) passed as an argument. Since the process that had this pipe open for reading (echo yo) exited a second ago, the attempt to write to the pipe fails with SIGPIPE (signal 13, which the shell reports as 128+signal_number).

As G-Man explains, whether hi is displayed in the second case depends on whether tee tries to write to its standard output or its file argument first.

Without the sleep calls, the timing could go either way (I get about half/half under Linux, a different kernel might make one timing a lot more likely than the other).

  • In my Debian, I always get yo, hi printed. There's always a SIGPIPE and return code is 141 in both case. I guess the problem came from the lack of /dev/fd in OSX. – cuonglm Apr 5 '16 at 1:47
  • @cuonglm On both Debian jessie and Ubuntu 14.04, I get either behavior randomly. I don't understand why you think the lack of /dev/fd on OSX matters. – Gilles Apr 5 '16 at 9:35
  • Because the lack of /dev/fd cause bash to use named pipe instead. I remember a bug was raised about the issue before bash 4.3 released. And in my above comment, I mean the command echo hi | tee >(echo yo) always produce the consistent output in my Debian Jessie. – cuonglm Apr 5 '16 at 9:40
  • @cuonglm I know that without /dev/fd bash uses a named pipe instead of an anonymous one. And I know that in some cases it makes an observable difference, because opening and closing a pipe are observable events. But how does this change the possible behaviors in this case? – Gilles Apr 5 '16 at 9:50
  • I'm not sure about this, that why I don't provide my own answer. I also don't have OSX now for tracing. It will be clearer if the OP can provide the dtruss output (I requested him in the comment but got no response). – cuonglm Apr 5 '16 at 9:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.