Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I piped my own shell scripts for some testing and accidentally noticed something strange. Namely, the stderr of those piped processes isn't always displayed on the screen.

I simplified the scripts and here's my session with bash:

$ cat file1
echo stdout
echo stderr 1>&2
$ cat file2
echo stdout2
echo stderr2 1>&2
$ cat file3
echo stdout3
echo stderr3 1>&2

$ ./file1 | ./file2 | ./file3
stderr2
stderr
stdout3
stderr3

$ ./file1 | ./file2 | ./file3
stderr
stdout3
stderr3

$ ./file1 | ./file2 | ./file3
stderr2
stdout3
stderr3

I know that the stdout of the first two scripts is lost, since file3 doesn't read anything, but it doesn't matter here.

What's happening with the stderr2 and stderr outputs in the 2nd and 3rd case, respectively?

share|improve this question

1 Answer 1

up vote 3 down vote accepted

Ooh, a race condition! There are a few things going on here behind the scenes to produce this effect.

First, if a process tries to write to a pipe whose other end has been closed, that process receives a SIGPIPE signal. By default, this ends the process. Why would you want this? If you run cat my_huge_file | head -3, you'll only see the first three lines of the file, and because of the signaling mechanism, cat knows to stop after those three lines. Without it, it would read the whole file.

Second, all processes in a pipeline run in parallel, and which process will finish first is unpredictable. When a process ends, all of its file descriptors are closed -- including the read ends of any pipes.

So what's happening is that sometimes, for example, file3 finishes completely before file2 has written to its stdout. Then, when file2 does try to write, it gets SIGPIPE, and quits without reaching the echo stderr2 line. But then other times, file2 manages to write to stdout before file3 completes, and in this case it gets to continue.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.