Bash: cannot break out of piped "while read" loop; process substitution works

Question

I intend to pipe the output of a program into a while read VAR loop and break when a pattern is found, but it doesn't.

Proof of concept:

inotifywait -qm -e create . | while read line; do echo $line; break; done
./ CREATE newfile

..

tail -f /var/log/syslog | while read line; do echo $line; break; done
Nov 6 22:44:05 section9 ntpdate[2381]: adjust time server 91.189.89.199 offset 0.272779 sec

These never exit no matter what the source program outputs. Setting set -x beforehand suggests that the loop never iterates to the second read. $BASH_SUBSHELL is 1 in these examples.

Shouldn't tail, inotifywait, etc. receive SIGPIPE and exit?

Note that process substitution (while read ... break; done < <(tail -f ...)) works OK. $BASH_SUBSHELL is 0 in this case.

Answer 1

The key is that, under bash (other may shells differ), a pipeline is not terminated until all commands in the pipeline are finished.

To understand, let's consider:

inotifywait -qm -e create . | while read line; do echo $line; break; done

When read reads a line, it is echoed and then break is executed and the last process terminates. The first process, however, continues until a write to stdout fails. Thus, the loop will continue at least until inotifywait tries to write its second line of output. Because of the vagaries of buffering, it may not happen even then. It may take several lines before the discovery occurs. When that attempted write fails, then SIGPIPE is issued.

Now, consider the other case:

while read line; do echo $line; break; done < <(inotifywait -qm -e create .)

Here, there is no pipeline. When break is executed, the while loop is done.

Documentation

From the "Pipelines" section of man bash:

The shell waits for all commands in the pipeline to terminate....

This behavior is a choice made by bash. It is not mandated by POSIX. POSIX states:

If the pipeline is not in the background (see Asynchronous Lists), the shell shall wait for the last command specified in the pipeline to complete, and may also wait for all commands to complete.

Answer 2

strace tail -f -n 5000 /var/log/messages | 
    while read line; do echo $line; break; done;

shows in detail what happens:

[...]
write(1, "Nov  1 22:20:01 inno systemd[1]:"..., 4096) = 4096

i.e. tail writes up to 4096 bytes at once to the pipeline (that is what a pipeline can buffer). Usually there is not so much data at all for tail:

strace tail -f /var/log/messages | while read line; do echo $line; break; done;

shows

write(1, "Nov  7 07:00:02 inno systemd[1]:"..., 730) = 730

Thus tail would not try to write again until something is appended to the file.

I thought unbuffer would solve this problem. I am surprised that it doesn't.

unbuffer tail -f file1 | while read line; do echo $line; break; done