Terminating the whole pipeline at once when its last process ends

Question

Given the pipeline

printf '%s\n'  1 2 3 4 5 6 7 8 9 |
while read -r num
do
  echo "$num" > /dev/stderr
  echo "$num"
done |
while read -r num
do
  echo $(( $num * 10 ))
  [ "$num" -eq 5 ] && break
done

bash will output something like

1
2
10
3
20
4
30
5
40
50
6
7
8
9

to the terminal. The places in the single digit sequence where the two-digit numbers are inserted will be different each time you run the pipeline because of how buffering and/or scheduling work.

I would like to have the first while-read loop terminate at once when the second loop terminates; that is, I want the pipeline to not output any single digits after it has printed the number 50.

I tried using the command stdbuf -o0 to set the buffer to 0, as suggested by this article and Force line-buffering of stdout in a pipeline.

stdbuf -o0 printf '%s\n'  1 2 3 4 5 6 7 8 9 |
while read -r num
do
  echo "$num" > /dev/stderr
  echo "$num"
done |
while read -r num
do
  echo $(( $num * 10 ))
  [ "$num" -eq 5 ] && break
done

hoping to achieve this, but the printf command and the first loop still stay alive and can print their output after the last loop has printed the number 50.

I also tried set -o pipefail and turning the second loop into the function:

g ()
{
  while read -r num
  do
    echo $(( $num * 10 ))
    [ "$num" -eq 5 ] && break
  done
  return 1
}

and modifying the pipeline accordingly:

printf '%s\n'  1 2 3 4 5 6 7 8 9 |
while read -r num
do
  echo "$num" > /dev/stderr
  echo "$num"
done |
g

(and then calling set +o pipefail afterwards), but that also fails to achieve what I want.

Apparently there is some fundamental error in my understanding of how pipelines and buffering work. Please explain to me what I am missing and how I can get the output I want.

Answer 1

The problem (as, I believe, you at least partly understand) is that the first loop runs all the way through 1 to 9, filling the pipe buffer, before the second loop gets to 5.  If you add sleep 1 (pause for one second) to the bottom of the first loop, not only will you get better sequencing of the output (1, 10, 2, 20, 3, 30, ...), but you will give the second loop a chance to finish before the first loop writes 6.

You can probably give sleep a fractional argument (e.g., sleep 0.5).

Answer 2

One could modify the pipeline into the following:

printf '%s\n'  1 2 3 4 5 6 7 8 9 |
while read -r num
do
  echo "$num" > /dev/stderr
  echo "$num" | cat -u || break
done |
while read -r num
do
  echo $(( $num * 10 ))
  [ "$num" -eq 5 ] && break
done

That would make the pipeline print the numbers in order. It would also make the first loop notice that the second one no longer exists when it reaches its sixth iteration so it will still print the number 6 after the number 50 though.

The option -u makes cat it operate without buffers. If you're using GNU cat then the -u option is not needed since GNU cat never uses buffers and simply ignores the option, according to the man page.

This does not modify the pipe buffers, which exist between the programs.