3

I observed the following behaviour in bash:

{ echo 'foo' ; sleep 10 ; }

-> standard output "foo" appears immediately, after 10 seconds the command is done (as expected)

{ echo 'foo' ; sleep 10 ; } > >(grep 'oo')

-> standard output "foo" appears immediately, after 10 seconds the command is done (as expected)

{ echo 'foo' ; sleep 10 ; } > >(grep 'oo' | grep 'oo')

-> standard output "foo" appears after 10 seconds

{ echo 'foo' ; sleep 10 ; } > >(grep 'oo' >&2)

-> standard error "foo" appears after 10 seconds

Why does the command line with the single grep in the command substitution output its result immediately whilst the variants with the pipe and redirect wait until the end of the sleep?

  • 1
    What if you do grep --line-buffered – sdkks Aug 12 '17 at 1:46
3

You don’t need to get so fancy; you can observe the same effect with

{ echo 'foo' ; sleep 10 ; } | grep oo | grep oo

or

{ echo 'foo' ; sleep 10 ; } | grep oo | cat

or

  • Open two terminals.  Go to the same directory in both (e.g., your home directory, or /tmp).
  • In one, do { echo 'foo' ; sleep 10 ; } | grep oo > foo.out.
  • In the other, do ls -ld foo.out repeatedly.
    You will see that the foo.out file appears immediately, but is size 0 for ten seconds, after which it becomes 4 bytes long.

Simply: grep tests to see whether its (standard) output is a terminal.  If it is, it writes output as quickly as it has output to write.  If it isn’t, it buffers its output, and writes N  bytes at a time, where N  is typically 512, but may be different in some implementations.

  • 1
    And you will see the same with many other Unix programs because it is the default for (standard) C and many Unix programs are written in C, although a few have options to change it. – dave_thompson_085 Aug 12 '17 at 2:36
1

Extending Scott's answer:

Compare

 { echo 'foo' ; sleep 3 ; } | grep oo | cat

with

 { echo 'foo' ; sleep 3 ; } | stdbuf -o 0 grep oo | cat

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.