If I follow a file somehow like this:
tail -f /var/log/syslog|grep s
I see all lines containing an "s"
Why does this not give any output, if I grep it again to the same "s"?
tail -f /var/log/syslog|grep s|grep s
2 Answers
As Rubo77 mentioned, the issue is solved by adding the --line-buffered to the first grep command:
tail -f /var/log/syslog|grep --line-buffered s|grep s
However, you then may ask, why isn't this needed for a single grep command? The difference between the two is that in the following command:
tail -f /var/log/syslog|grep s
STDOUT for grep is pointed to a terminal. grep most likely writes to STDOUT via functions contained in the stdio library. Per the documentation (stdio(3)):
Output streams that refer to terminal devices are always line buffered by default;
Thus, the underlying library calls are flushing the buffer after each line without any action on grep's part.
In this command:
tail -f /var/log/syslog|grep --line-buffered s|grep s
STDIO is now going to a pipe rather than a terminal device and the library functions that grep is using to write to STDOUT fully buffers these writes rather than using line buffering. When the --line-buffered flag is used, grep will call fflush, which will flush all of the buffered write.
answered Oct 28, 2014 at 20:47
You need to add --line-buffered to grep
tail -f /var/log/syslog|grep --line-buffered s|grep s
-
So to answer why: If does work, but because outputing is expensive programs buffer the output until they have a lot to output, the program calls the flush routine, or the program ends.
--line-bufferedwill cause grep to call flush after each line. When you are not following programs end so buffers are output. When there is only one grep after the tail … (can someone explain this?) Commented Oct 28, 2014 at 19:53
tail -f logfile, becausetail -fnever buffers its output. Nor is there a problem if we runtail -f logfile | grep 'foo bar'interactively, becausegrepdoes not buffer its output if its standard output is a terminal. However, if the output ofgrepis being piped into something else it starts buffering to improve efficiency.