44

I am trying to grep the ongoing tail of file log and get the nth word from a line. Example file:

$ cat > test.txt <<EOL
Beam goes blah
John goes hey
Beam goes what?
John goes forget it
Beam goes okay
Beam goes bye
EOL
^C

Now if I do a tail:

$ tail -f test.txt
Beam goes blah
John goes hey
Beam goes what?
John goes forget it
Beam goes okay
Beam goes bye
^C

If I grep that tail:

$ tail -f test.txt | grep Beam
Beam goes blah
Beam goes what?
Beam goes okay
Beam goes bye
^C

But if I awk that grep:

$ tail -f test.txt | grep Beam | awk '{print $3}'

Nothing no matter how long I wait. I suspect it's something to do with the way the stream works.

Anyone have any clue?

3 Answers 3

73

It's probably output buffering from grep. you can disable that with grep --line-buffered.

But you don't need to pipe output from grep into awk. awk can do regexp pattern matching all by itself.

tail -f test.txt | awk '/Beam/ {print $3}'

1
  • Could you help to comment why tail -f test.txt | awk -f CERTAINLIB.AWK -e '/Beam/ {print $3}' does not work ?
    – SOUser
    Commented Apr 28, 2020 at 8:50
13

Using tail -f test.txt | awk '/Beam/{print $3}' works for me. As well as using tail -f test.txt | grep --line-buffered Beam | awk '{print $3}' (gnu grep).

The issue here is if awk received the data line-by-line or as one larger data block. The GNU version of grep sends output in larger blocks since it is more efficient, but awk needs to read line-by-line in order to output line-by-line.

Put this way: grep will only send data when the buffer is filled, awk is waiting for that buffer to be filled, so it's sending out nothing.

4

See the --line-buffered option of grep.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .