43

I am trying to grep the ongoing tail of file log and get the nth word from a line. Example file:

$ cat > test.txt <<EOL
Beam goes blah
John goes hey
Beam goes what?
John goes forget it
Beam goes okay
Beam goes bye
EOL
^C

Now if I do a tail:

$ tail -f test.txt
Beam goes blah
John goes hey
Beam goes what?
John goes forget it
Beam goes okay
Beam goes bye
^C

If I grep that tail:

$ tail -f test.txt | grep Beam
Beam goes blah
Beam goes what?
Beam goes okay
Beam goes bye
^C

But if I awk that grep:

$ tail -f test.txt | grep Beam | awk '{print $3}'

Nothing no matter how long I wait. I suspect it's something to do with the way the stream works.

Anyone have any clue?

3 Answers 3

70

It's probably output buffering from grep. you can disable that with grep --line-buffered.

But you don't need to pipe output from grep into awk. awk can do regexp pattern matching all by itself.

tail -f test.txt | awk '/Beam/ {print $3}'

1
  • Could you help to comment why tail -f test.txt | awk -f CERTAINLIB.AWK -e '/Beam/ {print $3}' does not work ?
    – SOUser
    Apr 28, 2020 at 8:50
11

Using tail -f test.txt | awk '/Beam/{print $3}' works for me. As well as using tail -f test.txt | grep --line-buffered Beam | awk '{print $3}' (gnu grep).

The issue here is if awk received the data line-by-line or as one larger data block. The GNU version of grep sends output in larger blocks since it is more efficient, but awk needs to read line-by-line in order to output line-by-line.

Put this way: grep will only send data when the buffer is filled, awk is waiting for that buffer to be filled, so it's sending out nothing.

4

See the --line-buffered option of grep.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.