Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have to inspect a growing log in real time, and I found an issue that makes me miss some lines (I can't understand exactly what lines) using tail -f or tailf, grep and cut.

I use grep because I want to filter lines containing a specific word, and then cut -c -NUM because some lines are very long and I don't want them to wrap in my terminal window.

Here is a minimal example, in which I'm watching a sample log made of "Nth line" lines, filtering for word "line", missing all lines in this case (obviously ^C is me killing the process):

$ tail -n 3 -f log

13th line
14th line
15th line
^C

$ tail -n 3 -f log | grep --color=never 'line'

13th line
14th line
15th line
^C

$ echo $COLUMNS

100

$ tail -n 3 -f log | grep --color=never 'line' | cut -c -$COLUMNS

^C

Note that this issue occours only with tail -f (or tailf), grep and cut combined. If I do not use grep in the middle, or cut at the end of the pipe, there's no missing line. If I replace tail -f with tail or cat, no problem. You can see every possible case in this longer example.

I'm on Ubuntu 13.04, GNU coreutils (tail, cut) version 8.20 and GNU grep 2.14.

share|improve this question
add comment

1 Answer 1

up vote 3 down vote accepted

The problem with your example is buffering between grep and cut. The data will only by passed along the pipe when the buffer is full (4kB on my system).

Try adding --line-buffered to grep to have it flush the buffer after each line.

share|improve this answer
    
Thank you very much! So grep uses a buffer only when piped into another command? –  lorenzo-s Jul 4 '13 at 10:20
2  
The buffering is standard behavior of libc, it's not specific to grep. By default output to a terminal is line buffered, and gets flushed every line. Otherwise, the output is fully buffered. This includes piping to another command, redirecting output to a file and/or processes that are not attached to a terminal (e.g. cron jobs). –  David Baggerman Jul 4 '13 at 10:59
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.