Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I'm running tshark on a fifo, and the following is a bare example of a loop that prints the output of tshark as it comes:

tshark -i $fifo | while read line; do
    echo $line
done

The problem appears when I add filters to tshark. This example prints all $lines only after tshark exits (IP address is hidden):

tshark -i $fifo -T fields -e text -R '
    ip.src == **.**.***.**          &&
    http.response.code == 200       &&
    http.content_encoding == "gzip" &&
    http.content_type contains "text/html"
' | while read line; do
    echo $line
done

I have tried in other forms with no luck:

while read line; do
    echo $line
done < <(tshark ...)

while read line; do
    echo $line
done <<<"$(tshark ...)"

Even grep prints the lines only after tshark ends:

tshark ... | grep .

I have tried running tshark without a pipe and the lines are printed correctly as they come. Why is the command after the pipe being feeded only after tshark exits?

Additional details: | tee works, but I get everything printed again when tshark exits, so it is not a good deal.

share|improve this question
1  
Hard to believe that a program's output shall depend on the program reading from the pipe... I would expect a buffering problem. But how should the receiving end influence this? Stupid as it sounds: Maybe tshark checks what's at the other side of the pipe? strace is your friend. What about cat (instead of tee)? And why is the output printed twice? –  Hauke Laging Apr 14 '13 at 23:54
    
@HaukeLaging Definitely a buffering problem: unbuffer tshark worked! I read the man page for unbuffer, but I couldn't really understand the reasoning behind this... see: unix.stackexchange.com/questions/25372/… –  Teresa e Junior Apr 15 '13 at 0:22

3 Answers 3

up vote 2 down vote accepted

Check if your tshark version has the -l option for (nearly) line-buffered output.

share|improve this answer
    
Right, this is the fix on the tshark part. Although, as I have shown in my own answer, if more than one pipe is going to be used, the others will have to be fixed too. As a simple example, tshark -l ... | grep . | grep . doesn't work, whereas tshark -l ... | stdbuf -o 0 grep . | grep . works, so all information on this thread was useful. Thanks everybody so much for all the help! –  Teresa e Junior Apr 15 '13 at 15:20

I was able to get it working with stdbuf from coreutils. Note that every command after the pipe requires the buffer to be adjusted too:

stdbuf -o 0 tshark -i $fifo -T fields -e text -R '
    ip.src == **.**.***.**          &&
    http.response.code == 200       &&
    http.content_encoding == "gzip" &&
    http.content_type contains "text/html"
' | 
stdbuf -o 0 sed 's/\\r\\n,\?/\n/g; s/\\t/\t/g' |

From the man page:

`stdbuf': Run a command with modified I/O stream buffering

(...)

`-o MODE'
`--output=MODE'
     Adjust the standard output stream buffering.
share|improve this answer
    
Why should every command after the pipe need adjustment, too? –  Hauke Laging Apr 15 '13 at 3:27
    
@HaukeLaging Otherwise the output may get truncated. I got it truncated on the 80th line. –  Teresa e Junior Apr 15 '13 at 4:25
    
Great trick, but stdio buffering exists for a good reason, and it's lame that utilities which mess with stuff like isatty() force you to do this. –  Cerales Apr 15 '13 at 5:19

Some utilities call isatty() to determine whether their output is a terminal and adjust their behaviour accordingly. gzip is a good example of this.

Try running it with script(1):

-c, --command command

Run the command rather than an interactive shell. This makes it easy for a script to capture the output of a program that behaves differently when its stdout is not a tty.

If you run it like this:

script -c tshark -i $fifo -T fields -e text -R '
    ip.src == **.**.***.**          &&
    http.response.code == 200       &&
    http.content_encoding == "gzip" &&
    http.content_type contains "text/html"
' | while read line; do
    echo $line
done

You should be able to see lines come out live.

share|improve this answer
    
Thanks, but script requires something more. I get script: invalid option -- 'i' –  Teresa e Junior Apr 15 '13 at 2:13
    
Yo may need to put the entire command in quotes. Gets tricky with such a complex command. Try putting your entire call to tshark in a shell script, and then doing script -c myscript.sh. –  Cerales Apr 15 '13 at 5:17

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.