4

I'm running tshark on a fifo, and the following is a bare example of a loop that prints the output of tshark as it comes:

tshark -i $fifo | while read line; do
    echo $line
done

The problem appears when I add filters to tshark. This example prints all $lines only after tshark exits (IP address is hidden):

tshark -i $fifo -T fields -e text -R '
    ip.src == **.**.***.**          &&
    http.response.code == 200       &&
    http.content_encoding == "gzip" &&
    http.content_type contains "text/html"
' | while read line; do
    echo $line
done

I have tried in other forms with no luck:

while read line; do
    echo $line
done < <(tshark ...)

while read line; do
    echo $line
done <<<"$(tshark ...)"

Even grep prints the lines only after tshark ends:

tshark ... | grep .

I have tried running tshark without a pipe and the lines are printed correctly as they come. Why is the command after the pipe being feeded only after tshark exits?

Additional details: | tee works, but I get everything printed again when tshark exits, so it is not a good deal.

2
  • 1
    Hard to believe that a program's output shall depend on the program reading from the pipe... I would expect a buffering problem. But how should the receiving end influence this? Stupid as it sounds: Maybe tshark checks what's at the other side of the pipe? strace is your friend. What about cat (instead of tee)? And why is the output printed twice? Commented Apr 14, 2013 at 23:54
  • 1
    @HaukeLaging Definitely a buffering problem: unbuffer tshark worked! I read the man page for unbuffer, but I couldn't really understand the reasoning behind this... see: unix.stackexchange.com/questions/25372/… Commented Apr 15, 2013 at 0:22

3 Answers 3

5

I was able to get it working with stdbuf from coreutils. Note that every command after the pipe requires the buffer to be adjusted too:

stdbuf -o 0 tshark -i $fifo -T fields -e text -R '
    ip.src == **.**.***.**          &&
    http.response.code == 200       &&
    http.content_encoding == "gzip" &&
    http.content_type contains "text/html"
' | 
stdbuf -o 0 sed 's/\\r\\n,\?/\n/g; s/\\t/\t/g' |

From the man page:

`stdbuf': Run a command with modified I/O stream buffering

(...)

`-o MODE'
`--output=MODE'
     Adjust the standard output stream buffering.
3
  • Why should every command after the pipe need adjustment, too? Commented Apr 15, 2013 at 3:27
  • @HaukeLaging Otherwise the output may get truncated. I got it truncated on the 80th line. Commented Apr 15, 2013 at 4:25
  • Great trick, but stdio buffering exists for a good reason, and it's lame that utilities which mess with stuff like isatty() force you to do this.
    – Cera
    Commented Apr 15, 2013 at 5:19
4

Check if your tshark version has the -l option for (nearly) line-buffered output.

1
  • 1
    Right, this is the fix on the tshark part. Although, as I have shown in my own answer, if more than one pipe is going to be used, the others will have to be fixed too. As a simple example, tshark -l ... | grep . | grep . doesn't work, whereas tshark -l ... | stdbuf -o 0 grep . | grep . works, so all information on this thread was useful. Thanks everybody so much for all the help! Commented Apr 15, 2013 at 15:20
3

Some utilities call isatty() to determine whether their output is a terminal and adjust their behaviour accordingly. gzip is a good example of this.

Try running it with script(1):

-c, --command command

Run the command rather than an interactive shell. This makes it easy for a script to capture the output of a program that behaves differently when its stdout is not a tty.

If you run it like this:

script -c tshark -i $fifo -T fields -e text -R '
    ip.src == **.**.***.**          &&
    http.response.code == 200       &&
    http.content_encoding == "gzip" &&
    http.content_type contains "text/html"
' | while read line; do
    echo $line
done

You should be able to see lines come out live.

2
  • Thanks, but script requires something more. I get script: invalid option -- 'i' Commented Apr 15, 2013 at 2:13
  • Yo may need to put the entire command in quotes. Gets tricky with such a complex command. Try putting your entire call to tshark in a shell script, and then doing script -c myscript.sh.
    – Cera
    Commented Apr 15, 2013 at 5:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .