29

If I want to tcpdump DNS requests by clients (on an OpenWrt 10.04 router), then I

root@ROUTER:/etc# tcpdump -n -i br-lan dst port 53 2>&1       
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-lan, link-type EN10MB (Ethernet), capture size 96 bytes
22:29:38.989412 IP 192.168.1.200.55919 > 192.168.1.1.53: 5697+ A? foo.org. (25)
22:29:39.538981 IP 192.168.1.200.60071 > 192.168.1.1.53: 17481+ PTR? 150.33.87.208.in-addr.arpa. (44)
^C
2 packets captured
3 packets received by filter
0 packets dropped by kernel

That's fully ok. But. Why can't I pipe the tcpdumps output in realtime?

root@ROUTER:/etc# tcpdump -n -i br-lan dst port 53 2>&1 | awk '/\?/ {print $3}'
^C
root@ROUTER:/etc# 

If I awk, etc. anything after tcpdump, I don't get ANY output. Why is that? Why can't I process the output of tcpdump with pipelining in realtime? (so that e.g.: in the example in only outputs the 3rd column)

Are there any solutions for this?

38

Straight out of man tcpdump

-l     Make stdout line buffered.  Useful if you want to see the data while 
       capturing it.  E.g.,

              tcpdump -l | tee dat

       or

              tcpdump -l > dat & tail -f dat

       Note that on Windows,``line buffered'' means ``unbuffered'', so that 
       WinDump will write each character individually if -l is specified.

       -U is similar to -l in its behavior, but it will cause output to be 
       ``packet-buffered'', so that the output is written to stdout at the 
       end of each packet rather than at the end of each line; this is 
       buffered on all platforms, including Windows.
  • Unfortunately, this doesn't seem to work with tcpdump version 4.9.3 compiled with libpcap version 1.9.1 (with TPACKET_V3). The output seems to still be buffered and I get delayed+buffered output. According to man tcpdump this supports both -l and --immediate-mode but in reality it doesn't seem to work even with both of those added. – Mikko Rantalainen Aug 6 '20 at 8:43
  • It turns out that --immediate-mode -l -n works correctly with tcpdump version 4.9.3. If your DNS server cannot cope with the packet flow your output will be delayed unless you pass -n in addition to -l or --immediate-mode because without the -n the output will be buffered until DNS resolution has been completed or timeout occurs. – Mikko Rantalainen Aug 6 '20 at 9:30
7

Use the option -U in combination with -w so that tcpdump writes packets immediately.

3

The tcpdump is apparently buffering output when it writes to a pipe. It's not flushing output for each write, so the system will write the output in about 4k byte chunks. Your filter is limiting out put so you won't see anything until that filter has written enough output. Once it collects enough it will be written out in a chunk and you should see several lines emitted then.

Try to trigger DNS lookups many times and see what happens then.

3

I'm building a real-time monitoring wrapper around tcpdump that needs to see packets as soon as they are available. Even with -l there is some delay.

tcpdump now has --immediate-mode, which solved this problem for me. In order to get it to work I used it in conjunction with -l.

See this answer.

2

expect has an unbuffer command to fool commands into assuming they are writing to a tty, so they don't buffer.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.