Is it possible to view unicode text in tcpdump capture output?

Question

Unfortunately seems like tcpdump is not capable of detecting and displaying Unicode-encoded text (specifically utf-8) in captures. It treats unicode text like regular binary data and displays it in dots. I need this feature to debug an application.

To be more specific, I run tcpdump 4.3.0 in a remote ssh session inside tmux in Gnome terminal. OS is debian 7.

Is this possible to view unicode text in live network captures using tcpdump or combination of it with other linux commands?

Answer 1

The TXR Lisp program below, called pcap.tl is used as follows:

$ tcpdump -s 1024 -w - | ~/txr/txr pcap.tl

It parses the pcap format output by -w and produces output like:

192.168.1.102 --> 192.168.1.146
  ether hdr: #S(eth-header dst-mac #(8 0 39 249 113 4) src-mac #(0 30 79 164 102 184) eth-type ETH_IPV4)
  ipv4 hdr:  #S(ipv4-header ihl 5 ver 4 ecn 0 dscp 0 len 101 ident 7434 fragoffs-hi 0 flags 2
               fragoffs-lo 0 ttl 64 proto 6 hdr-sum 39232 src-ip 3232235878
               dst-ip 3232235922)
  ipv4 payload as text: P��.;;�.�+i�.6�...
KK-?9rrt2b
春が来た (Haru-ga Kita/Spring has Com

The code uses FFI type declarations to define the structure types corresponding to the pcap file and packet format, ethernet headers and ipv4 headers. The ipv4 header is defined in two different ways for big and little endian machines, because it relies on bit fields.

We get the whole IPv4 payload as a UTF-8 data, decode it, replace control characters with dots and print.

Edit May 24 2022: New code which takes advantage of the endian bitfield support in TXR 276. A bitfield may now be based on an endian type like be-uint16, so we no longer need to declare the IPv4 header structure in two ways depending on the endian of the host:

(typedef ll-t (enumed uint32 ll-t
                DLT_NULL DLT_EN10MB))

(typedef eth-t (enumed be-uint16 eth-t
                 (ETH_IPV4 #x0800)
                 (ETH_ARP  #x0806)
                 (ETH_IPV6 #x08DD)))

(typedef pcap-header (struct pcap-header
                       (magic uint32)
                       (majver uint16)
                       (minver uint16)
                       (tzoffs uint32)
                       (tzprec uint32)
                       (snaplen uint32)
                       (lltype ll-t)))

(typedef pkt-header (struct pkt-header
                      (tsec uint32)
                      (tfrac uint32)
                      (trunclen uint32)
                      (origlen uint32)))


(typedef eth-header (struct eth-header
                      (dst-mac (array 6 uint8))
                      (src-mac (array 6 uint8))
                      (eth-type eth-t)))

(typedef ipv4-header (struct ipv4-header
                       (ver (bit 4 be-uint16))
                       (ihl (bit 4 be-uint16))
                       (dscp (bit 6 be-uint16))
                       (ecn (bit 2 be-uint16))
                       (len be-uint16)
                       (ident be-uint16)
                       (flags (bit 3 be-uint16))
                       (fragoffs-hi (bit 5 be-uint16))
                       (fragoffs-lo uint8)
                       (ttl uint8)
                       (proto uint8)
                       (hdr-sum be-uint16)
                       (src-ip be-uint32)
                       (dst-ip be-uint32))))

;; Look for IPv4 packets and print headers
(defun decode-packet (phdr buf)
  (let ((eh (ffi-get buf (ffi eth-header))))
    (unless (eq eh.eth-type 'ETH_IPV4)
      (return-from decode-packet))
    (let* ((ih (ffi-get buf (ffi ipv4-header) (sizeof eth-header)))
           (hdrsz (+ (sizeof eth-header) (sizeof ipv4-header)))
           (len (- (length buf) hdrsz))
           (body (carray-buf buf (ffi char) hdrsz))
           (rawtext (carray-get body))
           (text (mapcar (iffi [andf chr-iscntrl [notf chr-isspace]] (ret #\.))
                         rawtext)))
      (put-line `@(str-inaddr ih.src-ip) --> @(str-inaddr ih.dst-ip)`)
      (put-line `  ether hdr: @eh`)
      (put-line `  ipv4 hdr:  @ih`)
      (put-line `  ipv4 payload as text: @text`))))

;; main program
(let ((*stdin* (open-fileno (fileno *stdin*) "rbu")) ;; binary, unbuffered
      (hdr (new pcap-header))
      (hdr-buf (make-buf (sizeof pcap-header)))
      (phdr (new pkt-header))
      (phdr-buf (make-buf (sizeof pkt-header)))
      (pay-buf (make-buf 65536)))

  ;; read pcap file header
  (when (< (fill-buf hdr-buf) (sizeof pcap-header))
    (return))

  ;; decode to structure
  (ffi-in hdr-buf hdr (ffi pcap-header) t)

  (unless (eq hdr.lltype 'DLT_EN10MB)
    (put-line "can only deal with Ethernet frames")
    (exit nil))

  ;; read and decode packets
  (while t
    (when (< (fill-buf phdr-buf) (sizeof pkt-header))
      (return))
    (ffi-in phdr-buf phdr (ffi pkt-header) t)
    (buf-set-length pay-buf phdr.trunclen)
    (when (< (fill-buf pay-buf) phdr.trunclen)
      (return))
    (decode-packet phdr pay-buf)))

body is bound to a carray object overlaid onto buf, with a displacement of hdrsz, to ship the ethernet and IPV4 header. The element type is char. It takes up the entire rest of the buffer after the header.

Then (carray-get body) turns the entire foreign value into a Lisp string. Because the element type is char, UTF-8 conversion kicks in: a special behavior for arrays of char. If the type were bchar, it would just take the bytes as characters 1:1. If the type were wchar, then the array would be of wchar_t characters, converted to a string accordingly. To get a vector of numeric bytes instead of a string, we can make the element type uchar or uint8.

This program is easy to extend to handle TCP, UDP, IPv6, whatever is needed. It can look for specific matches on specific header fields.