How to record an interactive socat TCP/TLS session?

Question

Socat is great for interactively testing line based human readable protocols like HTTP or IMAP.

For example:

$ socat -d -d READLINE,history=$HOME/s.hist openssl:host:port,crnl,cafile=some.ca

For better analyzing I need to capture such an interactive session - i.e. the bytes received and sent.

Just hardcopying the terminal output via e.g. tux is not enough, because client/server parts are not marked and characters like '\t' are lost/silently converted.

Using tcpdump to capture helps only for unencrypted connections.

Thus my question.

The answer does not have to be socat-based. If another tool is better suited for that use case I would like to read about it.

Bonus points for a solution where

• time stamps are recorded as well
• one can chose between interleaved recording (client/server side) or logging to separate files

Answer 1

That's typically what the -v and -x options of socat are for.

-v dumps the data to stderr with some transformation like the CR character becomes \r so you can see it.

-x does a hex dump (not very useful though as there's no timestamp or indication of what direction the dumped traffic is flowing in).

With -v and -x combined, you get a hd type of dump with timestamp and direction:

> 2014/03/08 08:46:23.370824  length=4 from=0 to=3
 61 64 73 0a                                      ads.
--
< 2014/03/08 08:46:23.398666  length=1371 from=0 to=1370
 48 54 54 50 2f 31 2e 30 20 34 30 30 20 42 61 64  HTTP/1.0 400 Bad
 20 52 65 71 75 65 73 74 0d 0a                     Request..
 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 74 65  Content-Type: te
[...]

If you want it dumped to a file, you can redirect stderr to a file and the debugging output redirected to stdout with:

socat -d -d -lf /dev/stdout -x -v 2> dump.txt \
   "READLINE,history=$HOME/s.hist" \
   openssl:host:port,crnl,cafile=some.ca

You can also convert that to a pcap with text2pcap (that comes with wireshark) after a little post-processing, using:

{
  socat -d -d -lf /dev/stdout -x -v 2>&1 >&3 3>&- \
    "READLINE,history=$HOME/s.hist" \
    openssl:host:port,crnl,cafile=some.ca |
    awk '/^[<>]/{a=0;print $1 == "<" ? "I" : "O", $2, $3; next}
         {$0 = substr($0, 1, 48);printf "%.4x %s\n",a,$0;a+=NF}' |
    text2pcap -l 147 -Dnqt '%Y/%m/%d %H:%M:%S.' - dump.pcap
} 3>&1

That pcap using encapsulation 147. That's not a link layer type, but wireshark understands it as a user encapsulation.

Then, we can tell wireshark that the packets contain HTTP traffic for instance with:

wireshark -o 'uat:user_dlts:"User 0 (DLT=147)","http","0","","0",""' \
  -o 'gui.column.format:"No.","%m","Time","%Yt",
    "Direction","%Cus:frame.packet_flags_direction:0:R","Protocol","%p",
    "Length","%L","Information", "%i"' -r dump.pcap

(here also modifying the displayed columns as well, since there's no address, but we have the traffic direction (provided by text2pcap -D after we've converted the <, > to I, O)).

You can see the traffic live with wireshark by piping the text2pcap output to wireshark called with -ki - instead of -r dump.pcap, but then wireshark only supports the old pcap format, not pcap-ng which means we have to drop the -n option of text2pcap and we lose the information of the direction:

{
  socat -d -d -lf /dev/fd/3 -x -v 2>&1 >&3 3>&- \
    "READLINE,history=$HOME/s.hist" \
    openssl:host:port,crnl,cafile=some.ca |
    awk '/^[<>]/{a=0;print $1 == "<" ? "I" : "O", $2, $3; next}
         {$0 = substr($0, 1, 48);printf "%.4x %s\n",a,$0;a+=NF}' |
    text2pcap -l 147 -Dqt '%Y/%m/%d %H:%M:%S.' - - |
    wireshark -o 'uat:user_dlts:"User 0 (DLT=147)","http","0","","0",""' \
      -o 'gui.column.format:"No.","%m","Time","%Yt",
        "Direction","%Cus:frame.packet_flags_direction:0:R","Protocol","%p",
        "Length","%L","Information", "%i"' -ki -
} 3>&1

Answer 2

I quite like tcpdump for recording network connections. You actually can use it for what you want to achieve. Instead of using the READLINE endpoint in your socat connection, make it listen to some port.

remote server with ssl
     ^
     |       (ssl-encrypted)
   socat
     |       (not ssl-encrypted)
     v
local port   <-- run tcpdump here
     ^
     |       
   socat
     |
     v
 your terminal

You then use a second socat connection to connect to the local port, where the first socat is listening. This is unencrypted. And on this port you can run tcpdump.

$ # easiest to use a separate terminal window for each command
$ socat TCP-LISTEN:9000,reuseaddr openssl:host:port,cafile=some.ca
$ tcpdump -i lo -w /tmp/tcpdump.output port 9000
$ socat READLINE,history=$HOME/.socat.hist TCP:localhost:9000

Answer 3

One kind of ghetto-style approach is using ltrace on socat:

$ ltrace -s $((100*1024)) -e memcpy@libssl.so'*' -o s.log \
    socat -d -d READLINE,history=$HOME/.socat.hist \
      openssl:host:port,crnl,cafile=some.ca

Then you have to identify send/receive buffers using a common string:

$ grep -i logout s.log
libssl.so.10->memcpy(0xd3bd30, "a5 logout\r\n", 11)               = 0xd3bd30
libssl.so.10->memcpy(0xd32230, "a5 OK Logout completed.\r\n", 25) = 0xd32230

Those addresses can then be used to separate the client/server parts of the communication:

$ grep 0xd3bd30 s.log | \
    sed 's/^[^(]\+([0-9a-fx]\+. \(".*"\), [0-9]\+) *= [0-9a-fx]\+$/\1/'

(prints the client/send side - for the server side use the other pointer)

On the plus side, ltrace also supports multiple options for timestamped output.