80

Looking in /proc/$mypid/fd/, I see these files

lrwx------ 1 cm_user cm_user 64 Oct 14 03:21 0 -> /dev/pts/36 (deleted)
lrwx------ 1 cm_user cm_user 64 Oct 14 03:21 3 -> socket:[1424055856]
lrwx------ 1 cm_user cm_user 64 Oct 14 03:21 4 -> socket:[1424055868]
lrwx------ 1 cm_user cm_user 64 Oct 14 03:21 5 -> socket:[1424055882]

Because I have access to the code, I know these sockets are tied to TCP connections (one is a connection to port 5672 on some machine, another is a connection to port 3306 on some other machine), but I want to know which socket is tied to which connection. How can I do that?

More generally, how can I ask the OS what is at the other end of the socket?

Improve this question
4

2 Answers 2

Reset to default
90

Command lsof

A good option might be lsof. As man lsof states it is handy for obtaining information about open files such as Internet sockets or Unix Domain sockets.

Using it

At first, get an overview about /proc/$PID/fd/ and the listed socket numbers.
For example, socket:[14240] might interest you.

Then use lsof -i -a -p $PID to print a list of all network files $PID uses.

  • -i produces a list of network files belonging to a user or process

  • -a logically combines or AND's given parameters

  • -p $PID selects info only about your process

A typical output for my browser running with a PID of 2543 might be: 

COMMAND  PID  USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
browser 2543 pidi   55u  IPv4  14240      0t0  TCP  pidi.router.lan:55038->stackoverflow.com:https (ESTABLISHED)

and more similar lines.

Great! Now take a closer look at the DEVICE column. It matches our previously listed socket from /proc/$PID/fd/!
And thanks to the NAME section we can say what the other end of our socket is.

In a real world run you might get a good amount of output, but just filter or grep for your socket of interest.

I'm pretty sure one could combine all commands, but that should be enough to get you started.

Improve this answer
5
  • 1
    Thank you! Upvoted and accepted. I do see a socket showing up in lsof as "unix 0xffff88002704d380 0t0 1423733559 socket", how do I dig into what 0xffff88002704d380 is?
    – benhsu
    Commented Oct 14, 2015 at 12:30
  • 2
    "unix 0xfff ..." seems like the kernel memory address of a unix_sock structure. This structure contains a pointer to your socket-end point. Take a look at grep -rni ffff88002704d380 /proc. somewhere under /proc/ may lie your answer, cat your file grep found. But dont pin me down on this... May be horrible wrong here.
    – pidi
    Commented Oct 14, 2015 at 13:58
  • 2
    lsof reads other information about the socket from /proc/net/tcp, /proc/net/tcp6, /proc/net/udp, and so on depending on socket type.
    – Jesin
    Commented Nov 15, 2017 at 22:48
  • Does not work for me. I think you can reproduce it (at least I can) by first doing sudo ls -l /proc/$(pidof wpa_supplicant)/fd | grep socket | wc -l to confirm the process has "socket" descriptors. For me it has 17 of them. Then you do sudo lsof -i -a -p $(pidof wpa_supplicant), and… nothing. Well, I got two warnings about can't stat() fuse.gvfsd-fuse and can't stat() fuse.portal, but I doubt they're relevant. For one, wpa_supplicant has stdout/stderr redirected to some socket (which is what I'm mainly interested in), and I don't think these "fuse" thingies could have a relation.
    – Hi-Angel
    Commented Oct 2, 2020 at 8:48
  • 3
    Okay, I figured: you need to replace -i option of the lsof command with -U. I.e. you use sudo lsof -U -a -p $(pidof wpa_supplicant). Unfortunately combining -i doesn't seem to work, so one have to run both commands while searching for socket files. But turns out there's a better way! You have to use ss -apn | grep your_socket — unlike lsof, the ss command shows all types of sockets, unless you explicitly restrict it.
    – Hi-Angel
    Commented Oct 2, 2020 at 9:09
1

As noted by Marki555’s answer to a question linked in a comment on this question: in addition to the lsof option, you can correlate the socket number to an inode in /proc/net, which gives details about the connection.

For example, I opened an SSH session to 1.1.1.1 to generate a TCP connection for process 2798586:

~$ ssh 1.1.1.1 &
[1] 2798586

Then I obtained the socket associated with that process, 10750519:

$ ls -al /proc/2798586/fd
total 0
dr-x------ 2 oshi oshi  4 May 18 19:13 .
dr-xr-xr-x 9 oshi oshi  0 May 18 19:13 ..
lrwx------ 1 oshi oshi 64 May 18 19:13 0 -> /dev/pts/1
lrwx------ 1 oshi oshi 64 May 18 19:13 1 -> /dev/pts/1
lrwx------ 1 oshi oshi 64 May 18 19:13 2 -> /dev/pts/1
lrwx------ 1 oshi oshi 64 May 18 19:13 3 -> 'socket:[10750519]'

I can find that entry as an inode in /proc/net/tcp (other connection output omitted):

  sl  local_address rem_address   st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode                                                     
   8: C80AA8C0:A640 01010101:0016 02 00000001:00000000 01:000002B6 00000008 61873        0 10750519 2 00000000bfa91f11 1600 0 0 1 7
(note →)                                                                                   ^^^^^^^^

I can see my ssh remote address (01010101:0016 is 1.1.1.1:22) as well as local address.  Kernel documentation describes all the rest of those fields, including connection state, transmit and received queues, UID of the owner, and more.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .