5

I tried both netstat and lsof, but it appears it's not possible to see the connections to my LXC guests.

Is there a way to achieve this ... for all guests at once?


Essentially what throws me off here is the fact that I can see the processes of the guests as long as I run as superuser. I can also see the veth interfaces that get dynamically created per guest. Why can I not see connections on processes that are otherwise visible?

2

I think https://stackoverflow.com/a/40352004/1951468 should answer you.

So basically, the only way is a loop around your containers and nsenter.

2

The kernel indicate the connections state on /proc/net/tcp, /proc/net/udp etc. but as the namespaces separate the network stack if an application is running inside a container (a different userspace) and is connected to the network the host /proc/net/tcp won't show its connection,

conntrack can be used to show the whole machine connection but this does not work for some interfaces like wireguard...

ip -all netns exec command can be used to run commands inside all the userspaces but this is limited to userspaces created with ip command.

On the perspective of an application running on a container its network stack state is still visible on the host on the location /proc/$pid/net/tcp so as a workaround awaiting to write a proper tool in c, i wrote a little bash script that loop on /proc/$pid/net/tcp[udp] and join all the states to be able to list the whole machine connection.

The script first join all /proc/$pid/net/tcp or /proc/$pid/net/udp sort them, remove duplicate, translate the value to a readable text and print them (the script require find, grep, xargs, awk, strtonum, sort and uniq)

For TCP

find /proc/ 2>/dev/null | grep tcp | grep -v task | grep -v sys/net | xargs grep -v rem_address 2>/dev/null | awk '{x=strtonum("0x"substr($3,index($3,":")-2,2)); y=strtonum("0x"substr($4,index($4,":")-2,2)); for (i=5; i>0; i-=2) x = x"."strtonum("0x"substr($3,i,2)); for (i=5; i>0; i-=2) y = y"."strtonum("0x"substr($4,i,2))}{printf ("%s\t:%s\t ----> \t %s\t:%s\t%s\n",x,strtonum("0x"substr($3,index($3,":")+1,4)),y,strtonum("0x"substr($4,index($4,":")+1,4)),$1)}' | sort | uniq --check-chars=25

For UDP

find /proc/ 2>/dev/null | grep udp | grep -v task | grep -v sys/net | xargs grep -v rem_address 2>/dev/null | awk '{x=strtonum("0x"substr($3,index($3,":")-2,2)); y=strtonum("0x"substr($4,index($4,":")-2,2)); for (i=5; i>0; i-=2) x = x"."strtonum("0x"substr($3,i,2)); for (i=5; i>0; i-=2) y = y"."strtonum("0x"substr($4,i,2))}{printf ("%s\t:%s\t ----> \t %s\t:%s\t%s\n",x,strtonum("0x"substr($3,index($3,":")+1,4)),y,strtonum("0x"substr($4,index($4,":")+1,4)),$1)}' | sort | uniq --check-chars=25

The output look like: (note that the pid is not accurate and is just used to identify the container)

127.0.0.1      :80       ---->   0.0.0.0        :0      /proc/10176/net/tcp:
192.168.0.2    :33882    ---->   192.30.253.125 :443    /proc/10176/net/tcp
192.168.0.2    :34020    ---->   192.30.253.125 :443    /proc/10176/net/tcp:
192.168.0.2    :34162    ---->   192.30.253.125 :443    /proc/10176/net/tcp:
192.168.0.2    :36242    ---->   192.30.253.124 :443    /proc/10176/net/tcp:
192.168.0.2    :37324    ---->   192.30.253.124 :443    /proc/10176/net/tcp:
192.168.0.2    :40122    ---->   216.239.38.21  :80     /proc/10176/net/tcp:
192.168.0.2    :40124    ---->   216.239.38.21  :80     /proc/10176/net/tcp:

Also i found a great tool for managing namespace with some very useful commands that is nsutils

1

I think you can use sudo conntrack -L

conntrack is a userspace command line program targeted at system administrators. It enables them to view and manage the in-kernel connection tracking state table.

If not installed, the required package is conntrack-tools (on Fedora etc.) or conntrack (on Debian, Ubuntu etc.).

  • Conntrack could also be queried by parsing /proc/net/ip_conntrack, which is effectively what your suggested program does. But both lsof and netstat use kernel information, for that matter, so what's the difference? – 0xC0000022L Mar 4 at 20:29
  • 1
    If run from host conntrack shows connections established inside containers. Both netstat and lsof does not show that info. – suside Mar 6 at 7:09
  • This does not work for me, it show the connections but not the namespaces connections... – intika May 27 at 23:59
  • @intika this is very strange, are you 100% sure there is active connection inside container? Can you leave ping running inside and verify again? Also in Ubuntu there is conntrack package not conntrack-tools. – suside May 28 at 12:02
  • Yes, ip -all netns exec ss show those connections, and lsmod | grep conntrack show nf_conntrack, xt_conntrack, nf_conntrack_ipv4 & nf_conntrack_* etc also i dont have /proc/net/ip_conntrack; conntrack -L does show the host connections but not those established inside other namespaces, and finally running conntrack -L from within other namespaces return an empty list @suside – intika May 28 at 12:15
0

A similar workaround is to execute the netstat/lsof command on all network namespace with

sudo ip -all netns exec ss -p -ut 

or

sudo ip -all netns exec lsof -i

or

sudo ip -all netns exec netstat -ltup 
-2

So far I had luck getting the container connections by running netstat from the container:

sudo docker exec -it <containerIdOrName> netstat

Obviously, the container needs to have it installed.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.