Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.
Sign up to join this community
I tried both netstat and lsof, but it appears it's not possible to see the connections to my LXC guests.
Is there a way to achieve this ... for all guests at once?
Essentially what throws me off here is the fact that I can see the processes of the guests as long as I run as superuser. I can also see the veth interfaces that get dynamically created per guest. Why can I not see connections on processes that are otherwise visible?
The kernel indicate the connections state on /proc/net/tcp, /proc/net/udp etc. but as the namespaces separate the network stack if an application is running inside a container (a different userspace) and is connected to the network the host /proc/net/tcp won't show its connection,
conntrack can be used to show the whole machine connection but this does not work for some interfaces like wireguard...
ip -all netns exec command can be used to run commands inside all the userspaces but this is limited to userspaces created with ip command.
On the perspective of an application running on a container its network stack state is still visible on the host on the location /proc/$pid/net/tcp so as a workaround awaiting to write a proper tool in c, i wrote a little bash script that loop on /proc/$pid/net/tcp[udp] and join all the states to be able to list the whole machine connection.
The script first join all /proc/$pid/net/tcp or /proc/$pid/net/udp sort them, remove duplicate, translate the value to a readable text and print them (the script require find, grep, xargs, awk, strtonum, sort and uniq)
For TCP
find /proc/ 2>/dev/null | grep tcp | grep -v task | grep -v sys/net | xargs grep -v rem_address 2>/dev/null | awk '{x=strtonum("0x"substr($3,index($3,":")-2,2)); y=strtonum("0x"substr($4,index($4,":")-2,2)); for (i=5; i>0; i-=2) x = x"."strtonum("0x"substr($3,i,2)); for (i=5; i>0; i-=2) y = y"."strtonum("0x"substr($4,i,2))}{printf ("%s\t:%s\t ----> \t %s\t:%s\t%s\n",x,strtonum("0x"substr($3,index($3,":")+1,4)),y,strtonum("0x"substr($4,index($4,":")+1,4)),$1)}' | sort | uniq --check-chars=25
For UDP
find /proc/ 2>/dev/null | grep udp | grep -v task | grep -v sys/net | xargs grep -v rem_address 2>/dev/null | awk '{x=strtonum("0x"substr($3,index($3,":")-2,2)); y=strtonum("0x"substr($4,index($4,":")-2,2)); for (i=5; i>0; i-=2) x = x"."strtonum("0x"substr($3,i,2)); for (i=5; i>0; i-=2) y = y"."strtonum("0x"substr($4,i,2))}{printf ("%s\t:%s\t ----> \t %s\t:%s\t%s\n",x,strtonum("0x"substr($3,index($3,":")+1,4)),y,strtonum("0x"substr($4,index($4,":")+1,4)),$1)}' | sort | uniq --check-chars=25
The output look like: (note that the pid is not accurate and is just used to identify the container)
127.0.0.1 :80 ----> 0.0.0.0 :0 /proc/10176/net/tcp:
192.168.0.2 :33882 ----> 192.30.253.125 :443 /proc/10176/net/tcp
192.168.0.2 :34020 ----> 192.30.253.125 :443 /proc/10176/net/tcp:
192.168.0.2 :34162 ----> 192.30.253.125 :443 /proc/10176/net/tcp:
192.168.0.2 :36242 ----> 192.30.253.124 :443 /proc/10176/net/tcp:
192.168.0.2 :37324 ----> 192.30.253.124 :443 /proc/10176/net/tcp:
192.168.0.2 :40122 ----> 216.239.38.21 :80 /proc/10176/net/tcp:
192.168.0.2 :40124 ----> 216.239.38.21 :80 /proc/10176/net/tcp:
Also i found a great tool for managing namespace with some very useful commands that is nsutils
I think https://stackoverflow.com/a/40352004/1951468 should answer you.
So basically, the only way is a loop around your containers and nsenter.
I think you can use sudo conntrack -L
conntrack is a userspace command line program targeted at system administrators. It enables them to view and manage the in-kernel connection tracking state table.
If not installed, the required package is conntrack-tools (on Fedora etc.) or conntrack (on Debian, Ubuntu etc.).
/proc/net/ip_conntrack, which is effectively what your suggested program does. But both lsof and netstat use kernel information, for that matter, so what's the difference?
Mar 4, 2019 at 20:29
conntrack shows connections established inside containers. Both netstat and lsof does not show that info.
ping running inside and verify again? Also in Ubuntu there is conntrack package not conntrack-tools.
ip -all netns exec ss show those connections, and lsmod | grep conntrack show nf_conntrack, xt_conntrack, nf_conntrack_ipv4 & nf_conntrack_* etc also i dont have /proc/net/ip_conntrack; conntrack -L does show the host connections but not those established inside other namespaces, and finally running conntrack -L from within other namespaces return an empty list @suside
A similar workaround is to execute the netstat/lsof command on all network namespace with
sudo ip -all netns exec ss -p -ut
or
sudo ip -all netns exec lsof -i
or
sudo ip -all netns exec netstat -ltup
So far I had luck getting the container connections by running netstat from the container:
sudo docker exec -it <containerIdOrName> netstat
Obviously, the container needs to have it installed.
