10

I've a question about docker containers and sockets. Today i've discovered that, docker containers with mapped port -p 80:80 (or etc ports) on the host system, doesn't show up when i use netstat, ss or lsof -i (also tried with sudo, no chance). Then i tried to test, open a listener port 80 with sudo nc -l -p 80, but no chance, it refused to connect port 80, although this port doesn't show up opened by some process.

I think this problem occurs due to containerization or network namespaces, but there should be some commands to list used sockets by low level, kernel level?

p.s: When I tried to use docker host network interface ip address on port mapping (-p 172.17.0.1:80:80), it showed up using above commands.

p.s-2: lsof | grep -w 80 output:

lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
      Output information may be incomplete.
Xorg        718                             root   37u      CHR              13,80       0t0      10891 /dev/input/event16
Xorg        718   749 Xorg:disk             root   37u      CHR              13,80       0t0      10891 /dev/input/event16
Xorg        718   778 Xorg:disk             root   37u      CHR              13,80       0t0      10891 /dev/input/event16
Xorg        718   787 Xorg:disk             root   37u      CHR              13,80       0t0      10891 /dev/input/event16
Xorg        718   841 InputThre             root   37u      CHR              13,80       0t0      10891 /dev/input/event16
11
  • What does docker inspect say? is it using bridge mode or is it actually providing a complete new IP address? Commented Sep 9, 2019 at 14:19
  • Perhaps the port is routed. Since no local process listens on it no process is shown.
    – Ned64
    Commented Sep 9, 2019 at 14:24
  • Can you see the mapped port by something like lsof | grep -w 80 (as root)?
    – Ned64
    Commented Sep 9, 2019 at 14:29
  • @TheresaForster , i've uploaded the inspect reults here: ctxt.io/2/AABAtIUXEg Commented Sep 9, 2019 at 15:01
  • 1
    @SafarSafarli Please do not reference external sources in your Question because they can be deleted. Edit your Question to include these details because it should be self-contained. Also, do not use images to represent text because images cannot be searched for text content (search function on SE).
    – Ned64
    Commented Sep 9, 2019 at 18:01

1 Answer 1

2

Docker makes the system behave as a router between outside and the running containers: a router doesn't use sockets to perform routing (only for its own services or accessing remote services). That's why container connections don't appear with netstat or ss. With lsof they fail to appear for a similar reason: while lsof can display container's running processes and determine some of its file descriptors are sockets, it will fail to associate a socket within the local network stack's sockets (since they are in an other network namespace) so will fail to discover what these sockets really are.

The only case where Docker can create local sockets for its containers is when it's running with the option to use the docker-proxy process to handle NAT hairpinning. The more recent default configuration doesn't use this anymore, because it can all be handled with iptables.

Docker also usually performs NAT. To do this it relies on iptables and Netfilter's conntrack to memorize the NAT associations. The conntrack entries can be queried with the conntrack tool (usually available in a package named conntrack or conntrack-tools).

For Docker's default mode: NAT, it's very easy to know traffic behind Docker. Any entry where the reply destination is different from the query source means a SNAT (incl MASQUERADE) was performed. Any conntrack entry where the reply source is different from the query destination means a DNAT (incl REDIRECT) was performed. conntrack can do this with adequate options:

conntrack -L --src-nat
conntrack -L --dst-nat
conntrack -L --any-nat

The first would usually show outgoing connection associations from containers to outside which are being MASQUERADE-d. The second would usually show incoming connection associations from outside to the containers which are being DNAT-ed, the 3rd would show everything. Of course, one can add more filters.

Using -E instead of -L can monitor such associations in event mode.

Of course if Docker is configured to not perform NAT, conntrack is still used for stateful firewalling so just not providing any filter option will still display sources and destinations.

Then for any IP address belonging to container namespace's addresses (eg typically 172.17.0.0/16 for docker0 , then further 172.18.0.0/16 or as it was configured), one can correlate this with information from Docker to figure out the actual destination container.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .