2

My initial state is something like this:

  • A "normal" home network with a residential gateway that hands out DHCP addresses to a combination of wired and wireless machines on subnet 192.168.15.0/24.
  • A linux (Ubuntu 18.04.3) virtual machine host with a mix of windows and linux guests. This VM host uses a single bridge interface to directly bridge the VM guest interfaces into the 192.168.15.x network. Here eno1 and ens2 are physical ethernet ports connecting the VM host to the main network and "extending" this bridge domain to another physical switch, and the vnetX interfaces are the interfaces of the VM guest machines.
lwobker@lwobker-vms:~$ brctl show 
bridge name     bridge id               STP enabled     interfaces
bridge1         8000.0017b60066e8       no              eno1    # goes to router/gateway
                                                        ens2    # extends the subnet to other machines
                                                        vnet0   # VM guest #1
                                                        vnet1   # VM guest #2
docker0         8000.0242bd3d4632       no
virbr0          8000.52540096aaf5       yes             virbr0-nic

This all worked as I wanted it to, where local wired, wireless, and VM guest clients could all get DHCP addresses from the main 192.168.15.1 res-gateway/router, some of which were reserved/static on the DHCP server side and some were dynamic. Anyway - all good... UNTIL...

I installed Docker. I did this to run the Ubiquity/Unifi wireless AP controller software in a container, it seemed like a good/simple use case to play around with containers. However, somewhere along the way the installation of Docker (and I'm pretty sure more accurately the Docker networking changes) broke DHCP for anything BEHIND the VM host machines bridge interface. To be explicit:

  • wired and wireless clients that do NOT sit behind the VM host machine still receive DHCP addresses normally from 192.168.15.1 (both reserved and dynamic)
  • all the machines that sit behind the VM host bridge no longer receive DHCP addresses. This includes BOTH virtual machine guest instances on the VM itself, as well as machines that are physically cabled through interface ens2.
  • importantly: IPv6 addressing does still work on machines behind this newly-Dockerfied VM host, and these machines can still get out to the internet successfully as long as the target is a routable v6 address.

I'm obviously new to docker and have a decent grasp of linux networking and bridging, but this is past my abilities. I've fired up tcpdump a couple of times and I can see the DHCP From reading around it may well be something related to iptables - I do not have the output of iptables --list from before the docker install, but there are obviously entries that were added ;-) I've included whatever I can convince myself is relevant below and tried to annotate entries that are particularly important. Help?!?!!

lwobker@lwobker-vms:~$ brctl showmacs bridge1 | uniq
port no mac addr                is local?       ageing timer
1     18:1d:ea:8a:86:e9       no                49.83
1     44:61:32:d0:43:02       no               117.27
1     44:61:32:fb:4d:7b       no               117.27
1     60:6d:c7:1a:8f:e1       no                 4.47
1     78:f2:9e:90:4c:a1       no                 1.21
1     7c:2e:bd:9c:4a:4a       no                 0.00
1     80:4a:14:ec:5a:ea       no                95.33
1     80:4a:14:f3:d8:8c       no                71.62
1     90:70:65:13:b7:16       no               117.16
1     a0:cc:2b:ff:a4:b4       no                 0.70
1     ac:1f:6b:b3:ad:fa       yes                0.00
1     ac:1f:6b:b7:d2:44       no               286.56
1     b4:fb:e4:d6:e2:35       no                 2.54
1     b8:27:eb:f9:c6:fe       no               117.18
1     d8:31:34:f3:2c:69       no                62.38
1     ec:11:27:58:a8:0d       no                 0.22
2     00:17:b6:00:66:e8       yes                0.00
2     00:30:93:10:05:8e       no                 3.61     >> physical machine behind intf ens2
3     52:54:00:ed:c9:fc       no                 0.67     >> linux VM guest
3     fe:54:00:ed:c9:fc       yes                0.00
4     52:54:00:d9:2d:b0       no                 1.29     >> windows VM guest
4     fe:54:00:d9:2d:b0       yes                0.00

Current iptables output:

lwobker@lwobker-vms:/storage/unifi$ sudo iptables --list
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     udp  --  anywhere             anywhere             udp dpt:domain
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:domain
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootps
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:bootps
ACCEPT     udp  --  anywhere             anywhere             multiport dports mdns
ACCEPT     tcp  --  anywhere             anywhere             multiport dports 4000

Chain FORWARD (policy DROP)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             192.168.122.0/24     ctstate RELATED,ESTABLISHED
ACCEPT     all  --  192.168.122.0/24     anywhere            
ACCEPT     all  --  anywhere             anywhere            
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
DOCKER-ISOLATION-STAGE-1  all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootpc

Chain DOCKER (1 references)
target     prot opt source               destination         

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target     prot opt source               destination         
DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere            
RETURN     all  --  anywhere             anywhere            

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target     prot opt source               destination         
DROP       all  --  anywhere             anywhere            
RETURN     all  --  anywhere             anywhere            



lwobker@lwobker-vms:/storage/unifi$ docker ps
CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS                 PORTS               NAMES
be342a77a105        jacobalberty/unifi:stable   "/usr/local/bin/dock…"   23 hours ago        Up 5 hours (healthy)                       unifi

Output from docker network inspect for the bridge...

lwobker@lwobker-vms:/storage/unifi$ docker network inspect 20d74e3d7efc
[
    {
        "Name": "bridge",
        "Id": "20d74e3d7efc41320c054dbd4fde76808a7c0e021e737e22cfceebefacb24b8c",
        "Created": "2019-10-15T12:36:22.059240054-04:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.17.0.0/16",
                    "Gateway": "172.17.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {},
        "Options": {
            "com.docker.network.bridge.default_bridge": "true",
            "com.docker.network.bridge.enable_icc": "true",
            "com.docker.network.bridge.enable_ip_masquerade": "true",
            "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
            "com.docker.network.bridge.name": "docker0",
            "com.docker.network.driver.mtu": "1500"
        },
        "Labels": {}
    }
]

the ifconfig output from the VM host...

lwobker@lwobker-vms:~$ ifconfig | egrep -v 'errors|0.0 B|device mem'
bridge1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.15.150  netmask 255.255.255.0  broadcast 192.168.15.255
        ether 00:17:b6:00:66:e8  txqueuelen 1000  (Ethernet)
        RX packets 105730  bytes 73076273 (73.0 MB)
        TX packets 47591  bytes 11607902 (11.6 MB)

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:bd:3d:46:32  txqueuelen 0  (Ethernet)

eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether ac:1f:6b:b3:ad:fa  txqueuelen 1000  (Ethernet)
        RX packets 479110  bytes 451330614 (451.3 MB)
        TX packets 352901  bytes 284906323 (284.9 MB)

eno2: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether ac:1f:6b:b3:ad:fb  txqueuelen 1000  (Ethernet)

eno2:avahi: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 169.254.11.170  netmask 255.255.0.0  broadcast 169.254.255.255
        ether ac:1f:6b:b3:ad:fb  txqueuelen 1000  (Ethernet)

ens2: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 00:17:b6:00:66:e8  txqueuelen 1000  (Ethernet)
        RX packets 375663  bytes 281196261 (281.1 MB)
        TX packets 292663  bytes 270964344 (270.9 MB)

ens2:avahi: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 169.254.8.164  netmask 255.255.0.0  broadcast 169.254.255.255
        ether 00:17:b6:00:66:e8  txqueuelen 1000  (Ethernet)

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 92697  bytes 34802204 (34.8 MB)
        TX packets 92697  bytes 34802204 (34.8 MB)

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 52:54:00:96:aa:f5  txqueuelen 1000  (Ethernet)

virbr0-nic: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 52:54:00:96:aa:f5  txqueuelen 1000  (Ethernet)

vnet0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether fe:54:00:ed:c9:fc  txqueuelen 1000  (Ethernet)
        RX packets 4563  bytes 509949 (509.9 KB)
        TX packets 28463  bytes 2796658 (2.7 MB)

vnet1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether fe:54:00:d9:2d:b0  txqueuelen 1000  (Ethernet)
        RX packets 50067  bytes 5253268 (5.2 MB)
        TX packets 82339  bytes 105281318 (105.2 MB)

vnet0:avahi: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 169.254.12.44  netmask 255.255.0.0  broadcast 169.254.255.255
        ether fe:54:00:ed:c9:fc  txqueuelen 1000  (Ethernet)

vnet1:avahi: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 169.254.8.4  netmask 255.255.0.0  broadcast 169.254.255.255
        ether fe:54:00:d9:2d:b0  txqueuelen 1000  (Ethernet)
1
  • Can you tell if sudo sysctl net.bridge.bridge-nf-call-iptables returns net.bridge.bridge-nf-call-iptables = 1 ? If not forget about this comment, if yes, you might have issues related to bridge plus iptables (yes the site's cert has expired), and it's hard to debug without all your configs (use iptables-save -c to dump all of iptables)
    – A.B
    Oct 16 '19 at 6:35
0

A quick fix:

iptables -t filter -P FORWARD ACCEPT

A better fix:

iptables -A FORWARD -p udp --sport 68 --dport 67 -m conntrack --ctstate NEW -j ACCEPT

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.