1

I have a tunnel between two hosts that is carried over L2TPv3. Each end of the tunnel is slaved to a bridge, as are other interfaces. The remote bridge has a DHCP server attached to it while the local bridge has a DHCP client. To test this, I create a veth pair on the host at my end. I slave one half of the pair to the bridge and put the other into a new network namespace. In that namespace, I then start a DHCP client on the other half of the pair. Like this:

ip netns add test
ip link add test1 type veth peer test2
ip link set test2 netns test up
ip link set test1 master tunnel_bridge up
ip netns exec test dhclient -d test2

This all works as expected - a DHCP lease for the test2 interface is obtained and configured. The already-running DHCP client on the tunnel_bridge interface also obtains an address.

Now I'm trying to replace the L2TPv3 tunnel with a VxLAN tunnel. There's no reason that the VxLAN should have only two peers, but in this case it does. The VxLAN is configured for unicast with static flooding.

Now, traffic between the bridges is fine; the local bridge can get an address by DHCP and can ping the remote bridge. But DHCP replies from the DHCP server don't propagate off the local bridge onto slave interfaces. I've tried adding Ethernet ports, veth interfaces and WiFi APs to the bridge. In every case, tcpdump shows the DHCP request entering the local bridge, traversing the tunnel to the remote bridge, and the reply traversing the tunnel and reaching the local bridge, but never reaching the interface where the request originated.

STP is turned on for both bridges (but I've tried with it turned off as well). /sys/class/net/tunnel_bridge/bridge/nf_call_arptables and /sys/class/net/tunnel_bridge/bridge/nf_call_iptables are both 0. All iptables are empty with their default policy set to ACCEPT. All ebtables are empty. brctl showstp shows all ports in the forwarding state.

As far as I can tell, the only difference between the working and non-working configuration is replacing an L2TPv3 tunnel with a VxLAN tunnel. How could this affect how traffic then propagates off the bridge to other interfaces? What else can I check?

Edit Part of the answer here is that the VxLAN is echoing packets arriving on the tunnel back to the originating bridge. So I see the original DHCP request arriving on the bridge and going into the tunnel, then a duplicate frame arrives on the bridge. This causes the bridge to update its ideas on which port that MAC address can be found on, meaning that the reply then gets directed back into the VxLAN tunnel, not onto the port where the request originated. Setting brctl setageing tunnel_bridge 0 causes the bridge to flood all packets onto all bridge ports, and then it "works" - but it's obviously not ideal. I don't have any direct evidence that it's the VxLAN tunnel that is doing this, except that it all works correctly when VxLAN is replaced with L2TPv3. Why the VxLAN tunnel is doing this, I'm not sure.

1 Answer 1

1

The problem here was in fact with the VxLAN. There is an automated process adding broadcast entries to the fdb for the remote end of the VxLAN tunnel (eg bridge fdb append 00:00:00:00:00:00 dst <remote ip> dev vxlan1); this process was mistakenly also adding the local IP address as a VxLAN endpoint.

So when the DHCP request was sent from the veth interface, a unicast fdb entry would be added for the veth interface's MAC address (the source MAC on the DHCP frame) to the bridge's port forwarding table. The frame would then get flooded onto all the bridge ports. The VxLAN interface would send the frame across the tunnel to the remote, but it would also send it to itself. When it "received" this frame, it would get copied onto the bridge, and the bridge would see a frame from that MAC address arriving on the VxLAN port; it would update its port forwarding table accordingly, recording the VxLAN port as the way to reach the veth interface's MAC address.

When the DHCP reply arrived, the bridge would look at it, see the veth interface's MAC address, consult its port forwarding table, see that it last saw that MAC coming from the VxLAN port and so send it on its way to the VxLAN port. It would never reach the veth port.

What made this twig for me was that setting the bridge's ageing time to 0 "fixed" the problem because then the bridge will flood every packet onto every port. Though I probably should have spotted the extra fdb entry.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .