Setting TCP MSS on a bridge port

Question

I have a VXLAN tunnel between two Linux hosts. This tunnel is carried over an Ethernet link with an MTU of 1500 bytes. At each end, the VXLAN interface is slaved to a bridge, along with another Ethernet interface. It looks something like this:

                 |         Host A               |                 Host B       |
Client Device  <--> eth0 <--> br0 <--> vxlan0 <-|-> vxlan0 <--> br0 <--> eth0 <--> Client Device
                 |                              |                              |
                 |                       eth1 <-|-> eth1                       |

Here only the client devices and eth1 have IP configuration - everything else is just a layer 2 link between two sites.

VXLAN involves 50 bytes of overhead. To avoid breaking certain protocols, I've set the MTU on the layer 2 link (eth0, br0, vxlan0) to be 1500 bytes. To avoid TCP fragmentation, I want to set the TCP MSS on the vxlan0 interfaces to be 1400 bytes. How can I do this?

I know that I can turn iptables on for the bridges (echo 1 > /sys/class/net/br0/bridge/nf_call_iptables). However, iptables has a lot of other rules that are unrelated to this part of the system and doing this comes with a considerable performance cost.

I know that I can set the MSS for a route with ip route add ... advmss 1400. But note that none of the packets involved are routed on Host A or Host B. They are just bridged through from interface to another.

Is there some way to apply a lower TCP MSS to this traffic without incurring the iptables performance penalty?

Answer 1

nftables aims to replace ebtables + iptables: most features are also available at the bridge level the same way they are available at the IP level without using br_netfilter (including stateful IP firewalling at bridge level, with kernel >= 5.3).

Here's a ruleset (let's call it bridgemss.nft) to load on HostA and HostB with nft -f bridgemss.nft:

table bridge t
delete table bridge t

table bridge t {
    chain c {
        type filter hook forward priority filter; policy accept;
        oifname vxlan0 tcp flags syn tcp option maxseg size set 1400
    }
}

Of course the ruleset can be changed to suit the needs (eg: have it used only on Host A with both directions instead of having it on Host A and Host B in a single direction each).

Should HostA and HostB assign an address on br0 to communicate over VXLAN instead of directly with eth1, then this rule should be duplicated for example in bridge output hooks (or on one system output + input hooks) since this won't count as forwarded frames.