Bonding with VLAN and bridge on Debian 11

Question

I'm trying to configure on Linux bonding active-backup (2 physical interfaces eth2 and eth3, one is used) but also with two VLANs, and later connect it to internal bridges. I see two options for connections, both are working,

1. First method works fine and looks easy, but all VLANs are routed through single physical interface (lower bandwidth)
2. The second is problematic (for me), but I prefer this second, because it allows to route one VLAN through eth2, and second VLAN through eth3 (greater bandwidth with fault-tolerant).

• I don't want bond-mode 4 = 802.3ad AKA LACP (servers are connected to different Cisco switches, I have heard and seen some problems with such configurations), so bond-mode active-backup need to be enough.

# 1. Two NICs agregated to one bonding, and then separate VLANS from bonding interface: 

eth2           bond1.10  - xenbr10
     > bond1 <
eth3           bond1.15 - xenbr15


# 2. VLANs separated form physical NIC, then aggregate to separate binding interfaces.

       eth2.15 ...
eth2 <
       eth2.10
                > bond10  - xenbr10
       eth3.10
eth3 <
       eth3.15 ...

I have server with Debian 11, 2 network interfaces, installed packages vlan, bridge-utils and ifenslave (in version 2.13 from testing repo, due to this problem from 2.12). Modules 8021q and bonding are loaded in system.

Question 1: Is first option of connection best-practices here? And why? I'm asking, because most (or maybe all) tutorials found on internet is about first connection (even Debian Wiki)

Option 1

This is working fine, I can do it both from console or by config file. Downside of this: both VLANs are sent always using single interface. My current /etc/network/interfaces (non-important elements removed)

iface eth2 inet manual
iface eth3 inet manual

auto bond1
iface bond1 inet manual
    bond-slaves eth2 eth3
    bond-mode active-backup
    bond-miimon 100
    bond-downdelay 200
    bond-updelay 200

iface bond1.10 inet manual
    vlan-raw-device bond1
iface bond1.15 inet manual
    vlan-raw-device bond1

auto xenbr10
iface xenbr10 inet static
    address 1.2.3.4/24
    bridge_ports bond1.10

After rebooting system this config is working correct, but I would like to configure and use second option.

Option 2: works from shell, unable to do from config file

I can do this configuration from the command line:

ifconfig eth2 up
ifconfig eth3 up
#setting up VLANs
ip link add link eth2 name eth2.10 type vlan id 10
ip link add link eth3 name eth3.10 type vlan id 10

#creating bonding interface with 2 slaves NIC
ip link add name bond1 type bond mode active-backup
ip link set dev eth2.10 down
ip link set dev eth3.10 down
ip link set master bond10 dev eth2.10
ip link set master bond10 dev eth3.10

ip link set up dev bond10
ip link set dev eth2.10 up
ip link set dev eth3.10 up

### Bridge + bonding
brctl addbr xenbr10
brctl addif xenbr10 bond10

ip addr add 1.2.3.4/24 dev xenbr10
ip link set dev xenbr10 up

And network is up:

cat /proc/net/bonding/bond10
Ethernet Channel Bonding Driver: v5.10.0-12-amd64

Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: eth2.10
MII Status: up
(..)

Slave Interface: eth2.10
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: (..)

Slave Interface: eth3.10
MII Status: up
(..)

But I'cant create config file to make this permanent. I tried many options like this, but with no luck. If i connect interface eth2.10 directly to some xenbr10 it works.

iface eth2.10 inet manual
    vlan-raw-device eth2
iface eth3.10 inet manual
    vlan-raw-device eth3
    bond-master bond10 #It doesn't change nothing, for testing

auto bond10
iface bond10 inet manual
    bond-slaves eth2.10 eth3.10
    #bond-slaves none
    bond-mode active-backup
    bond-miimon 100
    bond-downdelay 200
    bond-updelay 200

auto xenbr10
iface xenbr10 inet static
    address 1.2.3.4/24
    bridge_ports bond1.10

After rebooting system, usually I get errors like this:

ifup[686]: Failed to enslave eth2.10 to bond10. Is bond10 ready and a bonding interface ?

Question 2: What is wrong with this config? I've tried

• set eth2.10 config above or below the bond10 configuration,
• use bond-master bond10 entry for eth2.10 and bond-slaves none in bond10 config part
• use bond-slaves eth2.10 eth3.10 entry in bond10 part
• use auto eth2.10

Answer 1

After day of searching and testing I found two solutions. First config maybe not elegant, but works:

# Using default ifupdown on Debian 11:
# Set up NIC and NIC's alias with VLAN:
auto eth2
iface eth2 inet manual
    post-up ip link add link eth2 name eth2.10 type vlan id 10
    post-up ip link add link eth2 name eth2.15 type vlan id 15
auto eth3
iface eth3 inet manual
    post-up ip link add link eth3 name eth3.10 type vlan id 10
    post-up ip link add link eth3 name eth3.15 type vlan id 15

auto bond533
iface bond533 inet manual
    bond-slaves eth2.533 eth3.533
    bond-mode active-backup
# (...)

Thanks to A.B.'s comment (link) I found even better solution using ifupdown2. Beaware: during installation of this package (it removes older ifupdown) I've lost network until reboot (perhaps restarting network would be enough, not tested).

# After installing ifupdown2:
# No need to set up physical NIC or VLAN interface!
#iface eth2.10 inet manual
#   vlan-raw-device eth2
#iface eth2.533 inet manual
#   vlan-raw-device eth2

#Set up bonding and brige (the same as with ifupdown)
auto bond533
iface bond533 inet manual
    bond-slaves eth2.533 eth3.533
    bond-mode active-backup
# (...)

It is little bit weird for me that such simple config works correct. It looks that ifupdown2 can set up necessary bond-slaves network interfaces (even vlans) without configuring them earlier.

I was based on the configuration on this page: https://docs.nvidia.com/networking-ethernet-software/knowledge-base/Configuration-and-Usage/Network-Interfaces/Compare-ifupdown2-with-ifupdown/