I have a Dell PowerEdge T410 server with a Broadcom Limited NetExtreme II BCM5716 Gigabit Ethernet NIC and Debian 10 which I'm hoping to install Proxmox on top of. My home network is configured with various switches supporting VLANs and link aggregation all managed by OpenWrt. I want to configure a 2Gbs LACP/802.3ad (interface bonding) connection from the T410 server to my main 24-port switch (Netgear GS724T). Additionally I want to pass the VLAN trunk from my OpenWrt router through the 24-port switch to the virtual machines in Proxmox so I can place virtual machines in specific VLANs whilst keeping the Debian host on its own VLAN, usually handled by the native VLAN on the switch if I'm not mistaken?
The first thing I did after freshly installing the OS is check what interfaces I have. Debian didn't have net-tools installed by default but networkctl shows the following interfaces:
IDX Link Type OPERATIONAL SETUP
1 lo loopback n/a unmanaged
2 eno1 ether n/a unmanaged
3 eno2 ether n/a unmanaged
After installing net-tools I have the following information from ifconfig -a:
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.254.123 netmask 255.255.255.0 broadcast 192.168.254.15
inet6 fe80::d6ae:52ff:fe98:d5a4 prefixlen 64 scopeid 0x20<link>
ether d4:ae:52:98:d5:a4 txqueuelen 1000 (Ethernet)
RX packets 650 bytes 469142 (458.1 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 499 bytes 54964 (53.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eno2: flags=4098<BROADCAST,MULTICAST> mtu 1500
ether d4:ae:52:98:d5:a5 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Next I have installed the bonding package with sudo apt-get install ifenslave. I then have the following contents in the /etc/network/interfaces file:
allow-hotplug eno1
iface eno1 inet dhcp
allow-hotplug eno2
iface eno2 inet dhcp
# LAG
auto bond0
iface bond0 inet dhcp
bond-mode 802.3ad
bond-slaves eno1 eno2
bond-miimon 100
bond-downdelay 200
bond-updelay 200
bond-lacp-rate 1
bond-xmit-hash-policy layer2+3
This results in the following output
root@T410-Server:/home/will# ifup bond0
Internet Systems Consortium DHCP Client 4.4.1
Copyright 2004-2018 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
Listening on LPF/bond0/d4:ae:52:98:d5:a4
Sending on LPF/bond0/d4:ae:52:98:d5:a4
Sending on Socket/fallback
Created duid "\000\001\000\001'\373/w\324\256R\230\325\244".
DHCPDISCOVER on bond0 to 255.255.255.255 port 67 interval 6
DHCPDISCOVER on bond0 to 255.255.255.255 port 67 interval 12
DHCPOFFER of 192.168.254.123 from 192.168.254.1
DHCPREQUEST for 192.168.254.123 on bond0 to 255.255.255.255 port 67
DHCPACK of 192.168.254.123 from 192.168.254.1
Timeout too large reducing to: 2147483646 (TIME_MAX - 1)
bound to 192.168.254.123 -- renewal in 2147483648 seconds.
However, when I take the interface down and bring it back up with ifdown/ifup or simply restarting the networking service with systemctl restart networking.service I get the following error message:
bond0: invalid new link 3 on slave eno2
What am I do wrong? I've spent countless hours trying to get this to work, and non of the guides I've followed work. With the problems put aside, how do I setup a fallback that goes back to a single 1Gbps connection on eno1 if the link aggregation fails for example?
Lastly, how do I setup VLANs so that the host server is on one VLAN and the VLAN trunk is passed through to Proxmox where the virtual machines are? I would assume the server will automatically be listening on the desired VLAN because I have the PVID (AKA native VLAN) set on my switch. Is the correct way of doing this? I have been reading the example here on the Debian Wiki. Is this setup for receiving a VLAN trunk connection on one or more ports and untagging to another? If so this isn't what I need. On the other hand, the begging part of the example covers NIC bonding which I've closely followed but I'm still unable to get it to work.
