I have some servers with multiple network interfaces, setup with bonding and some VLANs. Whenever I reboot the server, one of the bonded network interfaces is not reachable from other servers, nor can any traffic leave that interface. The status from ifconfig on that interface does indicate that the link is up, however. Simply restarting the network at this point will restore everything to normal.
The fact that everything works as expected after I restart the network makes me think that my configuration is correct, but it is something in the startup order that isn't working right on reboot, but gets straightened out when restarting the network.
I have 7 identical servers with the same setup (other than IP addresses differing), and it happens on all of them, every time they are rebooted.
A bit more details on the setup:
- Servers: HP ProLiant DL380
- 6 network interfaces, setup as 3 bonded interfaces named: bondm, bondr, bondt.
- 4 interfaces are built-in, the remaining 2 are in an add-on PCI card
- bondm is configured with 2 VLANs
- bondm is used as the default route
- bondm is setup to use eth0 and eth2
- bondm is the interface that is failed on reboot
Update: I've restested this with the exact same configuration and kickstart files, but with SL 6.2 vs 6.3. Everything is fine with 6.2, but I get this behavior with 6.3. Is it due to the different kernels?
Here are some of the relevant config files from /etc/sysconfig/network-scripts:
$ cat ifcfg-eth0 ifcfg-eth2 ifcfg-bondm ifcfg-bondm.132 ifcfg-bondm.832 DEVICE=eth0 BOOTPROTO=none NM_CONTROLLED=no ONBOOT=yes TYPE=Ethernet HWADDR=44:1E:A1:03:71:C4 SLAVE=yes MASTER=bondm ETHTOOL_OPTS="-s eth0 speed 1000 duplex full" DEVICE=eth2 HWADDR=44:1E:A1:03:71:C8 NM_CONTROLLED=no ONBOOT=yes SLAVE=yes MASTER=bondm ETHTOOL_OPTS="-s eth2 speed 1000 duplex full" DEVICE=bondm BOOTPROTO=none NM_CONTROLLED=no ONBOOT=yes TYPE=Ethernet IPV4_FAILURE_FATAL=yes IPV6INIT=no BONDING_OPTS="mode=active-backup miimon=100" DEVICE=bondm.132 BOOTPROTO=none NM_CONTROLLED=no ONBOOT=yes TYPE=Ethernet IPADDR=192.168.13.19 PREFIX=28 GATEWAY=192.168.13.17 DEFROUTE=yes IPV4_FAILURE_FATAL=yes IPV6INIT=no BONDING_OPTS="mode=active-backup miimon=100" VLAN=yes DEVICE=bondm.832 BOOTPROTO=none NM_CONTROLLED=no ONBOOT=yes TYPE=Ethernet IPADDR=10.123.94.69 PREFIX=28 DEFROUTE=no IPV4_FAILURE_FATAL=yes IPV6INIT=no BONDING_OPTS="mode=active-backup miimon=100" VLAN=yes