I've been testing a bonded network based on Ubuntu Server 16.04 LTS. The far end of the bond is a VM also running Ubuntu Server 16.04 LTS. The two servers are tied together by two OpenVPN tunnels in TAP mode; the TAP interfaces at each end are then bonded together in balance-rr mode.
It all works perfectly fine and I can successfully use the bonded connection, however I noticed during a ping test that if I pull down one of the TAP interfaces on the client side, the ping starts to drop packets. If I bring the interface back up, the ping continues and there is no further packet loss.
According to the Linux Bonding Driver document balance-rr mode provides not only packet striping but also fault tolerance. To my mind, fault tolerance means that if one interface fails, all of the packets are then distributed among the other interfaces. If the dead interface returns, it will be back in the pool and ready to receive packets.
Has anyone noticed this issue? If so, how did you resolve it?