Linux Kernel before 3.6 used route caching to do IPv4 multipath routing, which meant routing between two separate lines/ISPs was quite easy. From 3.6 the algorithm changed to being per-packet, meaning that some route table/rule/iptables marker tricks were required to achieve the two lines/ISPs.

However, if you had two lines with the same ISP who could route a single IP down both lines on a per-packet basis in a balanced/failover fashion, then from 3.6 you could easily achieve line bonding (at the IP level) because of the per-packet routing in both directions.

From 4.4, the kernel changed again to flow-based load balancing based on a hash over the source and destination addresses.

I am currently running Kernel 4.4.36, and am using multipath routing over PPPoE connections. My downstream traffic from the ISP is routed across the two separate lines on a per-packet basis (one IP routed down both lines). This means I can achieve a download speed faster than the speed of one individual line. My upstream traffic is routed according to the newer flow-based algorithm across both ppp devices (which have the same IP address). This means that I cannot achieve an upload speed that is faster than the speed of a single line.

Is there a way to configure the current Kernel to use the per-packet algorithm? Would I need to revert to an older Kernel (which I don't want to do for various other reasons)?

My ISP does not support multi-link ppp.

In case it is relevant, I am currently running Arch Linux ARMv7 (on a Raspberry Pi 3) but could change to Raspian if necessary.


Ok, so after having had more time to investigate this I found a way to do it using Linux TEQL (True Link Equalizer). Here is a link I loosely followed, but with some tweaks.


This is how I got it working on Arch Linux ARMv7 (Raspberry Pi 3)

On boot:

The following command should be run on boot to load the appropriate Kernel module.

modprobe sch_teql

The following commands also to run on boot assuming you want to NAT from a local network on eth0.

sysctl -w net.ipv4.ip_forward
iptables -A INPUT -i ppp+ -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -i ppp+ -o eth0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A POSTROUTING -t nat -o teql+ -j MASQUERADE

The FORWARD return traffic is on ppp+, and the POSTROUTING MASQUERADE on teql+ because the outgoing traffic goes out on teql and the return traffic comes back on ppp.

When ppp links come up:

Assuming the links to be load-balanced are ppp, the following commands to be run in a script in an /etc/ppp/ip-up.d/ script.

sysctl -w net.ipv4.conf.ppp1.rp_filter=2
sysctl -w net.ipv4.conf.ppp2.rp_filter=2
tc qdisc add dev ppp1 root teql0
tc qdisc add dev ppp2 root teql0
ip address add dev teql0
# you can add additional public IP addresses teql0 if you need to
ip link set teql0 up
ip route replace default scope global dev teql0

Where is your ISP-facing public IP address. Additional public IPs can be assigned to the teql0 device, but don't need to be assigned to the ppp devices. In my setup the two ppp links share the same IP (negotiated by pppoe etc.) The teql link it manually assigned as shown above. The ISP needs to send traffic for the IP equally down both links.

The reverse path (rp_filter) is set to 2 (loose) both in the script above so that the return packets are not dropped due to them coming back on the ppp interfaces rather than teql0.

I have set it up that way, and it works perfectly. Very easy! When the links fail, there is seamless failover. When they come up, they just start working again. Seems like there is no packet loss or delay when it fails over, and none when it comes back up either.

Also, another way at the below link uses policy routing, with iptables to mark every other packet etc. but I will try in a few days to see whether it works any better than the above and provide feedback here accordingly.


Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.