Prevent local network route creation at ifup

Question

I have a multihomed host with 2 Ethernet interfaces and a usb-connected cell modem. The Ethernet interfaces are configured in a typical way on the main routing table, with only one default route assigned to one of them.

[/]# ip route show table main
default via 192.168.4.1 dev eth0  metric 50 
192.168.2.0/24 dev eth1 scope link  src 192.168.2.10 
192.168.4.0/22 dev eth0 scope link  src 192.168.4.73

There is another default route assigned to the cell interface on a separate "usbcell" routing table.

[/]# ip route show table usbcell
default via 10.26.128.2 dev usbcell  src 10.26.128.1

An IP rule directs all packets marked with value 0x100 to use the usbcell routing table. These packets are marked at the socket level within an application. We also add source based routing rule to prevent reverse path filtering from dropping usbcell traffic.

[/]# ip rule
0:  from all lookup local 
97: from 10.26.128.1 lookup usbcell 
100:    from all fwmark 0x100 lookup usbcell 
32766:  from all lookup main 
32767:  from all lookup default

The Cell interface has an IP, netmask and gateway provided dynamically by the modem hardware. This subnet is not owned by the carrier or us, so I would like to prevent collisions with any other networks connected to the Ethernet ports. This should be easy since the cell interface is on a separate routing table. But, unfortunately, whenever the cell interface is brought up via ifup, a local route to the interface is added to the main table. This routes all packets bound for that subnet to the cell interface instead of just the marked ones. If I remove the local route, it works as expected.

Main table when usbcell is up:

[/]# ip route show table main
default via 192.168.4.1 dev eth0  metric 50 
10.26.128.0/17 dev usbcell scope link  src 10.26.128.1   ## How do I prevent this? ##
192.168.2.0/24 dev eth1 scope link  src 192.168.2.10 
192.168.4.0/22 dev eth0 scope link  src 192.168.4.73 

Is there a way of preventing this local route from being added to the main table when the usbcell interface is brought up?

10.26.128.0/17 dev usbcell scope link  src 10.26.128.1

In this example, I want unmarked packets bound for the subnet 10.26.128.0/17 to be routed out the main table default route (to eth0).

Answer 1

Normally to prevent a new address to automatically create its associated LAN route, this address should be added to an interface with the flag noprefixroute (don't use iproute2 5.8 to test it):

ip address add 10.26.128.1/17 dev usbcell noprefixroute

This will add the address to the interface but will not create the automatic LAN route in the main table when the interface is brought up.

This will actually also prevent to run this command later:

# ip route add table usbcell default via 10.26.128.2 dev usbcell  src 10.26.128.1
Error: Nexthop has invalid gateway.

because there's a missing route which wasn't detected before (I'm surprised everything was working as expected in OP's scenario) which has to be added manually, or this table will never match. So in the end:

ip route add table usbcell 10.26.128.2/32 dev usbcell # or 10.26.128.0/17 if you prefer
ip route add table usbcell default via 10.26.128.2 dev usbcell src 10.26.128.1

Or if the usbcell device is doing Proxy ARP for any address (which is common with this kind of device) this would also work instead (but would start creating a big ARP table on the host):

ip route add table usbcell default dev usbcell src 10.26.128.1

---

If your backend can be tweaked, then it's easy to alter it there. For example with an ifupdown backend in /etc/network/interfaces:

iface usbcell inet manual
    pre-up ip addr add 10.26.128.1/17 dev usbcell noprefixroute
    down   ip addr del 10.26.128.1/17 dev usbcell noprefixroute

while at it the custom routing could be added too:

    pre-up ip rule add prio 100 fwmark 0x100 lookup usbcell
    pre-up ip rule add prio 97 from 10.26.128.1 lookup usbcell
    up ip route add table usbcell 10.26.128.2/32 dev usbcell
    up ip route add default via 10.26.128.2 dev usbcell src 10.26.128.1
    down ip rule del prio 97
    down ip rule del prio 100

---

If you can't do this, you can still override the routes so the route automatically added by the kernel is never used. If there was a metric, this would be simply a matter of doing:

ip route add 10.26.128.0/17 via 192.168.4.1 dev eth0

But as there's none (ie metric 0, which is the best precedence), the two will have the same precedence and this gets uncertain. Beside the precedence uncertainty one can't add this route after the interface is up because the kernel will complain with:

RTNETLINK answers: File exists

So just add two half routes (the same way OpenVPN adds two half default routes when using --redirect-gateway def1) to override this route:

ip route add 10.26.128.0/18 via 192.168.4.1 dev eth0
ip route add 10.26.192.0/18 via 192.168.4.1 dev eth0

Together they match the exact same range of 10.26.128.0/17. Being more specific than 10.26.128.0/17, they will always prevail. Remember that if eth0 is ever brought administratively down, these routes will disappear and have to be added back. So you should add them in the settings configuring eth0.