2

I have a multihomed host with 2 Ethernet interfaces and a usb-connected cell modem. The Ethernet interfaces are configured in a typical way on the main routing table, with only one default route assigned to one of them.

[/]# ip route show table main
default via 192.168.4.1 dev eth0  metric 50 
192.168.2.0/24 dev eth1 scope link  src 192.168.2.10 
192.168.4.0/22 dev eth0 scope link  src 192.168.4.73

There is another default route assigned to the cell interface on a separate "usbcell" routing table.

[/]# ip route show table usbcell
default via 10.26.128.2 dev usbcell  src 10.26.128.1

An IP rule directs all packets marked with value 0x100 to use the usbcell routing table. These packets are marked at the socket level within an application. We also add source based routing rule to prevent reverse path filtering from dropping usbcell traffic.

[/]# ip rule
0:  from all lookup local 
97: from 10.26.128.1 lookup usbcell 
100:    from all fwmark 0x100 lookup usbcell 
32766:  from all lookup main 
32767:  from all lookup default

The Cell interface has an IP, netmask and gateway provided dynamically by the modem hardware. This subnet is not owned by the carrier or us, so I would like to prevent collisions with any other networks connected to the Ethernet ports. This should be easy since the cell interface is on a separate routing table. But, unfortunately, whenever the cell interface is brought up via ifup, a local route to the interface is added to the main table. This routes all packets bound for that subnet to the cell interface instead of just the marked ones. If I remove the local route, it works as expected.

Main table when usbcell is up:

[/]# ip route show table main
default via 192.168.4.1 dev eth0  metric 50 
10.26.128.0/17 dev usbcell scope link  src 10.26.128.1   ## How do I prevent this? ##
192.168.2.0/24 dev eth1 scope link  src 192.168.2.10 
192.168.4.0/22 dev eth0 scope link  src 192.168.4.73 

Is there a way of preventing this local route from being added to the main table when the usbcell interface is brought up?

10.26.128.0/17 dev usbcell scope link  src 10.26.128.1

In this example, I want unmarked packets bound for the subnet 10.26.128.0/17 to be routed out the main table default route (to eth0).

7
  • 2
    You can add the ip commands which delete the route in a post-up option below the iface usbcell ... stanza in /etc/network/interfaces (iiuc, you're using ifupdown; if that's not right, give more details about your system) Instead of not adding a default route, add one with a high metric e.g. with post-up r=$(ip route show to default dev $IFACE); ip route del $r; ip route add $r metric 800 (the metric option described in the manpage doesn't always work right). IIRC, NetworkManager does this by default. – user414777 Sep 27 '20 at 5:47
  • AFAIK, you always want link-local routes for the addr/netmask of an interface, because otherwise routing through this interface will not work. So if you delete the 10.26.128.0/17 dev usbcell scope link, basically you are making usbcell unusable. If you do want this, you could also bring the interface down. You didn't explain what you need usbcell, but a possible solution to prevent conflict with the eth interfaces is to put one party (or both) into a network namespace. – dirkt Sep 27 '20 at 5:52
  • @dirkt It works fine without the link-local route because there is a default route and none of the cell requests are to the link-local subnet. – Tanner Sep 27 '20 at 12:14
  • @user414777 I was hoping there was some option I'd missed outside of deleting it. I am adding a default route on the usbcell table. Its the local route I want to prevent. – Tanner Sep 27 '20 at 12:22
  • @Tanner as I mentioned, you can just use the metric option in the iface stanza, with a large value. However, last time I had used it, I had problems with it only being applied to a single interface, if two interfaces happened to be on the same network. Or something like that, and some other problems I forgot about ;-) – user414777 Sep 27 '20 at 12:41
1

Normally to prevent a new address to automatically create its associated LAN route, this address should be added to an interface with the flag noprefixroute (don't use iproute2 5.8 to test it):

ip address add 10.26.128.1/17 dev usbcell noprefixroute

This will add the address to the interface but will not create the automatic LAN route in the main table when the interface is brought up.

This will actually also prevent to run this command later:

# ip route add table usbcell default via 10.26.128.2 dev usbcell  src 10.26.128.1
Error: Nexthop has invalid gateway.

because there's a missing route which wasn't detected before (I'm surprised everything was working as expected in OP's scenario) which has to be added manually, or this table will never match. So in the end:

ip route add table usbcell 10.26.128.2/32 dev usbcell # or 10.26.128.0/17 if you prefer
ip route add table usbcell default via 10.26.128.2 dev usbcell src 10.26.128.1

Or if the usbcell device is doing Proxy ARP for any address (which is common with this kind of device) this would also work instead (but would start creating a big ARP table on the host):

ip route add table usbcell default dev usbcell src 10.26.128.1

If your backend can be tweaked, then it's easy to alter it there. For example with an ifupdown backend in /etc/network/interfaces:

iface usbcell inet manual
    pre-up ip addr add 10.26.128.1/17 dev usbcell noprefixroute
    down   ip addr del 10.26.128.1/17 dev usbcell noprefixroute

while at it the custom routing could be added too:

    pre-up ip rule add prio 100 fwmark 0x100 lookup usbcell
    pre-up ip rule add prio 97 from 10.26.128.1 lookup usbcell
    up ip route add table usbcell 10.26.128.2/32 dev usbcell
    up ip route add default via 10.26.128.2 dev usbcell src 10.26.128.1
    down ip rule del prio 97
    down ip rule del prio 100

If you can't do this, you can still override the routes so the route automatically added by the kernel is never used. If there was a metric, this would be simply a matter of doing:

ip route add 10.26.128.0/17 via 192.168.4.1 dev eth0

But as there's none (ie metric 0, which is the best precedence), the two will have the same precedence and this gets uncertain. Beside the precedence uncertainty one can't add this route after the interface is up because the kernel will complain with:

RTNETLINK answers: File exists

So just add two half routes (the same way OpenVPN adds two half default routes when using --redirect-gateway def1) to override this route:

ip route add 10.26.128.0/18 via 192.168.4.1 dev eth0
ip route add 10.26.192.0/18 via 192.168.4.1 dev eth0

Together they match the exact same range of 10.26.128.0/17. Being more specific than 10.26.128.0/17, they will always prevail. Remember that if eth0 is ever brought administratively down, these routes will disappear and have to be added back. So you should add them in the settings configuring eth0.

1
  • Thanks, noprefixroute is what I was lookng for. I ended up using a point to point connection from the usbcell interface to the gateway. – Tanner Nov 16 '20 at 16:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.