Linux Source Routing, Strong End System Model / Strong Host Model?

Question

Can a multihomed Linux machine implement a true Strong ES Model?

Specific Use Case

I have a system with five different interfaces, each connected to the same subnet, thus the same gateway to the Internet.

• I would like to listen on each interface separately on the same port, and ensure that packets always go out the same interface they came in, and ensure that packets trying to come in the "wrong" interface are discarded.
• I would like to be able to bind to each interface and make outgoing connections to Internet destinations that always originate from the same source IP I bound to. For example,

  curl --interface interface_ip http://ipecho.net/plain

  should always show the same IP address I bound to with --interface.
• Static routes may be problematic due to using DHCP on one of these interfaces.

RFC 1122

From RFC 1122 - Requirements for Internet Hosts - Communication Layers, Section 3.3.4.2 – Multihoming Requirements:

Internet host implementors have used two different conceptual models for multihoming, briefly summarized in the following discussion.  This document takes no stand on which model is preferred; each seems to have a place.  This ambivalence is reflected in the issues (A) and (B) being optional.

•   Strong ES Model

  The Strong ES (End System, i.e., host) model emphasizes the host/gateway (ES/IS) distinction, and would therefore substitute MUST for MAY in issues (A) and (B) above.  It tends to model a multihomed host as a set of logical hosts within the same physical host.
  
  With respect to (A), proponents of the Strong ES model note that automatic Internet routing mechanisms could not route a datagram to a physical interface that did not correspond to the destination address.
  
  Under the Strong ES model, the route computation for an outgoing datagram is the mapping:

       route(src IP addr, dest IP addr, TOS) -> gateway
  

  Here the source address is included as a parameter in order to select a gateway that is directly reachable on the corresponding physical interface.  Note that this model logically requires that in general there be at least one default gateway, and preferably multiple defaults, for each IP source address.

  
  

•   Weak ES Model

  This view de-emphasizes the ES/IS distinction, and would therefore substitute MUST NOT for MAY in issues (A) and (B).  This model may be the more natural one for hosts that wiretap gateway routing protocols, and is necessary for hosts that have embedded gateway functionality.
  
  The Weak ES Model may cause the Redirect mechanism to fail.  If a datagram is sent out a physical interface that does not correspond to the destination address, the first-hop gateway will not realize when it needs to send a Redirect.  On the other hand, if the host has embedded gateway functionality, then it has routing information without listening to Redirects.
  
  In the Weak ES model, the route computation for an outgoing datagram is the mapping:

       route(dest IP addr, TOS) -> gateway, interface
  

Linux is a Weak ES model by default, whereas FreeBSD and other Unix varieties act as Strong ES systems. Is there any way to make it behave more like a Strong ES system?

What sysctl or compile-time config would need to be set to make it behave like a Strong ES by default, without adding specific routing rules for any new interface you add? I know we can do strict source-route filtering via net.ipv4.conf.default.rp_filter = 1, but there seems to be much more to it than that. How can I do Source-Based Routing by default?

Answer 1

Just adding firewall rules won't be enough for this one. You want the system to route traffic as if it was two independent systems that just happen to share the same hardware and processes: that's what the Strong ES Model effectively is.

When aiming for Strong ES Model in Linux, you'll first need these sysctl settings:

net.ipv4.conf.all.arp_filter=1 
net.ipv4.conf.all.arp_ignore=1 # or even 2
net.ipv4.conf.all.arp_announce=2

These settings will make ARP behave as appropriate with the Strong ES Model, i.e. when an ARP request is received, only the interface with the exact requested address will answer, and only if traffic to the originating address would in fact be sent out through that specific interface.

Then, since you have five interfaces which you want to be behave differently in terms of routing, you'll need to set up five custom routing tables. You could use numbers to identify them, but generally it is clearer to specify names for them. So, pick numbers for each of them between 1 and 252, and some suitable names. (Numbers 0, 253, 254 and 255 are reserved.)

For example, let's pick 100 = rtable0, 101 = rtable1, 102 = rtable2, 103 = rtable3 and 104 = rtable4. Add these numbers and names to the end of /etc/iproute2/rt_tables file:

# ...default stuff above...
100    rtable0
101    rtable1
102    rtable2
103    rtable3
104    rtable4

Then, populate each custom routing table with a minimal set of route entries appropriate for each interface. (I'm replacing actual values with hopefully descriptively named environment variable names.)

ip route add $ETH0_NET dev eth0 proto static src $ETH0_IP table rtable0
ip route add default via $ETH0_GW dev eth0 proto static src $ETH0_IP table rtable0

ip route add $ETH1_NET dev eth1 proto static src $ETH1_IP table rtable1
ip route add default via $ETH1_GW dev eth1 proto static src $ETH1_IP table rtable1

# ... and so on, for all 5 interfaces

Finally, add advanced routing rules that will check the source address of each package and choose the routing table to be used accordingly:

ip rule add from $ETH0_IP table rtable0
ip rule add from $ETH1_IP table rtable1
#...

To make all this configuration persistent over reboots, you may have to write custom start-up scripts (or possibly ifup-pre or ifup-post scripts) to suit your Linux distribution's conventions.

For an extra insurance, you might add per-interface iptables rules to silently drop any incoming packets that may get received on the wrong interface. If all is well, the packet counts for these should remain zeroes: if they start increasing, you may have missed something in the configuration.

iptables -A INPUT -m addrtype --dst-type UNICAST -i eth0 ! -d $ETH0_IP -j DROP
iptables -A INPUT -m addrtype --dst-type UNICAST -i eth1 ! -d $ETH1_IP -j DROP
# ... and so on for each interface

Note: I've once implemented a set-up like this based on an old internet discussion by Rick Jones and other networking gurus. They said, paraphrased, "while all this is clearly necessary to achieve Strong Host Model behavior in Linux, I cannot guarantee that it is sufficient for all possible use cases". It worked flawlessly for me; it may or may not be enough for you, depending on exactly what you are going to use it for.

Warning: ensure that you have some sort of local or remote console access to the system when setting this configuration up. This set-up is extremely likely to completely mess up your network access while it's only half-done.

While it would be possible to set up N interfaces with only (N-1) custom routing tables, my personal preference is to move all the routing configuration to custom tables when using advanced routing. Having route -n or ip route show come up essentially empty while the system is clearly having network connections will be a very big clue that something very special is going on. Nevertheless, back when I set up a system like this, I also set up a permanent notice in /etc/motd about advanced routing being in effect, and the locations of the actual scripts that set it up.

Answer 2

There is another option which allows more control about the ARP handling. Have a look at arp_ignore.