44

I have a system with two interfaces. Both interfaces are connected to the internet. One of them is set as the default route; a side effect of this is that if a packet comes in on the non-default-route interface, the reply is sent back through the default route interface. Is there a way to use iptables (or something else) to track the connection and send the reply back through the interface it came from?

49
echo 200 isp2 >> /etc/iproute2/rt_tables
ip rule add from <interface_IP> dev <interface> table isp2
ip route add default via <gateway_IP> dev <interface> table isp2

The above doesn't require any packet marking with ipfilter. It works because the outgoing (reply) packets will have the IP address that was originally used to connect to the 2nd interface as the source (from) address on the outgoing packet.

  • 6
    Wow, this is exactly what I was looking for. In case anyone is looking, for RH-based distros, you can put the relevant rule and route commands in files named 'rule-eth0' or 'route-eth0' (for example) which will be added or removed on ifup/ifdown. Place these files alongside the ifcfg-eth0 file. For IPv6, there is 'route6-eth0' functionality built in, but no 'rule6-eth0' built in (yet). – Kyle Brantley Dec 30 '11 at 3:58
  • 11
    For me it worked only when I left out the dev param in the ip rule command, so running ip rule add from <interface_IP> table isp2 – cdauth Dec 13 '15 at 17:09
  • 2
    You can make these be created when the interface goes up by adding up ip rule add from <interface_IP> table isp2 and up ip route add default via <gateway_IP> dev ppp0 table isp2 to your /etc/network/interfaces under the relevant interface. – g.rocket Apr 5 '16 at 4:59
  • 2
    I had to remove dev <interface> from ip rule to get it to work on my box. If I’m understanding right, dev <interface> was filtering out packets which were somehow set on the wrong interface which needed to be wrested over to the correct interface by the overridden route but the rule filtering by interface was preventing that from happening. – binki Oct 2 '16 at 3:52
  • I've tried this on my home machine and it works perfectly (with dev <interface> removed), but I can't for the life of me get it working on a dedicated server I rent. – goji Nov 8 '16 at 7:34
6

The following commands create an alternate routing table via eth1 for packets that have the mark 1 (except packets to localhost). The ip command is from the iproute2 suite (Ubuntu: iproute Install iproute http://bit.ly/software-small, iproute-doc Install iproute-doc http://bit.ly/software-small).

ip rule add fwmark 1 table 1
ip route add 127.0.0.0/0 table 1 dev lo
ip route add 0.0.0.0/0 table 1 dev eth1

The other half of the job is recognizing packets that must get the mark 1; then use iptables -t mangle -A OUTPUT … -j MARK --set-mark 1 on these packets to have them routed through routing table 1. I think the following should do it (replace 1.2.3.4 by the address of the non-default-route interface):

iptables -t mangle -A OUTPUT -m conntrack --ctorigdst 1.2.3.4 -j MARK --set-mark 1

I'm not sure if that's enough, maybe another rule is needed on the incoming packets to tell the conntrack module to track them.

  • Nice. I forgot all about marking. That should get me there. – Shawn J. Goff Nov 30 '10 at 2:33
3

I'm assuming you are running Linux and, further, that you are utilising a RedHat/CentOS-based distribution. Other Unix's and distributions will require similar steps - but the details will be different.


Start by testing (note that this is very similar to @Peter's answer. I am assuming the following:

  • eno0 is isp0 and has the overall default gateway
  • eno1 is isp1 and has the IP/range 192.168.1.2/24 with gateway 192.168.1.1

The commands are as follows:

$ echo 200 isp1 >> /etc/iproute2/rt_tables
$ ip rule add from eno1 table isp1
$ ip route add default via 192.168.1.1 dev eno1 table isp1

The firewall is not involved in any way. Reply packets would always have been sent from the correct IP - but previously were being sent out via the wrong interface. Now these packets from the correct IP will be sent via the correct interface.


Assuming the above worked, you can now make the rule and route changes permanent. This depends on what version of Unix you are using. As before, I'm assuming a RH/CentOS-based Linux distribution.

$ echo "from eno1 table isp1" > /etc/sysconfig/network-scripts/rule-eno1
$ echo "default via 192.168.1.1 dev eno1 table isp1" > /etc/sysconfig/network-scripts/route-eno1

Test that the network change is permanent:

$ ifdown eno1 ; ifup eno1

If that didn't work, on the later versions of RH/CentOS you also need to do go with one of two options:

  • Don't use the default NetworkManager.service; Use network.service instead. I haven't explored the exact steps needed for this. I would imagine it involves the standard chkconfig or systemctl commands to enable/disable services.
  • Install the NetworkManager-dispatcher-routing-rules package

Personally I prefer installing the rules package as it is the simpler more supported approach:

$ yum install NetworkManager-dispatcher-routing-rules

Another strong recommendation is to enable arp filtering as this prevents other related issues with dual network configurations. With RH/CentOS, add the following content to the /etc/sysctl.conf file:

net.ipv4.conf.default.arp_filter=1
net.ipv4.conf.all.arp_filter=1
3

I had issues with the locally generated packets with the solution suggested by Peter, I've found that the following corrects that:

echo 200 isp2 >> /etc/iproute2/rt_tables
ip rule add from <interface_IP> table isp2 priority 900
ip rule add from dev <interface> table isp2 priority 1000
ip route add default via <gateway_IP> dev <interface> table isp2
ip route add <interface_prefix> dev <interface> proto static scope link src <interface_IP> table isp2

NOTE: You may run into syntax issues with the 4th line above. In such cases the syntax for the 4th command may be this now:

ip rule add iif <interface> table isp2 priority 1000
  • Tried so much but nothing worked at my scenario, except this.. many thanks. – agaggi May 31 '17 at 20:48

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.