I'm running Gentoo Linux, so using plain iptbales to manage my firewall and networking. I usually use wan0 for all my traffic, but since I have already a web-server behind that i would like wan1 (bind to another domain) for my second web-server.

I have three interfaces:

  • eth0 = LAN
  • wan0 = Primary used WAN (default gateway)
  • wan1 = Secondary WAN

Some Infos about the Gateways

> route -n 

Kernel IP Routentabelle
Ziel Router          Genmask         Flags Metric Ref    Use Iface         80.108.x.x         UG    0      0        0 wan0     U     0      0        0 eth0
80.108.x.0   U     0      0        0 wan0
84.114.y.0   U     0      0        0 wan1       UG    0      0        0 lo

The default init for NAT/MASQUEARDING is

sysctl -q -w net.ipv4.conf.all.forwarding=1

iptables -N BLOCK
iptables -A BLOCK -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A BLOCK -m state --state NEW,ESTABLISHED -i eth0 -j ACCEPT
iptables -A BLOCK -m state --state NEW,ESTABLISHED -i lo -j ACCEPT

iptables -t nat -A POSTROUTING -o eth0 -s -j MASQUERADE
iptables -t nat -A POSTROUTING -o wan0 -j MASQUERADE
iptables -t nat -A POSTROUTING -o wan1 -j MASQUERADE

Behind this gateway I'm running several web-servers. On one machine I'm running a HTTP Server on Port 8000 instead of 80. Usually when I'm using wan0 as the incoming interface I use the following rules:

iptables -A FORWARD -i wan0 -p TCP -d $lan_host1--dport 80 -j ACCEPT
iptables -t nat -A PREROUTING -i wan0 -p TCP --dport 8000 -j DNAT --to-destination "$lan_host1":80
iptables -A FORWARD -i wan0 -p UDP -d $lan_host1--dport 80 -j ACCEPT
iptables -t nat -A PREROUTING -i wan0 -p UDP --dport 8000 -j DNAT --to-destination "$lan_host1":80

That works fine. Now I would like wan1 to be used, as wan0 is tied to an IP/domain I usually use for something else.

I thought a simple change to wan1 would do it.

iptables -A FORWARD -i wan1 -p TCP -d $lan_host1--dport 80 -j ACCEPT
iptables -t nat -A PREROUTING -i wan1 -p TCP --dport 8000 -j DNAT --to-destination "$lan_host1":80
iptables -A FORWARD -i wan1 -p UDP -d $lan_host1--dport 80 -j ACCEPT
iptables -t nat -A PREROUTING -i wan1 -p UDP --dport 8000 -j DNAT --to-destination "$lan_host1":80

But that doesn't work. I guess the issue is that wan0 is the default GW. So I guess packages received by wan1 are forwarded to lan_host1, but when being send back to the gateway they are send through wan0 instead of wan1 or at least using the ip from wan0.

Any suggestions how I could manage this?

Thanks in advance, Rob

  • What I can tell: The packages are received on the firewall via the wan1 interface, but are not forwarded to the internal server... – NightDragon Mar 20 '20 at 1:10
  • Not relevant to your question, but anyway, what's the goal of iptables -t nat -A POSTROUTING -o eth0 -s -j MASQUERADE? Is that about hairpinning? Just curious. – A.B Mar 20 '20 at 16:34
  • @A.B it was or is part of a more complex setup (another tun0, VPN, etc...) But yes It's also used for hairpinning – NightDragon Mar 20 '20 at 19:44

As the answer is tied to the configuration, I make some assumptions. You'll have to adapt the answer to fit the actual configuration.

  • wan1's LAN and gateway for wan1 arbitrarily chosen as and

  • no consideration of firewall rules made, but all this shouldn't interact with them.

On Linux ip link, ip address and ip route should always be used instead of deprecated ifconfig and route. route probably can't handle additional routing tables anyway.

Just as a reminder, iptables or actually netfilter, doesn't route, but it can by its actions alter routing decisions made by the IP routing stack. This schematic shows where routing decisions can happen. For routed (rather than locally originated) traffic that's only in one place and alterations must happen before: raw/PREROUTING, mangle/PREROUTING or nat/PREROUTING, with raw often impractical, and nat only for limited cases, mostly leaving mangle.

A basic multi-homed system, to use multiple paths to internet, usually requires policy routing, where the route can change not only with the destination as usual, but also with the source or with other selectors (as will be done here) used in policy rules. On Linux additional rules made with ip rule can select a different routing table to select for example a different default route (there will still be only one default route, but one per routing table).

So here the principle, while still keeping active Strict Reverse Path Forwarding (rp_filter), is to accept packets coming from wan1 and route them as usual toward eth0 using an alternate table (which will allow to pass rp_filter). This additional routing table should duplicate the main routing table, but using only routes needed for the alternate path (wan1) and thus not including the usual routes with the "normal" path (wan0). If other routes (such as VPNs etc.) have to be involved in flows going through wan1, chances are that their route too have to be added, or other additional rules and tables have to be created to cope with that.

Since Linux discontinued the use of a routing cache in kernel 3.6, nothing in the routing stack would tell to send back reply packets from host1 to the client through wan1 and they would end up going out using the main default route through wan0, NATed with the wrong IP for this interface (netfilter is route-agnostic and had already chosen the NAT to be done when receiving the first packet of the connection) and probably dropped by the next router of the ISP also doing Strict Reverse Path Filtering. There's a netfilter feature allowing to copy a packet's mark in the conntrack's mark and put it back in the packet: this will act as route memory for the connection. So iptables and netfilter's conntrack will be used for two related features: to mark the packet in order to alter the routing decision, and to restore this mark on the reply packets identified as part of the same connection.

All this translates to these commands:

  • routing part

    Use for marked packets (arbitrary mark value 101) an extra routing table (unrelated arbitrary value also 101) :

    ip rule add fwmark 101 lookup 101

    Populate the table with entries similar the main routing table, minus wan0 entries:

    ip route add table 101 dev eth0
    ip route add table 101 dev wan1
    ip route add table 101 default via dev wan1
  • iptables/netfilter part

    There are various optimizations possible in the following commands, It can probably be improved.

    Restore a potential previous mark already saved, so reply packets will get the same mark as original packets:

    iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark

    Mark packets arriving from wan1 to alter routing decision above:

    iptables -t mangle -A PREROUTING -i wan1 -j MARK --set-mark 101 

    If there's a mark, save it in conntrack (could have been done in the nat table to do it only once per connection flow rather than for every packet):

    iptables -t mangle -A PREROUTING -m mark ! --mark 0 -j CONNMARK --save-mark

Actually this will still fail a Strict Reverse Path Forwarding check, since this undocumented feature was added in 2010. It must be used here:

sysctl -w net.ipv4.conf.wan1.src_valid_mark=1
  • Hi! You are right, i really should start using the ip command instead of the old ones. Anyway. As i haven't used multiple routing tables yet, the output was just to show what's currently there. – NightDragon Mar 21 '20 at 9:28
  • Thanks a lot for your help. It was exactly what i was looking for. The only thing I need to fix now is that also some specific source/ports combos shall also be routed over that. – NightDragon Mar 21 '20 at 20:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.