Linux as router: I have 3 Internet providers, each with its own modem.

Provider1, which is gateway address
Connected to linux router eth1/

Provider2, gateway address
Connected to linux router eth2/

Provider3, gateway address
Connected to linux router eth3/

                                                   +------------+         /
                                                   |            |        |
                            +----------------------+ Provider 1 +--------|
        __                  |           | |       /
    ___/  \_         +------+-------+              +------------+      |
  _/        \__      |    eth1      |              +------------+      /
 /             \ eth0|              |   |            |      |
|Client network -----+  ROUTER  eth2|--------------+ Provider 2 +------|     Internet
 \ __/    |              |              | |      |
   \__     __/       |    eth3      |              +------------+      \
      \___/          +------+-------+              +------------+       |
                            |           |            |       \
                            +----------------------+ Provider 3 +-------|
                                                   | |       |
                                                   +------------+       \________

I would like to route the clients in network by source IP to different gateways.
The interface to the client network is eth0/, which is the default gateway for all clients.

For example: should be routed to Provider1 @ eth1 should be routed to Provider2 @ eth2
...and so on...

I think I need to use ip route and iptables for SNAT, but I have not figured out exactly how.
Here is the script I have so far.
ipv4 forwarding is enabled.

# flush tables
ip route flush table connection1
ip route flush table connection2
ip route flush table connection3

# add the default gateways for each table
ip route add table connection1 default via
ip route add table connection2 default via
ip route add table connection3 default via

# add some IP addresses for marking
iptables -t mangle -A PREROUTING -s -j MARK --set-mark 1
iptables -t mangle -A PREROUTING -s -j MARK --set-mark 2
iptables -t mangle -A PREROUTING -s -j MARK --set-mark 3

# add the source nat rules for each outgoing interface
iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source
iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source
iptables -t nat -A POSTROUTING -o eth3 -j SNAT --to-source

# link routing tables to connections (?)
ip rule add fwmark 1 table connection1
ip rule add fwmark 2 table connection2
ip rule add fwmark 3 table connection3

#default route for anything not configured above should be eth2
  • You need to add in CONNMARK, I think, to save/restore the mark so it can be applied to packets 2..n (which will be NAT'd by connection tracking)
    – derobert
    Commented Aug 23, 2013 at 22:26
  • I've added an answer with excerpts from the config we use here. I'll try to check in over the weekend to clarify anything...
    – derobert
    Commented Aug 23, 2013 at 22:49

1 Answer 1


Here is a similar setup from one of our routers (with some irrelevant stuff snipped). Note that this handles incoming connections as well.

Note the use of variables instead of hard-coded mark numbers. So much easier to maintain! They're stored in a separate script, and sourced in. Table names are configured in /etc/iproute2/rt_tables. Interface names are set in /etc/udev/rules.d/70-persistent-net.rules.

##### fwmark ######
iptables -t mangle -F
iptables -t mangle -X

iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A PREROUTING -m mark ! --mark 0 -j RETURN # if already set, we're done
iptables -t mangle -A PREROUTING -i wan      -j MARK --set-mark $MARK_CAVTEL
iptables -t mangle -A PREROUTING -i comcast  -j MARK --set-mark $MARK_COMCAST
iptables -t mangle -A PREROUTING -i vz-dsl   -j MARK --set-mark $MARK_VZDSL

iptables -t mangle -A POSTROUTING -o wan     -j MARK --set-mark $MARK_CAVTEL
iptables -t mangle -A POSTROUTING -o comcast -j MARK --set-mark $MARK_COMCAST
iptables -t mangle -A POSTROUTING -o vz-dsl  -j MARK --set-mark $MARK_VZDSL
iptables -t mangle -A POSTROUTING -j CONNMARK --save-mark

##### NAT ######
iptables -t nat -F
iptables -t nat -X
for local in «list of internal IP/netmask combos»; do
    iptables -t nat -A POSTROUTING -s $local -o wan     -j SNAT --to-source «IP»
    iptables -t nat -A POSTROUTING -s $local -o comcast -j SNAT --to-source «IP»
    iptables -t nat -A POSTROUTING -s $local -o vz-dsl  -j SNAT --to-source «IP»

# this is an example of what the incoming traffic rules look like
for extip in «list of external IPs»; do
    iptables -t nat -A PREROUTING   -p tcp -d $extip --dport «port» -j DNAT --to-destination «internal-IP»:443

And the rules:

ip rule flush
ip rule add from all               pref 1000  lookup main 
ip rule add from A.B.C.D/29        pref 1500  lookup comcast # these IPs are the external ranges (we have multiple IPs on each connection)
ip rule add from E.F.G.H/29        pref 1501  lookup cavtel
ip rule add from I.J.K.L/31        pref 1502  lookup vzdsl
ip rule add from M.N.O.P/31        pref 1502  lookup vzdsl # yes, you can have multiple ranges
ip rule add fwmark $MARK_COMCAST   pref 2000  lookup comcast
ip rule add fwmark $MARK_CAVTEL    pref 2001  lookup cavtel
ip rule add fwmark $MARK_VZDSL     pref 2002  lookup vzdsl
ip rule add                        pref 2500  lookup comcast # the pref order here determines the default—we default to Comcast.
ip rule add                        pref 2501  lookup cavtel
ip rule add                        pref 2502  lookup vzdsl
ip rule add                        pref 32767 lookup default

The routing tables get set up in /etc/network/interfaces, so that taking down an interface makes it switch to using a different one:

iface comcast inet static
        address A.B.C.Q
        up ip route add table comcast default via A.B.C.R dev comcast
        down ip route flush table comcast

Note: If you're doing filtering as well (which you probably are) you'll also need to add the appropriate rules to FORWARD to ACCEPT the traffic. Especially for any incoming traffic.

  • Thank you very much! I am now going to modify this to my needs, load it on the box and update this post.
    – Flav
    Commented Aug 24, 2013 at 7:10
  • Works like a charm, thanks again. Except for the pref order / default route to 'comcast'. (For me that should be eth2 ) But I think I worked around it by adding a general rule ip rule add from pref 1400 lookup eth2 and creating the exceptions afterwards.
    – Flav
    Commented Aug 24, 2013 at 8:11
  • 1
    @Flav you can also set your exceptions with the firewall marks (in PREROUTING). BTW: One of the linked questions (unix.stackexchange.com/questions/70440/…) has more explanation of part of this config. Those ip/mask rules are actually for non-NAT'd traffic in my config (SNAT happens in POSTROUTING, hence after the ip rule stuff)
    – derobert
    Commented Aug 25, 2013 at 8:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .