Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Linux as router: I have 3 Internet providers, each with its own modem.

Provider1, which is gateway address 192.168.1.1
Connected to linux router eth1/192.168.1.2

Provider2, gateway address 192.168.2.1
Connected to linux router eth2/192.168.2.2

Provider3, gateway address 192.168.3.1
Connected to linux router eth3/192.168.3.2

                                                                           ________
                                                   +------------+         /
                                                   |            |        |
                            +----------------------+ Provider 1 +--------|
        __                  |192.168.1.2           |192.168.1.1 |       /
    ___/  \_         +------+-------+              +------------+      |
  _/        \__      |    eth1      |              +------------+      /
 /             \ eth0|              |192.168.2.2   |            |      |
|Client network -----+  ROUTER  eth2|--------------+ Provider 2 +------|     Internet
 \10.0.0.0/24 __/    |              |              |192.168.2.1 |      |
   \__     __/       |    eth3      |              +------------+      \
      \___/          +------+-------+              +------------+       |
                            |192.168.3.2           |            |       \
                            +----------------------+ Provider 3 +-------|
                                                   |192.168.3.1 |       |
                                                   +------------+       \________

I would like to route the clients in network 10.0.0.0/24 by source IP to different gateways.
The interface to the client network is eth0/10.0.0.1, which is the default gateway for all clients.

For example:
10.0.0.11 should be routed to Provider1 @ eth1
10.0.0.12 should be routed to Provider2 @ eth2
...and so on...

I think I need to use ip route and iptables for SNAT, but I have not figured out exactly how.
Here is the script I have so far.
ipv4 forwarding is enabled.

#!/bin/bash
# flush tables
ip route flush table connection1
ip route flush table connection2
ip route flush table connection3

# add the default gateways for each table
ip route add table connection1 default via 192.168.1.1
ip route add table connection2 default via 192.168.2.1
ip route add table connection3 default via 192.168.3.1

# add some IP addresses for marking
iptables -t mangle -A PREROUTING -s 10.0.0.11 -j MARK --set-mark 1
iptables -t mangle -A PREROUTING -s 10.0.0.12 -j MARK --set-mark 2
iptables -t mangle -A PREROUTING -s 10.0.0.13 -j MARK --set-mark 3

# add the source nat rules for each outgoing interface
iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 192.168.1.2
iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 192.168.2.2
iptables -t nat -A POSTROUTING -o eth3 -j SNAT --to-source 192.168.3.2

# link routing tables to connections (?)
ip rule add fwmark 1 table connection1
ip rule add fwmark 2 table connection2
ip rule add fwmark 3 table connection3

#default route for anything not configured above should be eth2
share|improve this question
    
You need to add in CONNMARK, I think, to save/restore the mark so it can be applied to packets 2..n (which will be NAT'd by connection tracking) –  derobert Aug 23 '13 at 22:26
    
I've added an answer with excerpts from the config we use here. I'll try to check in over the weekend to clarify anything... –  derobert Aug 23 '13 at 22:49
add comment

1 Answer

up vote 10 down vote accepted

Here is a similar setup from one of our routers (with some irrelevant stuff snipped). Note that this handles incoming connections as well.

Note the use of variables instead of hard-coded mark numbers. So much easier to maintain! They're stored in a separate script, and sourced in. Table names are configured in /etc/iproute2/rt_tables. Interface names are set in /etc/udev/rules.d/70-persistent-net.rules.

##### fwmark ######
iptables -t mangle -F
iptables -t mangle -X

iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A PREROUTING -m mark ! --mark 0 -j RETURN # if already set, we're done
iptables -t mangle -A PREROUTING -i wan      -j MARK --set-mark $MARK_CAVTEL
iptables -t mangle -A PREROUTING -i comcast  -j MARK --set-mark $MARK_COMCAST
iptables -t mangle -A PREROUTING -i vz-dsl   -j MARK --set-mark $MARK_VZDSL

iptables -t mangle -A POSTROUTING -o wan     -j MARK --set-mark $MARK_CAVTEL
iptables -t mangle -A POSTROUTING -o comcast -j MARK --set-mark $MARK_COMCAST
iptables -t mangle -A POSTROUTING -o vz-dsl  -j MARK --set-mark $MARK_VZDSL
iptables -t mangle -A POSTROUTING -j CONNMARK --save-mark

##### NAT ######
iptables -t nat -F
iptables -t nat -X
for local in «list of internal IP/netmask combos»; do
    iptables -t nat -A POSTROUTING -s $local -o wan     -j SNAT --to-source «IP»
    iptables -t nat -A POSTROUTING -s $local -o comcast -j SNAT --to-source «IP»
    iptables -t nat -A POSTROUTING -s $local -o vz-dsl  -j SNAT --to-source «IP»
done

# this is an example of what the incoming traffic rules look like
for extip in «list of external IPs»; do
    iptables -t nat -A PREROUTING   -p tcp -d $extip --dport «port» -j DNAT --to-destination «internal-IP»:443
done

And the rules:

ip rule flush
ip rule add from all               pref 1000  lookup main 
ip rule add from A.B.C.D/29        pref 1500  lookup comcast # these IPs are the external ranges (we have multiple IPs on each connection)
ip rule add from E.F.G.H/29        pref 1501  lookup cavtel
ip rule add from I.J.K.L/31        pref 1502  lookup vzdsl
ip rule add from M.N.O.P/31        pref 1502  lookup vzdsl # yes, you can have multiple ranges
ip rule add fwmark $MARK_COMCAST   pref 2000  lookup comcast
ip rule add fwmark $MARK_CAVTEL    pref 2001  lookup cavtel
ip rule add fwmark $MARK_VZDSL     pref 2002  lookup vzdsl
ip rule add                        pref 2500  lookup comcast # the pref order here determines the default—we default to Comcast.
ip rule add                        pref 2501  lookup cavtel
ip rule add                        pref 2502  lookup vzdsl
ip rule add                        pref 32767 lookup default

The routing tables get set up in /etc/network/interfaces, so that taking down an interface makes it switch to using a different one:

iface comcast inet static
        address A.B.C.Q
        netmask 255.255.255.248
        up ip route add table comcast default via A.B.C.R dev comcast
        down ip route flush table comcast

Note: If you're doing filtering as well (which you probably are) you'll also need to add the appropriate rules to FORWARD to ACCEPT the traffic. Especially for any incoming traffic.

share|improve this answer
    
Thank you very much! I am now going to modify this to my needs, load it on the box and update this post. –  Flav Aug 24 '13 at 7:10
    
Works like a charm, thanks again. Except for the pref order / default route to 'comcast'. (For me that should be eth2 ) But I think I worked around it by adding a general rule ip rule add from 10.0.0.0/24 pref 1400 lookup eth2 and creating the exceptions afterwards. –  Flav Aug 24 '13 at 8:11
1  
@Flav you can also set your exceptions with the firewall marks (in PREROUTING). BTW: One of the linked questions (unix.stackexchange.com/questions/70440/…) has more explanation of part of this config. Those ip/mask rules are actually for non-NAT'd traffic in my config (SNAT happens in POSTROUTING, hence after the ip rule stuff) –  derobert Aug 25 '13 at 8:25
    
Thanks for the update, appreciate it. –  Flav Aug 25 '13 at 9:42
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.