Output traffic on different interfaces based on destination port

Question

My question is basically the same as Only allow certain outbound traffic on certain interfaces.

I have two interfaces eth1 (10.0.0.2) and wlan0 (192.168.0.2). My default route is for eth1. Let's say I want all https-traffic to go through wlan0. Now if I use the solution suggested in the other question, https traffic will go through wlan0, but will still have the source-address of eth1 (10.0.0.2). Since this address is not routeable for the wlan0 gateway, answers won't ever come back. The easy way would be to just set the bind-addr properly in the application, but in this case it is not applicable.

I figure I need to rewrite the src-addr:

# first mark it so that iproute can route it through wlan0
iptables -A OUTPUT -t mangle -o eth1 -p tcp --dport 443 -j MARK --set-mark 1
# now rewrite the src-addr
iptables -A POSTROUTING -t nat -o wlan0 -p tcp --dport 443 -j SNAT --to 192.168.0.2

Now tcpdump sees the outgoing packets just fine and ingoing packets arrive for 192.168.0.2, however they probably never end up in the application, because all I ever get to see, is that the application is resending the SYN-packet, although the SYN-ACK was already received.

So I thought, maybe I need to rewrite the incoming address too:

iptables -A PREROUTING -t nat -i wlan0 -p tcp --sport 443 -j DNAT --to 10.0.0.2

but that didn't work either. So I’m kind of stuck here. Any suggestions?

Answer 1

You're close.

The actual reason that the application isn't seeing the return traffic is because of the kernel's built in IP spoofing protection. I.e., the return traffic doesn't match the routing table and is therefore dropped. You can fix this by turning off spoofing protection like this:

sudo sysctl net.ipv4.conf.wlan0.rp_filter=0

But I wouldn't recommend it. The more proper way is to create an alternate routing instance.

1. The mark is necessary. Keep it.
2. Source NAT is also necessary.
3. The final DNAT is unnecessary, so you can remove it.

Make sure you have the iproute package installed. If you have the ip command then you're set (which it looks like you do, but if not get that first).

Edit /etc/iproute2/rt_tables and add a new table by appending the following line:

200 wlan-route

You then need to configure your new routing table named wlan-route with a default gateway and create rules to conditionally send traffic to that table. I'll assume your default gateway is 192.168.0.1. Naturally this needs to match your actual network, and not just my assumptions.

ip route add default via 192.168.0.1 dev wlan0 table wlan-route
ip rule add fwmark 0x1 table wlan-route

Your final annotated script would look like this:

# Populate secondary routing table
ip route add default via 192.168.0.1 dev wlan0 table wlan-route
# Anything with this fwmark will use the secondary routing table
ip rule add fwmark 0x1 table wlan-route
# Mark these packets so that iproute can route it through wlan-route
iptables -A OUTPUT -t mangle -o eth1 -p tcp --dport 443 -j MARK --set-mark 1
# now rewrite the src-addr
iptables -A POSTROUTING -t nat -o wlan0 -p tcp --dport 443 -j SNAT --to 192.168.0.2

Answer 2

bahamat's solution is correct; however, please note that the only way for me to make this work was to disable the rp_filter for every interface in the system, not only the two (eth1 and wlan0 in this case) involved in the NATing.

for f in /proc/sys/net/ipv4/conf/*/rp_filter; do echo 0 > $f; done
echo 1 > /proc/sys/net/ipv4/route/flush

(see the IMPORTANT note at the end of this page: Advanced Routing Howto -- the link posted there doesn't exist anymore, but I found it through the wayback machine)

Answer 3

One suggestion: you should always use --sport instead of --dport in output chain.

NATs change the dport and that will make your rule unuasble.

Answer 4

I think below is needed:

ip ru add from 192.168.0.2 table 3 prio 300
ip ro add table 3 default via 192.168.0.1 dev wlan0

Answer 5

I try to do a similar thing. I have two interfaces wwan0 (metric 700) and eth0 (metric 100). I want all the https (443) traffic to go via wwan0. I follow the example above posted, it works for the first few packets, and then I start seeing FIN-WAIT-1 states and then it stops working, the list of FIN-WAIT-1 gets bigger. No sure what is happening. The system boots up and starts sending some 443 traffic to eth0, then I run the script, the traffic goes to wwan0 but after a few hundred packets something saturates and it keeps giving FIN-WAIT-1.

I set it up with the following:

ip route add default dev wwan0 table wwan-route
ip rule add fwmark 0x1 table wwan-route
iptables -A OUTPUT -t mangle -p tcp --dport 443 -j MARK --set-mark 1
nft add table nat
nft add chain nat post { type nat hook postrouting priority 0 \; }
nft add rule ip nat post oifname wwan0 tcp dport 443 masquerade

The route table :

default via 192.168.1.1 dev eth0 proto dhcp metric 100 
default via 10.5.1.101 dev wwan0 proto static metric 700 
10.5.1.96/29 dev wwan0 proto kernel scope link src 10.5.1.100 metric 700 
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.127 metric 100 

All ip route show tables all:

# ip route show table all
default via 192.168.1.1 dev eth0 table eth0rule 
default dev wwan0 table wwan-route scope link 
default via 192.168.1.1 dev eth0 proto dhcp metric 100 
default via 10.5.1.101 dev wwan0 proto static metric 700 
10.5.1.96/29 dev wwan0 proto kernel scope link src 10.5.1.100 metric 700 
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.127 metric 100 
broadcast 10.5.1.96 dev wwan0 table local proto kernel scope link src 10.5.1.100 
local 10.5.1.100 dev wwan0 table local proto kernel scope host src 10.5.1.100 
broadcast 10.5.1.103 dev wwan0 table local proto kernel scope link src 10.5.1.100 
broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1 
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1 
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1 
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1 
broadcast 192.168.1.0 dev eth0 table local proto kernel scope link src 192.168.1.127 
local 192.168.1.127 dev eth0 table local proto kernel scope host src 192.168.1.127 
broadcast 192.168.1.255 dev eth0 table local proto kernel scope link src 192.168.1.127 

The ip rules:

 ip rule
0:      from all lookup local
0:      from all fwmark 0x1 lookup wwan-route
1:      from 192.168.1.127 lookup eth0rule
32766:  from all lookup main
32767:  from all lookup default

And:

cat /etc/iproute2/rt_tables 
#
# reserved values
#
255     local
254     main
253     default
0       unspec
#
# local
#
#1      inr.ruhep
210 wwan-route
200 eth0rule

It is nearly there, something is missing though.

Answer 6

For modern Linux distros, this can be achieved entirely using policy routing, w/o iptables.

# First, match https traffic with ip rule, and force them to use a stand-alone routing table
sudo ip rule add dport 443 ipproto tcp table 10000

# Second, add default route to wlan0 in this table, assuming your gateway is 192.168.0.1
sudo ip route add default via 192.168.0.1 dev wlan0 src 192.168.0.2 table 10000