3

I have the following iptables rule:

iptables -t mangle -A PREROUTING -p udp -m udp --dport 10000 -j MARK --set-xmark 0x4/0xffffffff

which sets fwmark 4 on all udp packets with destination port 10000. I forward it to a tunnel (without any nat) with policy-based routing:

[root@localhost ~]# ip rule
0:  from all fwmark 0x4/0x4 lookup 87 
32765:  from all lookup local 
32766:  from all lookup main 
32767:  from all lookup default 
[root@localhost ~]# ip route show table 87
default dev tunnel  scope link 

Everything works correctly if the packet comes from a normal NIC (eth0 etc.), but packets that travel on the loopback interface (lo, for example packets generated with socat -u - UDP:localhost:10000) seem to skip the routing decision after the PREROUTING chain, and get received by the local host (which in fact replies with an ICMP port unreachable packet over the lo interface).

Is this expected behavior? If so how can I workaround that? I need packets not to have different paths for different input devices, as I want to use lo to test my more complex iptables ruleset (that is, an extra rule with nat won't be a solution for me).

2

There are a few things involved here.

Output from the local box uses the OUTPUT table, not PREROUTING. PREROUTING is for traffic flowing through the box.

The iptables rule only matches when a packet is sent. When the client opens a connection, it has to bind to an address before a single packet is generated. When it binds, it picks the source address according to the routing rules, without consulting iptables.

When you test using 127.0.0.1 as a destination and it sends the packet, it tries to send using 127.0.0.1 as the source address, which by default the kernel doesn't allow to be routed.

If you fix that issue, you'll then run into another problem when the packet leaves the interface and has a source address the remote system doesn't know how to return.

Thus the solution is 3 things:

1) Add the rule to the OUTPUT table:

iptables -t mangle -A OUTPUT -p udp -m udp --dport 10000 -j MARK --set-xmark 0x4/0xffffffff

2) Enable localnet routing with:

sysctl -w net.ipv4.conf.$iface.route_localnet=1

where $iface is your tunnel interface.

3) Add a MASQUERADE rule so that source address of traffic leaving the interface is rewritten:

iptables -t nat -A POSTROUTING -o $iface -m addrtype --src-type LOCAL -j MASQUERADE

 

Note though, that when testing sending traffic to 127.0.0.1, you're still likely to encounter issues. Nothing is rewriting the destination address, so it's geting routed out $iface with a destination of 127.0.0.1. The remote system is likely to reject this traffic.

  • Uhm yeah I initially thought that your answer was completely missing the point because you mentioned the nat table, but now I see the problem is the difference between OUTPUT and PREROUTING (such a dumb error on my side). – Lorenzo Pistone Sep 29 '14 at 22:15
  • Oh, well you are right, I did accidentally reference nat instead of mangle, but the same design is present there. I'll update the answer to reference mangle. – Patrick Sep 29 '14 at 22:18
  • sorry, with the same rules and route and adding iptables -t mangle -A OUTPUT -p udp -m udp --dport 10000 -j MARK --set-xmark 0x4/0xffffffff (which I could confirm it sets the mark with -j LOG), still the packet is received by the local host. It still seems there's something special about lo. – Lorenzo Pistone Sep 30 '14 at 12:06
  • I also modified my routing table 87 to have only blackhole default (which should kill all packets), still seeing the packet going through lo and receiving an ICMP port unreachable. I think strongly suggests that the table is not being consulted at all. – Lorenzo Pistone Sep 30 '14 at 12:21
  • Oh, sorry yes. If you're trying to access 127.0.0.1, you need to do more than this. This should work if you're using any other address on the host. The reason this doesn't work for 127.0.0.1 (localhost) is that this only affects packets once they get sent. IIRC, the application binds to an address before a single packet is sent. So it binds to 127.0.0.1, which is unroutable. If you add a MASQUERADE rule, and set net.ipv4.$iface.route_localnet, this should work. – Patrick Sep 30 '14 at 13:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.