I'm trying to get some traffic to go through a VPN and other traffic to not do so.

Packets with a given fwmark are supposed to go to my default interface (wlo1), and all other traffic to a tunnel interface (tun0, using OpenVPN) in the main table. I have added this rule in nftables:

table ip test {
    chain test {
        type route hook output priority mangle; policy accept;
        meta cgroup 1234 meta mark set 1

In the main routing table I have these entries:

default via dev tun0 dev tun0 proto kernel scope link src 
[WANIP.0]/24 dev wlo1 proto kernel scope link src [WANIP] via dev tun0 
[VPNIP] via [WANGATEWAY] dev wlo1

Packets with fwmark 1 is instead led to their own routing table: ip rule add from all fwmark 1 lookup test. In the test table I have added the following route:

default via [WANGATEWAY] dev wlo1

When I run ping from this cgroup, it's stuck. It seems able to send but not receive any packets.

VPN traffic works as expected.

What exactly is going on?

  • you can try disabling rp_filter and see if that helps
    – A.B
    Mar 30, 2020 at 15:18
  • Thanks. It seems to make no difference.
    – user402922
    Mar 30, 2020 at 15:26
  • If I specify the interface, it seems to work: ping -I wlo1
    – user402922
    Mar 30, 2020 at 16:42
  • got it working. But this requires a few tweaks
    – A.B
    Mar 30, 2020 at 16:52

1 Answer 1


When a packet is emitted, there's a routing decision made: this decision chooses the outgoing interface and the matching source IP address to use.

When the route/output chain sets a mark, it triggers a reroute check, as seen in this schematic (which was made for iptables in mind but is totally usable for nftables). The reroute check alters the route... but doesn't change the source IP address. So more work has to be done.

  • add a NAT rule to alter the source IP address.

This has to be done after the reroute check, so it's done in nat/postrouting. Note that the same table can have different chain types (contrary to iptables where table <=> type).

    nft add chain ip test testnat '{ type nat hook postrouting priority srcnat; policy accept; }'
    nft add rule ip test testnat meta mark 1 masquerade

Now the correct packet really leaves.

  • and allow reply flow to be accepted despite not arriving through the default route
  1. You can either relax Reverse Path Forwarding to Loose mode by changing rp_filter:

        sysctl -w net.ipv4.conf.wlo1.rp_filter=2
  2. or mark reply flow to use the same routing table as outgoing flow. It won't really improve security but anyway:

        nft add chain ip test testpre '{ type filter hook prerouting priority mangle; policy accept; }'
        nft add rule ip test testpre iif "wlo1" meta mark set 1

Alas, this won't work without yet-an-other-tweak which is needed since a new undocumented feature appeared in 2010:

        sysctl -w net.ipv4.conf.wlo1.src_valid_mark=1


  • It's possible to do better security-wise by storing the mark in conntrack's connmark (ct mark) to only allow the correct reply flows and not anything else to bypass the Strict Reverse Path Forwarding. In Strict mode, nothing will reach through wlo1 anymore unless it's a reply from outgoing traffic. Here's the full corresponding nftables rule file for this (to be used with option 2. above while replacing nftables rules):

      table ip test {
              chain test {
                      type route hook output priority mangle; policy accept;
                      meta cgroup 1234 meta mark set 1 ct mark set meta mark
              chain testnat {
                      type nat hook postrouting priority srcnat; policy accept;
                      meta mark 1 masquerade
              chain testpre {
                      type filter hook prerouting priority mangle; policy accept;
                      ct mark 1 meta mark set ct mark
  • Also, the mark reroute check might interfere with correct PMTU / TCP MSS in some cases, according to what I could read there in the uidrange entry: https://kernelnewbies.org/Linux_4.10#Networking

  • If you can convert the cgroups usage into a limited set of uids instead, then this can be done correctly using only the routing stack, without netfilter or nftables, using ip rule add ... uidrange ... as described above. See my answer there about it: Routing traffic for a user through specific interface (tum1).

  • Thanks for the detailed answer. Is it possible to set the source address using only the routing tables?
    – user402922
    Mar 30, 2020 at 20:23
  • as I wrote in the last note: if you're using a mark you can't. so if you're using cgroups you can't either. but altering route for a specific user is possible since kernel 4.10 directly
    – A.B
    Mar 31, 2020 at 0:31
  • using container technology helps: either as network namespace to be routed instead of being the local endpoint, then there's no trouble with the IP when prerouting (and just requires ip rule, probably no mark), either as a user namespace in the same host network: you can map the whole translated uidrange for use by ip rule and then again there's no problem with the initial choice of outgoing IP.
    – A.B
    Mar 31, 2020 at 9:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .