nftables table and chain priority

Question

I have a problem with my nftables setup.

I have two tables, each one has a chain with the same hook but a different name and priority.

The tables are in different files which are loaded by an include argument.

Because of the priority, I would think that the VPN-POSTROUTING chain will be executed before the INTERNET chain. But in my setup, the INTERNET chain is executed first.

table ip nat {
        chain INTERNET {
                type nat hook postrouting priority srcnat + 1; policy accept;
                oifname br2 masquerade
        }
}

table ip vpn {
        chain VPN-POSTROUTING {
                type nat hook postrouting priority srcnat - 1; policy accept;
                oifname br2 ip saddr 10.0.0.0/24 ip daddr 192.168.0.0/24 accept
        }
}

where is my mistake?

Edit: I changed the rules and add all chains to the same table, with the same result.

In the next step, I followed A.B.'s advice and add counters and logs to the rules.

The order of the chains corresponds to the priority, but the accept rule for the VPN is not triggered.

When I add the VPN accept rule to the INTERNET chain, right before the masquerade rule, it works like expected.

Answer 1

While A.B's answer is correct, I'd like to amend it a bit and clarify the explanation.

All of the following statements are related to situations where the chains in question are registered at the same family and the same hook.

First, you may have misunderstood the word priority (and experience tells that you're not the only one if that's the case). A lower numerical priority number means higher priority, and a higher priority means that the respective chain is traversed earlier.

After a chain has been traversed, another chain may be traversed (see below). That other (later) chain may in fact override the verdict of the earlier one. Given that, it is very problematic to talk about priorities:

In normal speech, and in common sense, the chain of highest priority would be the chain that actually determines the final verdict, either overriding other chain's verdicts or blocking further chains from being executed.

However, in nftables terms, the chain with the highest priority is the chain that is executed first. In other words, it may be in fact be the chain of lowest priority that determines the final verdict. This definition of "priority" is very worrying IMHO; a better term perhaps would have been "evaluation order".

Second, if a chain has been traversed, it may be surprising that and under which circumstances further chains are traversed. The nftables wiki explains it:

NOTE: If a packet is accepted and there is another chain, bearing the same hook type and with a later priority, then the packet will subsequently traverse this other chain. Hence, an accept verdict - be it by way of a rule or the default chain policy - isn't necessarily final. However, the same is not true of packets that are subjected to a drop verdict. Instead, drops take immediate effect, with no further rules or chains being evaluated.

This explains at once what happens in your case. Most people would consider the accept verdict to be final, but that's not the case with nftables. Instead, after an accept verdict, traversal of the current chain is stopped, but later chains (if they exist, i.e. chains with lower priority) are traversed despite of the previous accept verdict.

Hence, in your case, when a packet comes in, it first traverses your VPN-POSTROUTING chain, because that chain is assigned the lower priority number. Since that chain ends with a accept verdict, the packet then traverses your INTERNET chain and gets masqueraded (making the VPN fail).

As for the solution: There are many ways to achieve what you want. A.B's proposition surely is a good one. However, I'm usually trying to not split up the rules too much. As you've noticed yourself, it's easy to make it work if you put your rules into one chain in the right order.

Answer 2

Historically there used to be one NAT chain in a given hook (prerouting, input, output, ...). Executing a nat statement or simply accept-ing the packet being terminal for the chain, with a single chain it was also ending treatment within the hook. With nftables allowing to use more than one chain in the same hook, terminating a chain will just continue to the next chain. So if the first chain doesn't do anything (by accept-ing), the next chain gets its chance to do it instead, which is not what is intended.

To solve this, the first chain (or any other chain) can leave a message passed to each next chain so it can act upon it: set a mark (an arbitrary value) for the next chain to change its behavior.

Instead of accept which has zero effect (leaving the chain VPN-POSTROUTING empty also executes the default policy: accept), set a mark. So replace the rule in VPN-POSTROUTING with this one instead:

nft add rule ip vpn VPN-POSTROUTING oifname br2 ip saddr 10.0.0.0/24 ip daddr 192.168.0.0/24 mark set 0xdeaf

When this mark is set, it can then be used on the other chain to change behavior by not executing remaining rules. Insert this rule first in ip nat INTERNET:

nft insert rule ip nat INTERNET mark 0xdeaf accept