1

There is the requirement to set up a stateless NAT for two UDP connections from a physical network adapter in global network namespace via a linked pair of virtual network adapters to a service running in a special network namespace. This should be done on a CPU (Intel Atom) in an industrial device running Linux (Debian) with kernel 5.9.7.

Here is a scheme of the network configuration which should be set up:

=====================    =====================================================
|| application CPU ||    ||                communication CPU                ||
||                 ||    ||                                                 ||
||                 ||    ||    global namespace    |   nsprot1 namespace    ||
||                 ||    ||                        |                        ||
||     enp4s0      ||    ||       enp1s0           |          enp3s0        ||
||    0.0.0.5/30  ==========     0.0.0.6/30        |    192.168.2.15/24    =======
||                 ||    ||                        |                        ||
|| UDP port 50001  ||    || UDP port 50001 for sv1 |  TCP port 2404 for sv2 ||
|| UDP port 50002  ||    || UDP port 50002 for sv1 |                        ||
|| UDP port 53401  ||    || UDP port 50401 for sv1 |                        ||
|| UDP port 53402  ||    || UDP port 50402 for sv1 |                        ||
||                 ||    ||                        |                        ||
||                 ||    ||      vprot0            |         vprot1         ||
||                 ||    ||     0.0.0.16/31       ---      0.0.0.17/31      ||
||                 ||    ||                        |                        ||
|| UDP port 53404  ||    || UDP port 50404 for sv2 - UDP port 50404 for sv2 ||
|| UDP port 53441  ||    || UDP port 50441 for sv2 - UDP port 50441 for sv2 ||
=====================    =====================================================

The application CPU always starts first and opens several UDP ports for communication with service sv1 and service sv2 on the communication CPU via its physical network adapter enp4s0 with the IP address 0.0.0.5.

The output of ss --ipv4 --all --numeric --processes --udp executed on application CPU is:

Netid  State   Recv-Q  Send-Q   Local Address:Port    Peer Address:Port   Process
udp    UNCONN  0       0              0.0.0.0:50001        0.0.0.0:*      users:(("sva",pid=471,fd=5))
udp    UNCONN  0       0              0.0.0.0:50002        0.0.0.0:*      users:(("sva",pid=471,fd=6))
udp    ESTAB   0       0              0.0.0.5:53401        0.0.0.6:50401  users:(("sva",pid=471,fd=12))
udp    ESTAB   0       0              0.0.0.5:53402        0.0.0.6:50402  users:(("sva",pid=471,fd=13))
udp    ESTAB   0       0              0.0.0.5:53404        0.0.0.6:50404  users:(("sva",pid=471,fd=19))
udp    ESTAB   0       0              0.0.0.5:53441        0.0.0.6:50441  users:(("sva",pid=471,fd=21))

The communication CPU starts second and has finally two services running:

  • sv1 in global namespace and
  • sv2 in special network namespace nsprot1.

The output of ss --ipv4 --all --numeric --processes --udp executed in global namespace of the communication CPU is:

Netid  State   Recv-Q  Send-Q   Local Address:Port    Peer Address:Port   Process
udp    UNCONN  0       0              0.0.0.0:50001        0.0.0.0:*      users:(("sv1",pid=812,fd=18))
udp    UNCONN  0       0              0.0.0.6:50002        0.0.0.0:*      users:(("sv1",pid=812,fd=17))
udp    UNCONN  0       0              0.0.0.6:50401        0.0.0.0:*      users:(("sv1",pid=812,fd=13))
udp    UNCONN  0       0              0.0.0.6:50402        0.0.0.0:*      users:(("sv1",pid=812,fd=15))

The output of ip netns exec nsprot1 ss --ipv4 --all --numeric --processes --udp (nsprot1 namespace) is:

Netid  State   Recv-Q  Send-Q   Local Address:Port    Peer Address:Port   Process
udp    ESTAB   0       0             0.0.0.17:50404        0.0.0.5:53404  users:(("sv2",pid=2421,fd=11))
udp    ESTAB   0       0             0.0.0.17:50441        0.0.0.5:53441  users:(("sv2",pid=2421,fd=12))

Forwarding for IPv4 is enabled in sysctl in general and for all physical network adapters.
Just broadcast and multicast forwarding is disabled as not needed and not wanted.

The network configuration is set up on communication CPU with the following commands:

ip netns add nsprot1
ip link add vprot0 type veth peer name vprot1 netns nsprot1
ip link set dev enp3s0 netns nsprot1
ip address add 0.0.0.16/31 dev vprot0
ip netns exec nsprot1 ip address add 0.0.0.17/31 dev vprot1
ip netns exec nsprot1 ip address add 192.168.2.15/24 dev enp3s0
ip link set dev vprot0 up
ip netns exec nsprot1 ip link set vprot1 up
ip netns exec nsprot1 ip link set enp3s0 up
ip netns exec nsprot1 ip route add 0.0.0.4/30 via 0.0.0.16 dev vprot1

The network address translation is set up with the following commands:

nft add table ip prot1
nft add chain ip prot1 prerouting '{ type nat hook prerouting priority -100; policy accept; }'
nft add rule prot1 prerouting iif enp1s0 udp dport '{ 50404, 50441 }' dnat 0.0.0.17
nft add chain ip prot1 postrouting '{ type nat hook postrouting priority 100; policy accept; }'
nft add rule prot1 postrouting ip saddr 0.0.0.16/31 oif enp1s0 snat 0.0.0.6

The output of nft list table ip prot1 is:

table ip prot1 {
    chain prerouting {
        type nat hook prerouting priority -100; policy accept;
        iif "enp1s0" udp dport { 50404, 50441 } dnat to 0.0.0.17
    }

    chain postrouting {
        type nat hook postrouting priority 100; policy accept;
        ip saddr 0.0.0.16/31 oif "enp1s0" snat to 0.0.0.6
    }
}

There is defined additionally in global namespace only the table inet filter with:

table inet filter {
    chain input {
        type filter hook input priority 0; policy accept;
    }

    chain forward {
        type filter hook forward priority 0; policy accept;
    }

    chain output {
        type filter hook output priority 0; policy accept;
    }
}

That NAT configuration is for a stateful NAT. It works for the UDP channel with the port numbers 50404 and 53404 because of sv2 started last opens 0.0.0.17:50404 and sends a UDP packet to 0.0.0.5:53404 on which source network address translation is applied in postrouting hook for enp1s0 in global namespace. The service sva of application CPU sends back a UDP packet from 0.0.0.5:53404 to 0.0.0.6:50404 which reaches 0.0.0.17:50404. The UDP packet does not pass the prerouting rule for dnat to 0.0.0.17. It is send directly via connection tracking to 0.0.0.17 as I found out later.

But this stateful NAT configuration does not work for the UDP channel with the port numbers 50441 and 534441. It looks like the reason is that sva of application CPU sends several UDP packets already from 0.0.0.5:53441 to 0.0.0.6:50441 before the service sv2 is started at all and the destination port is opened in network namespace nsprot1. There is returned by ICMP that the destination port is unreachable. That is no surprise on taking into account that the destination port is not yet opened at all. It is unfortunately not possible to block the UDP packet sends in service sva until service sv2 is started and opened the two UDP ports. Service sva sends periodically and sometimes additionally triggered spontaneous UDP packets from 0.0.0.5:53441 to 0.0.0.6:50441 independent on connection state.

So the problem with this configuration seems to be the stateful NAT as the dnat rule in prerouting hook is still not used on destination port finally opened in network namespace nsprot1. There is still continued to route the UDP packets to 0.0.0.6:50441 which results in dropping the UDP packet and returning with ICMP that the destination port is not reachable.

Therefore the solution is maybe the usage of a stateless NAT. So there are executed additionally the commands:

nft add table ip raw
nft add chain ip raw prerouting '{ type filter hook prerouting priority -300; policy accept; }'
nft add rule ip raw prerouting udp dport '{ 50404, 50441, 53404, 53441 }' notrack

But the result was not as expected. The prerouting rule to change the destination address from 0.0.0.6 to 0.0.0.17 for UDP packets from input interface enp1s0 with destination port 50404 and 50441 is still not taken into account.

There was executed next by me:

nft add table ip filter
nft add chain filter trace_in '{ type filter hook prerouting priority -301; }'
nft add rule filter trace_in meta nftrace set 1
nft add chain filter trace_out '{ type filter hook postrouting priority 99; }'
nft add rule filter trace_out meta nftrace set 1
nft monitor trace

I looked on the trace and could see that the notrack rule is taken into account, but then the UDP packets with destination port 50441 are passed directly to the input hook. I don't know why.

I studied many, many hours very carefully following pages:

  • nft manual (read several times completely from top to bottom)
  • nftables wiki (most pages completely)
  • nftables on ArchWiki
  • and many, many other web pages regarding to usage of network namespaces and network address translation.

I tried really many different configurations, used Wireshark, used nft monitor trace, but I cannot find out a solution which works for the UDP channel with the ports 50441 and 53441 on sva sending UDP packets already before destination port 0.0.0.17:50441 is opened at all.

The stateful NAT configuration works if I manually terminate on application CPU the service sva, set up the network configuration on communication CPU with starting the two services sv1 and sv2 and start last manually the service sva again on all UDP ports already opened on communication CPU. But this order of starting the services cannot be done in the industrial device by default. The application service sva must run independent on communication services are ready for communication or not.

Which commands (chains/rules) are necessary to have a stateless NAT for the two UDP channels 0.0.0.5:53404 - 0.0.0.17:50404 and 0.0.0.5:53441 - 0.0.0.17:50441 independent on the open states of the destination ports and which service sends first an UDP packet to the other service?

PS: The service sv2 can be started depending on configuration of the device also in global namespace using a different physical network adapter on which no NAT and network namespace are necessary. In this network configuration there is absolutely no problem with the UDP communication between the three services.

0

I found the solution by myself finally after many, many hours of reading documentations, tutorials, suggestions on various web pages, making lots of trials, and doing deep and comprehensive network and netfilter monitorings and analyzes.

nft add table ip prot1
nft add chain ip prot1 prerouting '{ type filter hook prerouting priority -300; policy accept; }'
nft add rule ip prot1 prerouting iif enp1s0 udp dport '{ 50404, 50441 }' ip daddr set 0.0.0.17 notrack accept
nft add rule ip prot1 prerouting iif vprot0 ip saddr 0.0.0.17 notrack accept
nft add chain ip prot1 postrouting '{ type filter hook postrouting priority 100; policy accept; }'
nft add rule ip prot1 postrouting oif enp1s0 ip saddr 0.0.0.17 ip saddr set 0.0.0.6 accept

The netfilter hooks page should be opened and read first to understand the following explanation.

Explanation for the used commands:

  1. A netfilter table is added for protocol ip (IPv4) with name prot1.
  2. A chain is added to table prot1 with name prerouting of type filter for the hook prerouting with priority -300. It is important to use a priority number lower than -200 to be able to bypass the connection tracking conntrack. That excludes the usage of a chain of type nat for the destination network address translation as having an even lower priority.
  3. A filter rule is added to table prot1 to chain prerouting which is applied only on IPv4 packets received on input interface enp1s0 of protocol type udp having as destination port either 50404 or 50441 which modifies the ip destination address of the packet from 0.0.0.6 to 0.0.0.17 and activates no tracking of the connection for this UDP packet. The verdict is specified explicitly with accept although not really necessary to pass the UDP packet received from the service sva of application CPU for the service sv2 of communication CPU as fast as possible to the next hook which is in this case the forward hook.
  4. A second filter rule is added to table prot1 to chain prerouting which is applied only on all IPv4 packets received on input interface vprot0 independent on protocol type (udp, icmp, ...) having the ip source address 0.0.0.17 to activate no tracking of the connection for this packet. It would be of course also possible to filter just on UDP packets with appropriate source or destination port number, but this additional limitation is not needed here and this rule is also good for ICMP packets send back from 0.0.0.17 to 0.0.0.5 on destination port not yet opened because of the service sv2 is not running at the moment. The verdict is again specified explicitly with accept instead of using the implicit default continue to pass the packet as fast as possible to the forward hook.
  5. A second chain is added to table prot1 with name postrouting of type filter for the hook postrouting with priority 100. It is important to use a chain of type filter and not of type nat to be able to apply a source address translation on the UDP (and ICMP) packets which bypassed the connection tracking.
  6. A filter rule is added to table prot1 to second chain postrouting which is applied only on IPv4 packets sent on output interface enp1s0 independent on protocol type (udp, icmp, ...) having as source address 0.0.0.17 which modifies the ip source address of the packet from 0.0.0.17 to 0.0.0.6. The verdict is specified once more explicitly with accept although not really necessary to pass the UDP packet received from the service sv2 of the communication CPU to the service sva of the application CPU as fast as possible. This rule changes also the source address to 0.0.0.6 of the ICMP packet sent from 0.0.0.17 on destination port not reachable because of the service sv2 is not yet running. So the application CPU never notices that it communicates for two UDP channels with a different interface than 0.0.0.6 which was a second requirement to fulfill although being not really important.

It was a hard work to find out that a stateless network translation was needed for this very special network configuration and kind of communication between the services sva and sv2 and that the NAT must be done without using the nat hook.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.