There is the requirement to set up a stateless NAT for two UDP connections from a physical network adapter in global network namespace via a linked pair of virtual network adapters to a service running in a special network namespace. This should be done on a CPU (Intel Atom) in an industrial device running Linux (Debian) with kernel 5.9.7.
Here is a scheme of the network configuration which should be set up:
===================== =====================================================
|| application CPU || || communication CPU ||
|| || || ||
|| || || global namespace | nsprot1 namespace ||
|| || || | ||
|| enp4s0 || || enp1s0 | enp3s0 ||
|| 0.0.0.5/30 ========== 0.0.0.6/30 | 192.168.2.15/24 =======
|| || || | ||
|| UDP port 50001 || || UDP port 50001 for sv1 | TCP port 2404 for sv2 ||
|| UDP port 50002 || || UDP port 50002 for sv1 | ||
|| UDP port 53401 || || UDP port 50401 for sv1 | ||
|| UDP port 53402 || || UDP port 50402 for sv1 | ||
|| || || | ||
|| || || vprot0 | vprot1 ||
|| || || 0.0.0.16/31 --- 0.0.0.17/31 ||
|| || || | ||
|| UDP port 53404 || || UDP port 50404 for sv2 - UDP port 50404 for sv2 ||
|| UDP port 53441 || || UDP port 50441 for sv2 - UDP port 50441 for sv2 ||
===================== =====================================================
The application CPU always starts first and opens several UDP ports for communication with service sv1 and service sv2 on the communication CPU via its physical network adapter enp4s0 with the IP address 0.0.0.5.
The output of ss --ipv4 --all --numeric --processes --udp executed on application CPU is:
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
udp UNCONN 0 0 0.0.0.0:50001 0.0.0.0:* users:(("sva",pid=471,fd=5))
udp UNCONN 0 0 0.0.0.0:50002 0.0.0.0:* users:(("sva",pid=471,fd=6))
udp ESTAB 0 0 0.0.0.5:53401 0.0.0.6:50401 users:(("sva",pid=471,fd=12))
udp ESTAB 0 0 0.0.0.5:53402 0.0.0.6:50402 users:(("sva",pid=471,fd=13))
udp ESTAB 0 0 0.0.0.5:53404 0.0.0.6:50404 users:(("sva",pid=471,fd=19))
udp ESTAB 0 0 0.0.0.5:53441 0.0.0.6:50441 users:(("sva",pid=471,fd=21))
The communication CPU starts second and has finally two services running:
sv1in global namespace andsv2in special network namespacensprot1.
The output of ss --ipv4 --all --numeric --processes --udp executed in global namespace of the communication CPU is:
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
udp UNCONN 0 0 0.0.0.0:50001 0.0.0.0:* users:(("sv1",pid=812,fd=18))
udp UNCONN 0 0 0.0.0.6:50002 0.0.0.0:* users:(("sv1",pid=812,fd=17))
udp UNCONN 0 0 0.0.0.6:50401 0.0.0.0:* users:(("sv1",pid=812,fd=13))
udp UNCONN 0 0 0.0.0.6:50402 0.0.0.0:* users:(("sv1",pid=812,fd=15))
The output of ip netns exec nsprot1 ss --ipv4 --all --numeric --processes --udp (nsprot1 namespace) is:
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
udp ESTAB 0 0 0.0.0.17:50404 0.0.0.5:53404 users:(("sv2",pid=2421,fd=11))
udp ESTAB 0 0 0.0.0.17:50441 0.0.0.5:53441 users:(("sv2",pid=2421,fd=12))
Forwarding for IPv4 is enabled in sysctl in general and for all physical network adapters.
Just broadcast and multicast forwarding is disabled as not needed and not wanted.
The network configuration is set up on communication CPU with the following commands:
ip netns add nsprot1
ip link add vprot0 type veth peer name vprot1 netns nsprot1
ip link set dev enp3s0 netns nsprot1
ip address add 0.0.0.16/31 dev vprot0
ip netns exec nsprot1 ip address add 0.0.0.17/31 dev vprot1
ip netns exec nsprot1 ip address add 192.168.2.15/24 dev enp3s0
ip link set dev vprot0 up
ip netns exec nsprot1 ip link set vprot1 up
ip netns exec nsprot1 ip link set enp3s0 up
ip netns exec nsprot1 ip route add 0.0.0.4/30 via 0.0.0.16 dev vprot1
The network address translation is set up with the following commands:
nft add table ip prot1
nft add chain ip prot1 prerouting '{ type nat hook prerouting priority -100; policy accept; }'
nft add rule prot1 prerouting iif enp1s0 udp dport '{ 50404, 50441 }' dnat 0.0.0.17
nft add chain ip prot1 postrouting '{ type nat hook postrouting priority 100; policy accept; }'
nft add rule prot1 postrouting ip saddr 0.0.0.16/31 oif enp1s0 snat 0.0.0.6
The output of nft list table ip prot1 is:
table ip prot1 {
chain prerouting {
type nat hook prerouting priority -100; policy accept;
iif "enp1s0" udp dport { 50404, 50441 } dnat to 0.0.0.17
}
chain postrouting {
type nat hook postrouting priority 100; policy accept;
ip saddr 0.0.0.16/31 oif "enp1s0" snat to 0.0.0.6
}
}
There is defined additionally in global namespace only the table inet filter with:
table inet filter {
chain input {
type filter hook input priority 0; policy accept;
}
chain forward {
type filter hook forward priority 0; policy accept;
}
chain output {
type filter hook output priority 0; policy accept;
}
}
That NAT configuration is for a stateful NAT. It works for the UDP channel with the port numbers 50404 and 53404 because of sv2 started last opens 0.0.0.17:50404 and sends a UDP packet to 0.0.0.5:53404 on which source network address translation is applied in postrouting hook for enp1s0 in global namespace. The service sva of application CPU sends back a UDP packet from 0.0.0.5:53404 to 0.0.0.6:50404 which reaches 0.0.0.17:50404. The UDP packet does not pass the prerouting rule for dnat to 0.0.0.17. It is send directly via connection tracking to 0.0.0.17 as I found out later.
But this stateful NAT configuration does not work for the UDP channel with the port numbers 50441 and 534441. It looks like the reason is that sva of application CPU sends several UDP packets already from 0.0.0.5:53441 to 0.0.0.6:50441 before the service sv2 is started at all and the destination port is opened in network namespace nsprot1. There is returned by ICMP that the destination port is unreachable. That is no surprise on taking into account that the destination port is not yet opened at all. It is unfortunately not possible to block the UDP packet sends in service sva until service sv2 is started and opened the two UDP ports. Service sva sends periodically and sometimes additionally triggered spontaneous UDP packets from 0.0.0.5:53441 to 0.0.0.6:50441 independent on connection state.
So the problem with this configuration seems to be the stateful NAT as the dnat rule in prerouting hook is still not used on destination port finally opened in network namespace nsprot1. There is still continued to route the UDP packets to 0.0.0.6:50441 which results in dropping the UDP packet and returning with ICMP that the destination port is not reachable.
Therefore the solution is maybe the usage of a stateless NAT. So there are executed additionally the commands:
nft add table ip raw
nft add chain ip raw prerouting '{ type filter hook prerouting priority -300; policy accept; }'
nft add rule ip raw prerouting udp dport '{ 50404, 50441, 53404, 53441 }' notrack
But the result was not as expected. The prerouting rule to change the destination address from 0.0.0.6 to 0.0.0.17 for UDP packets from input interface enp1s0 with destination port 50404 and 50441 is still not taken into account.
There was executed next by me:
nft add table ip filter
nft add chain filter trace_in '{ type filter hook prerouting priority -301; }'
nft add rule filter trace_in meta nftrace set 1
nft add chain filter trace_out '{ type filter hook postrouting priority 99; }'
nft add rule filter trace_out meta nftrace set 1
nft monitor trace
I looked on the trace and could see that the notrack rule is taken into account, but then the UDP packets with destination port 50441 are passed directly to the input hook. I don't know why.
I studied many, many hours very carefully following pages:
- nft manual (read several times completely from top to bottom)
- nftables wiki (most pages completely)
- nftables on ArchWiki
- and many, many other web pages regarding to usage of network namespaces and network address translation.
I tried really many different configurations, used Wireshark, used nft monitor trace, but I cannot find out a solution which works for the UDP channel with the ports 50441 and 53441 on sva sending UDP packets already before destination port 0.0.0.17:50441 is opened at all.
The stateful NAT configuration works if I manually terminate on application CPU the service sva, set up the network configuration on communication CPU with starting the two services sv1 and sv2 and start last manually the service sva again on all UDP ports already opened on communication CPU. But this order of starting the services cannot be done in the industrial device by default. The application service sva must run independent on communication services are ready for communication or not.
Which commands (chains/rules) are necessary to have a stateless NAT for the two UDP channels 0.0.0.5:53404 - 0.0.0.17:50404 and 0.0.0.5:53441 - 0.0.0.17:50441 independent on the open states of the destination ports and which service sends first an UDP packet to the other service?
PS: The service sv2 can be started depending on configuration of the device also in global namespace using a different physical network adapter on which no NAT and network namespace are necessary. In this network configuration there is absolutely no problem with the UDP communication between the three services.
