I have two namespaces srv1 and srv2, interconnected via a softswitch (p4 bmv2) with veth pairs. The softswitch does just simple forwarding. The veth interfaces inside the namespaces have IP addresses assigned to them (respectively 192.168.1.1 and 192.168.1.2). I could ping between the two namespaces using those IP addresses:
sudo ip netns exec srv1 ping 192.168.1.2
PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data.
64 bytes from 192.168.1.2: icmp_seq=1 ttl=64 time=1.03 ms
64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=1.04 ms
but when I try netcat I don't receive messages on the server side:
client:
sudo ip netns exec srv1 netcat 192.168.1.2 80 -u
hello!
server:
sudo ip netns exec srv2 netcat -l 80 -u
The interface receives the packets with proper format. I verified with tcpdump on both namespaces and I saw the packets being sent and received properly:
client:
sudo ip netns exec srv1 tcpdump -XXvv -i srv1p
tcpdump: listening on srv1p, link-type EN10MB (Ethernet), capture size 262144 bytes
^C06:09:41.088601 IP (tos 0x0, ttl 64, id 14169, offset 0, flags [DF], proto UDP (17), length 35)
192.168.1.1.55080 > 192.168.1.2.http: [bad udp cksum 0x8374 -> 0x5710!] UDP, length 7
0x0000: 00aa bbcc dd02 00aa bbcc dd01 0800 4500 ..............E.
0x0010: 0023 3759 4000 4011 801d c0a8 0101 c0a8 .#7Y@.@.........
0x0020: 0102 d728 0050 000f 8374 6865 6c6c 6f21 ...(.P...thello!
0x0030: 0a .
1 packet captured
1 packet received by filter
0 packets dropped by kernel
server:
sudo ip netns exec srv2 tcpdump -XXvv -i srv2p
tcpdump: listening on srv2p, link-type EN10MB (Ethernet), capture size 262144 bytes
^C06:09:41.089232 IP (tos 0x0, ttl 64, id 14169, offset 0, flags [DF], proto UDP (17), length 35)
192.168.1.1.55080 > 192.168.1.2.http: [bad udp cksum 0x8374 -> 0x5710!] UDP, length 7
0x0000: 00aa bbcc dd02 00aa bbcc dd01 0800 4500 ..............E.
0x0010: 0023 3759 4000 4011 801d c0a8 0101 c0a8 .#7Y@.@.........
0x0020: 0102 d728 0050 000f 8374 6865 6c6c 6f21 ...(.P...thello!
0x0030: 0a .
1 packet captured
1 packet received by filter
0 packets dropped by kernel
I added on srv2 iptable rules to ACCEPT udp packets on port 80 and to LOG:
sudo ip netns exec srv2 iptables -t filter -A INPUT -p udp --dport 80 -j ACCEPT
sudo ip netns exec srv2 iptables -I INPUT -p udp --dport 80 -j LOG --log-prefix " IPTABLES " --log-level=debug
I could see the stats increasing on the entry and the packets being logged on /var/log/kern.log, but the message never reaches the netcat's listening socket.
sudo ip netns exec srv2 iptables -L -n -v
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
1 33 LOG udp -- * * 0.0.0.0/0 0.0.0.0/0 udp dpt:80 LOG flags 0 level 7 prefix " IPTABLES "
4 133 ACCEPT udp -- * * 0.0.0.0/0 0.0.0.0/0 udp dpt:80
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
kernel logs:
kernel: [581970.306032] IPTABLES IN=srv2p OUT= MAC=00:aa:bb:cc:dd:02:00:aa:bb:cc:dd:01:08:00 SRC=192.168.1.1 DST=192.168.1.2 LEN=33 TOS=0x00 PREC=0x00 TTL=64 ID=51034 DF PROTO=UDP SPT=48784 DPT=80 LEN=13
When I replace the softswitch with a bridge, the netcat works. I thought maybe the softswitch corrupts the packets but the tcpdump shows the right format. The UDP checksum is not correct but it is generated like that from the source server, and it is the same thing when using the linux bridge anyways but it works in that case. Is there a way to know the reason those packets do not reach the netcat server?
EDIT: Here are more informations requested in the comments:
$ sudo ip netns exec srv1 ip -br link
lo DOWN 00:00:00:00:00:00 <LOOPBACK>
srv1p@if2948 UP 00:aa:bb:cc:dd:01 <BROADCAST,MULTICAST,UP,LOWER_UP>
$ sudo ip netns exec srv2 ip -br link
lo DOWN 00:00:00:00:00:00 <LOOPBACK>
srv2p@if2944 UP 00:aa:bb:cc:dd:02 <BROADCAST,MULTICAST,UP,LOWER_UP>
$ sudo ip netns exec srv1 ip route
192.168.1.0/24 dev srv1p proto kernel scope link src 192.168.1.1
$ sudo ip netns exec srv2 ip route
192.168.1.0/24 dev srv2p proto kernel scope link src 192.168.1.2
$ sudo ip netns exec srv2 ip neighbour
192.168.1.7 dev srv2p lladdr 00:aa:bb:cc:dd:0d STALE
192.168.1.4 dev srv2p lladdr 00:aa:bb:cc:dd:06 STALE
192.168.1.8 dev srv2p lladdr 00:aa:bb:cc:dd:0e STALE
192.168.1.6 dev srv2p lladdr 00:aa:bb:cc:dd:0a STALE
192.168.1.1 dev srv2p lladdr 00:aa:bb:cc:dd:01 STALE
192.168.1.3 dev srv2p lladdr 00:aa:bb:cc:dd:05 STALE
192.168.1.5 dev srv2p lladdr 00:aa:bb:cc:dd:09 STALE
$ sudo ip netns exec srv1 ip neighbour
192.168.1.8 dev srv1p lladdr 00:aa:bb:cc:dd:0e STALE
192.168.1.4 dev srv1p lladdr 00:aa:bb:cc:dd:06 STALE
192.168.1.2 dev srv1p lladdr 00:aa:bb:cc:dd:02 STALE
192.168.1.7 dev srv1p lladdr 00:aa:bb:cc:dd:0d STALE
192.168.1.5 dev srv1p lladdr 00:aa:bb:cc:dd:09 STALE
192.168.1.3 dev srv1p lladdr 00:aa:bb:cc:dd:05 STALE
192.168.1.6 dev srv1p lladdr 00:aa:bb:cc:dd:0a STALE
$ /sbin/sysctl -ar '\.rp_filter'
net.ipv4.conf.all.rp_filter = 2
net.ipv4.conf.default.rp_filter = 2
net.ipv4.conf.docker0.rp_filter = 0
net.ipv4.conf.eth0.rp_filter = 0
net.ipv4.conf.lo.rp_filter = 0
net.ipv4.conf.veth0b5517e.rp_filter = 2
net.ipv4.conf.veth5c69974.rp_filter = 2
net.ipv4.conf.veth6c63fa9.rp_filter = 2
net.ipv4.conf.veth87b4442.rp_filter = 2
net.ipv4.conf.vethaecf041.rp_filter = 2
net.ipv4.conf.vethc9014e5.rp_filter = 2
net.ipv4.conf.vethf280de4.rp_filter = 2
net.ipv4.conf.vethf442944.rp_filter = 2
xtermin each namespace, andsuit to my normal user; that makes debugging so much easier. Also try a non-system port instead of 80, e.g. 2345, just in case something on port 80 gets in the way. – dirkt Jun 28 '20 at 16:05