2

I have a CentOS 8 guest running on a Fedora 31 host. The guest is attached to a bridge network, virbr0, and has address 192.168.122.217. I can log into the guest via ssh at that address.

If I start a service on the guest listening on port 80, all connections from the host to the guest fail like this:

$ curl 192.168.122.217
curl: (7) Failed to connect to 192.168.122.217 port 80: No route to host

The service is bound to 0.0.0.0:

guest# ss -tln
State    Recv-Q    Send-Q        Local Address:Port        Peer Address:Port

LISTEN   0         128                 0.0.0.0:22               0.0.0.0:*
LISTEN   0         5                   0.0.0.0:80               0.0.0.0:*
LISTEN   0         128                    [::]:22                  [::]:*

Using tcpdump (either on virbr0 on the host, or on eth0 on the guest), I see that the guest appears to be replying with an ICMP "admin prohibited" message.

19:09:25.698175 IP 192.168.122.1.33472 > 192.168.122.217.http: Flags [S], seq 959177236, win 64240, options [mss 1460,sackOK,TS val 3103862500 ecr 0,nop,wscale 7], length 0
19:09:25.698586 IP 192.168.122.217 > 192.168.122.1: ICMP host 192.168.122.217 unreachable - admin prohibited filter, length 68

There are no firewall rules on the INPUT chain in the guest:

guest# iptables -S INPUT
-P INPUT ACCEPT

The routing table in the guest looks perfectly normal:

guest# ip route
default via 192.168.122.1 dev eth0 proto dhcp metric 100
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.122.0/24 dev eth0 proto kernel scope link src 192.168.122.217 metric 100

SELinux is in permissive mode:

guest# getenforce
Permissive

If I stop sshd and start my service on port 22, it all works as expected.

What is causing these connections to fail?


In case someone asks for it, the complete output of iptables-save on the guest is:

*filter
:INPUT ACCEPT [327:69520]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [285:37235]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
COMMIT
*security
:INPUT ACCEPT [280:55468]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [285:37235]
COMMIT
*raw
:PREROUTING ACCEPT [348:73125]
:OUTPUT ACCEPT [285:37235]
COMMIT
*mangle
:PREROUTING ACCEPT [348:73125]
:INPUT ACCEPT [327:69520]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [285:37235]
:POSTROUTING ACCEPT [285:37235]
COMMIT
*nat
:PREROUTING ACCEPT [78:18257]
:INPUT ACCEPT [10:600]
:POSTROUTING ACCEPT [111:8182]
:OUTPUT ACCEPT [111:8182]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A DOCKER -i docker0 -j RETURN
COMMIT
0

Well, I figured it out. And it's a doozy.

CentOS 8 uses nftables, which by itself isn't surprising. It ships with the nft version of the iptables commands, which means when you use the iptables command it actually maintains a set of compatibility tables in nftables.

However...

Firewalld -- which is installed by default -- has native support for nftables, so it doesn't make use of the iptables compatibility layer.

So while iptables -S INPUT shows you:

# iptables -S INPUT
-P INPUT ACCEPT

What you actually have is:

        chain filter_INPUT {
                type filter hook input priority 10; policy accept;
                ct state established,related accept
                iifname "lo" accept
                jump filter_INPUT_ZONES_SOURCE
                jump filter_INPUT_ZONES
                ct state invalid drop
                reject with icmpx type admin-prohibited  <-- HEY LOOK AT THAT!
        }

The solution here (and honestly probably good advice in general) is:

systemctl disable --now firewalld

With firewalld out of the way, the iptables rules visible with iptables -S will behave as expected.

  • That seems like more of a workaround or hack then a solution – FreeSoftwareServers Nov 19 at 12:44
  • Note that depending on the environment I disable things too but this sounds like a WAN facing VM host? In which case I'd think security is important – FreeSoftwareServers Nov 19 at 12:45
  • It's not a WAN facing VM, and I'm not sure what gave you that impression. But in any case, you'll note that I didn't suggest disabling your firewall. Just firewalld itself. In fact, I explicitly point out that you can now use the legacy iptables command to manage your firewall. – larsks Nov 19 at 12:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.