2

I installed libvirt out of the box on Ubuntu 16.04, it creates a virbr0 virtual bridge, at 192.168.122.0/24, a dnsmasq instance to serve that range and sets up basic NAT masquerading.

I created a tap interface beneath that for my virtualization software to use. The software itself is able to access the internet and remainder of the network perfectly, so masquerading is working properly. I can also access the virtualization software ssh port from the host, using the IP address claimed by that software. So I think the routing tables "out of the box" are generally correct.

What I want to do is port forward an inbound ssh connection from outside to this software, and the host system in general. Now the host system of course has ssh, and I need that to work. So I'm trying to use another port, such as port 2022.

I've created a DNAT rule in iptables to direct inbound traffic on port 2022 to the IP address of the software at port 22. I get destination unreachable when I attempt to ssh in from the outside (ex. computer A below). It looks like the address translation is happening properly, but immediately after traversing to the "FORWARD" step in iptables, I start getting outbound unreachable ICMP packets.

I'm stumped on how to proceed. I've tried some SNAT rules, but I either botched it or that's not the problem. Does anyone have ideas on this?

UPDATE: Wireshark on tap0 does not show having received the forwarded packet. Whatever is returning the packet is not related to the virtualization software.

Here's a picture of my network:

           ----------
          | Internet |
           ----------
               |
               |
            --------
           | Router | 10.211.255.1
            --------
               |
               | 
   ____________|_______________
  |                            |
 ----------                   ---------
|          |                 |  enp0s5 | 10.211.255.4
| Computer | 10.211.255.2    |  Host   | 192.168.122.1
|    A     |                 |  virbr0 |
 ----------                   ---------
                                  |
                                  | LAN NAT 192.168.122.0/24 
                                  |
                              ----------
                             |   tap0   |
                             | Guest OS | 192.168.122.118
                             |          |
                              ----------

Here are my IP tables rules:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination
LOG        all  --  anywhere             anywhere             LOG level debug prefix "INTRACE: "
ACCEPT     udp  --  anywhere             anywhere             udp dpt:domain
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:domain
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootps
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:bootps

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
LOG        all  --  anywhere             anywhere             LOG level debug prefix "FORTRACE: "
ACCEPT     all  --  anywhere             192.168.122.0/24     ctstate RELATED,ESTABLISHED
ACCEPT     all  --  192.168.122.0/24     anywhere
ACCEPT     all  --  anywhere             anywhere
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
LOG        all  --  anywhere             anywhere             LOG level debug prefix "OUTTRACE: "
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootpc

NAT tables:
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
LOG        tcp  --  anywhere             anywhere             tcp dpt:2022 LOG level debug prefix "NATPTRACE: "
DNAT       tcp  --  anywhere             anywhere             tcp dpt:2022 to:192.168.122.118:22

Chain INPUT (policy ACCEPT)
target     prot opt source               destination
LOG        all  --  anywhere             anywhere             LOG level debug prefix "NATITRACE: "

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
LOG        all  --  anywhere             anywhere             LOG level debug prefix "NATOTRACE: "

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
LOG        all  --  anywhere             anywhere             LOG level debug prefix "NATQTRACE: "
SNAT       tcp  --  anywhere             anywhere             tcp spt:ssh to::2022
RETURN     all  --  192.168.122.0/24     base-address.mcast.net/24
RETURN     all  --  192.168.122.0/24     broadcasthost.localdomain
MASQUERADE  tcp  --  192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535
MASQUERADE  udp  --  192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535
MASQUERADE  all  --  192.168.122.0/24    !192.168.122.0/24

Finally this is an example of what I see in syslog with the above rules:

Oct 20 09:49:15 ubuntu kernel: [85149.812291] RAWPTRACE: IN=enp0s5 OUT= MAC=00:1c:42:3a:00:df:00:1c:42:00:00:08:08:00 SRC=10.211.55.2 DST=10.211.55.4 LEN=64 TOS=0x00 PREC=0x00 TTL=64 ID=40132 DF PROTO=TCP SPT=53245 DPT=2022 WINDOW=65535 RES=0x00 SYN URGP=0
Oct 20 09:49:15 ubuntu kernel: [85149.812318] MGPTRACE: IN=enp0s5 OUT= MAC=00:1c:42:3a:00:df:00:1c:42:00:00:08:08:00 SRC=10.211.55.2 DST=10.211.55.4 LEN=64 TOS=0x00 PREC=0x00 TTL=64 ID=40132 DF PROTO=TCP SPT=53245 DPT=2022 WINDOW=65535 RES=0x00 SYN URGP=0
Oct 20 09:49:15 ubuntu kernel: [85149.812326] NATPTRACE: IN=enp0s5 OUT= MAC=00:1c:42:3a:00:df:00:1c:42:00:00:08:08:00 SRC=10.211.55.2 DST=10.211.55.4 LEN=64 TOS=0x00 PREC=0x00 TTL=64 ID=40132 DF PROTO=TCP SPT=53245 DPT=2022 WINDOW=65535 RES=0x00 SYN URGP=0
Oct 20 09:49:15 ubuntu kernel: [85149.812336] MGFTRACE: IN=enp0s5 OUT=virbr0 MAC=00:1c:42:3a:00:df:00:1c:42:00:00:08:08:00 SRC=10.211.55.2 DST=192.168.122.118 LEN=64 TOS=0x00 PREC=0x00 TTL=63 ID=40132 DF PROTO=TCP SPT=53245 DPT=22 WINDOW=65535 RES=0x00 SYN URGP=0
Oct 20 09:49:15 ubuntu kernel: [85149.812340] FORTRACE: IN=enp0s5 OUT=virbr0 MAC=00:1c:42:3a:00:df:00:1c:42:00:00:08:08:00 SRC=10.211.55.2 DST=192.168.122.118 LEN=64 TOS=0x00 PREC=0x00 TTL=63 ID=40132 DF PROTO=TCP SPT=53245 DPT=22 WINDOW=65535 RES=0x00 SYN URGP=0
Oct 20 09:49:15 ubuntu kernel: [85149.812354] RAWOTRACE: IN= OUT=enp0s5 SRC=10.211.55.4 DST=10.211.55.2 LEN=92 TOS=0x00 PREC=0xC0 TTL=64 ID=50608 PROTO=ICMP TYPE=3 CODE=3 [SRC=10.211.55.2 DST=192.168.122.118 LEN=64 TOS=0x00 PREC=0x00 TTL=63 ID=40132 DF PROTO=TCP SPT=53245 DPT=22 WINDOW=65535 RES=0x00 SYN URGP=0 ]
Oct 20 09:49:15 ubuntu kernel: [85149.812357] MGOTRACE: IN= OUT=enp0s5 SRC=10.211.55.4 DST=10.211.55.2 LEN=92 TOS=0x00 PREC=0xC0 TTL=64 ID=50608 PROTO=ICMP TYPE=3 CODE=3 [SRC=10.211.55.2 DST=192.168.122.118 LEN=64 TOS=0x00 PREC=0x00 TTL=63 ID=40132 DF PROTO=TCP SPT=53245 DPT=22 WINDOW=65535 RES=0x00 SYN URGP=0 ]
Oct 20 09:49:15 ubuntu kernel: [85149.812361] OUTTRACE: IN= OUT=enp0s5 SRC=10.211.55.4 DST=10.211.55.2 LEN=92 TOS=0x00 PREC=0xC0 TTL=64 ID=50608 PROTO=ICMP TYPE=3 CODE=3 [SRC=10.211.55.2 DST=192.168.122.118 LEN=64 TOS=0x00 PREC=0x00 TTL=63 ID=40132 DF PROTO=TCP SPT=53245 DPT=22 WINDOW=65535 RES=0x00 SYN URGP=0 ]
Oct 20 09:49:15 ubuntu kernel: [85149.812364] MGQTRACE: IN= OUT=enp0s5 SRC=10.211.55.4 DST=10.211.55.2 LEN=92 TOS=0x00 PREC=0xC0 TTL=64 ID=50608 PROTO=ICMP TYPE=3 CODE=3 [SRC=10.211.55.2 DST=192.168.122.118 LEN=64 TOS=0x00 PREC=0x00 TTL=63 ID=40132 DF PROTO=TCP SPT=53245 DPT=22 WINDOW=65535 RES=0x00 SYN URGP=0 ]
1

The problem was in the FORWARD rules. As I did not set them up, I was misinterpreting output from iptables --list.

In that list rule 4 reads:

ACCEPT     all  --  anywhere             anywhere

This appears to accept all incoming packets from anywhere bound to anywhere.

However if I look at iptables -S rule 4 is actually:

-A FORWARD -i virbr0 -o virbr0 -j ACCEPT

Which is not going to be matched by a packet coming in from enp0s5 destined for vibr0. So it was hitting the REJECT rule 5, which was actually a reject for all packets destined for vibr0.

The lesson I have learned is not to trust the output of iptables --list but rather to check iptables -S output when something seems wrong.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.