I'm setting up KVM virtualization host with policy based routing (192.168.1.0/24 management network for host related traffic like updates,ssh etc, 192.168.2.0/24 DMZ network routed to VMs)
Setup looks like this (by ^^^ i mean like above to represent tree structure)
#external connectivity eth0 -> vlan110 -> brhost (192.168.1.9) ^^^ -> vlan120 -> brguest (192.168.2.4) #internal connectivity brguestA (10.0.1.1) -> tapA0 ^^^ -> tapA1 ^^^ -> tapA2 brguestB (10.0.2.1) -> tapB0 ^^^ -> tapB1 # ... and so on for each VM group subnet
machines An are in one "switch", Bn in second one etc. Host behaves like router and has iptables installed. Source ip is validated per bridge on iptables level.
In general everything "normal" works but when I'm performing some border case operations it still behaves the way it should, but logs suggest that it's more of coincidence than actual proper functioning. For example:
connecting to 192.168.1.3 from VM (different host in management network) which is blocked operation on iptables level (tcp-reset) gives
(iptables reject log) IN=brguestA OUT=brguest SRC=10.0.1.2 DST=192.168.1.3 ... IPv4: martian source 10.0.1.2 from 192.168.1.3 on dev brhost
OUT interface is actually correct according to what it should do considering logic of host/guest networking separation and even if such packets would be forwarded by host they'd get dropped on firewall anyways. Nevertheless host iptables can't properly reject with tcp-reset so such connections just hang on VM and don't receive reset
connecting to 192.168.1.9 from VM gives
IPv4: martian source 192.168.1.9 from 10.0.1.2 on dev brguestA
and nothing else. Swapping places of 10.0.1.x and 192.168.1.x in those logs makes me really confused and I don't really understand what is meaning of those (which ones is src ip and which one is dst ip). I don't really know what host is trying to do and why it fails. Here's output of my
ip rule and
ip rule 0: from all lookup local 32763: from 10.0.0.0/16 lookup guest 32764: from 192.168.2.0/24 lookup guest 32765: from all iif lo lookup host 32766: from all lookup main 32767: from all lookup default ip route show table host default via 192.168.1.1 dev brhost proto static 192.168.1.0/24 dev brhost proto static ip route show table guest default via 192.168.2.1 dev brguest proto static 10.0.0.0/24 dev brguestA proto static 10.0.1.0/24 dev brguestB proto static # ... so on for other networks ip route show table main 192.168.1.0/24 dev brhost proto static 10.0.0.0/24 dev brguestA proto static 10.0.1.0/24 dev brguestB proto static # ...
systemd-networkd for network management. I thought it is happening because host always tries to respond using
brhost device as being
OUTPUT type of packet it has
iif set to
lo so it's caught by
from iif lookup host rule. However it doesn't seem to be a case as connecting to
192.168.2.4 is rejected properly so it doens't always pick
brhost as output source.