Setup VLAN on linux bridge for virtual machines with systemd

Question

I want to switch to systemd and with this I have to setup the linux bridge for my virtual machines in KVM to use VLAN.

Having Debian GNU/Linux 9.1 (stretch), using its systemd-networkd and libvirt. NOT using oldstyle networking with ifupdown (deinstalled it).

I'm sitting on harley, my all day workstation for testing and setup the bridge on the host.

harley$ cat /etc/systemd/network/08-br0.netdev
[NetDev]
Name=br0
Kind=bridge
[Bridge]
VLANFiltering=true
STP=false  
harley$ cat /etc/systemd/network/12-br0_add-enp1s0.network
[Match]
Name=enp1s0
[Network]
Bridge=br0
[BridgeVLAN]
VLAN=10
[BridgeVLAN]
VLAN=20
[BridgeVLAN]
VLAN=30  
harley$ cat /etc/systemd/network/16-br0_up.network
[Match]
Name=br0

Cleanup an old bridge from testing and setup a new one from scratch:

harley$ sudo ip link set dev br0 down && sudo ip link del dev br0
harley$ sudo systemctl restart systemd-networkd
harley$

It's all in place now:

harley$ cat /sys/class/net/br0/bridge/vlan_filtering
1
harley$ cat /sys/class/net/br0/bridge/stp_state
0
harley$ sudo bridge vlan show
port    vlan ids
enp1s0   1 PVID Egress Untagged
         10
         20
         30  
br0      1 PVID Egress Untagged
harley$

Now I start the virtual machine, login and show its interface setting:

harley$ virsh start --console deb9-test

login

deb9-test$ cat /etc/systemd/network/08-vlan10.netdev
[NetDev]
Name=vlan10
Kind=vlan
[VLAN]
Id=10  
deb9-test$ cat /etc/systemd/network/12-vlan10_attach-to-if.network
[Match]
Name=ens2
[Network]
VLAN=vlan10  
deb9-test$ cat /etc/systemd/network/16-vlan10_up.network
[Match]
Name=vlan10
[Network]
Address=192.168.10.57/24
Gateway=192.168.10.1
DNS=192.168.10.8

But pinging the gateway gives no result at this stage:

deb9-test$ ping -c1 192.168.10.1
PING 192.168.10.1 (192.168.10.1) 56(84) bytes of data.
From 192.168.10.57 icmp_seq=1 Destination Host Unreachable  
--- 192.168.10.1 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

The bridge shows now:

harley$ sudo bridge vlan show
port    vlan ids
enp1s0   1 PVID Egress Untagged
         10
         20
         30  
br0      1 PVID Egress Untagged  
vnet0    1 PVID Egress Untagged

I have to modify it:

harley$ sudo bridge vlan del dev enp1s0 vid 1
harley$ sudo bridge vlan del dev br0 vid 1 self
harley$ sudo bridge vlan del dev vnet0 vid 1
harley$ sudo bridge vlan add vid 10 dev vnet0

VM works now:

deb9-test$ ping -c1 192.168.10.1
PING 192.168.10.1 (192.168.10.1) 56(84) bytes of data.
64 bytes from 192.168.10.1: icmp_seq=1 ttl=64 time=0.843 ms  
--- 192.168.10.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.843/0.843/0.843/0.000 ms

After modification this is what I want, simple and clean:

harley$ sudo bridge vlan show
port    vlan ids
enp1s0   10
         20
         30  
br0     None
vnet0    10

And finaly now my Question:
how can I make Debian to do this for me right up before the VMs interface comes up, so it can get its ip-address with DHCP?

Answer 1

PLEASE NOTE:
This configuration is deprecated and no longer supported. I have decided to continue it on serverfault at VLAN support with libvirt for linux bridge to virtual machines, because I think it better fit to that site according to the policies. If you want improved scripts, then look there.

---

I've worked on it and found a solution. Here is it in short. Look there for a more detailed description.

We have to tell libvirt to set the right VLAN-ID to the dynamic created virtual network interface (e.g. vnet0) added to the bridge on startup of a domain (guest). For this we can use libvirt hook scripts. I do this in three steps.

Step 1: define VLAN-ID the virtual machine belongs to

For this we have an extra element <metadata> in Domain XML format for custom metadata. We can simply add the information to the static configuration with harley$ virsh edit deb9-test like this (look only at the <metadata> element):

harley$ virsh dumpxml deb9-test | head -n9
 <domain type='kvm' id='1'>
   <name>deb9-test</name>
  <uuid>70d56a28-795d-4010-9403-513a4bd6b66a</uuid>
  <metadata>
    <my:home xmlns:my="http://hoeft-online.de/my/">
      <my:vlan>10</my:vlan>
    </my:home>
  </metadata>
  <memory unit='KiB'>1048576</memory>

Step 2: get information on startup from the runtime XML-config of the domain

The hook script gets information on its standard input. This is the XML-config of the running VM. We can also get it with harley$ virsh dumpxml deb9-test when the VM is running. I use XSLT with xmlstarlet to get needed information with a xsl-stylesheet. I can test with harley$ virsh dumpxml deb9-test | xmlstarlet tr qemu.xsl. Here is the stylesheet:

harley$ cat /etc/libvirt/hooks/qemu.xsl
<?xml version="1.0" encoding="UTF-8"?>
<!-- This stylesheet extracts the VLAN-ID and the target device of the
     bridge from the domain-xml given to the libvirt hook-script "qemu".
     Example output: <meta><vlan>10</vlan><dev>vnet0</dev></meta>
-->
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
     xmlns:my="http://hoeft-online.de/my/" exclude-result-prefixes="my">
  <xsl:output omit-xml-declaration="yes" indent="no"
       encoding="utf-8" media-type="text/xml"/>
  <xsl:strip-space elements="*"/>
  <xsl:template match="text()|@*"/>

  <xsl:template match="/domain">
    <meta>
      <xsl:apply-templates/>
    </meta>
  </xsl:template>

  <xsl:template match="metadata/my:home/my:vlan">
    <vlan>
      <xsl:value-of select="."/>
    </vlan>
  </xsl:template>

  <xsl:template match='interface[@type="bridge"]/target'>
    <dev>
      <xsl:value-of select="@dev"/>
    </dev>
  </xsl:template>

</xsl:stylesheet>
harley$

Step 3: set VLAN-ID to the dynamic virtual network interface vnet*

With information from the stylesheet we can now setup the network interface with a hook script. Make it executable.

harley$ cat /etc/libvirt/hooks/qemu
#!/bin/bash
#/etc/libvirt/hooks/qemu
# Docs: https://www.libvirt.org/hooks.html
# If you make a new hook script then 'sudo systemctl restart libvirtd'.

# On startup of the domain (guest) This script does:
# Get Metadata VLAN-ID of the guest and target device of the bridge from
    # the domain-xml available on standard input. It is the runtime
    # version from 'virsh dumpxml domainname'. For extracting the
    # information we use a XSL-stylesheet. Example input into $META:
    # <meta><vlan>10</vlan><dev>vnet0</dev></meta>
# Select $DEV  from $META
# Select $VLAN from $META
# Set $VLAN to $DEV on the bridge

case "$2" in
  prepare)
    ;;
  start)
    META=$(/usr/bin/xmlstarlet tr /etc/libvirt/hooks/qemu.xsl -)
    DEV=$(echo "$META" | /usr/bin/xmlstarlet sel -t -v '/meta/dev')
    VLAN=$(echo "$META" | /usr/bin/xmlstarlet sel -t -v '/meta/vlan')
    if [[ -n $DEV && -n $VLAN ]]; then
      /sbin/bridge vlan add vid "$VLAN" dev "$DEV"
    fi
    ;;
  started)
    ;;
  stopped)
    ;;
  release)
    ;;
  migrate)
    ;;
  restore)
    ;;
  reconnect)
    ;;
  attach)
    ;;
  *)
    echo "qemu hook called with unexpected options $*" >&2
    exit 1
    ;;
esac
harley$