1

My goal is to setup a few QEMU/libvirt based guests using a bridge interface on the host so each guest-VM has a DHCP assigned IP address on my LAN network. I have successfully setup and used less complex VM configurations and know my hardware supports virtualization, so I'll try to cut to the chase so to speak.

Topology

Each node is connected with 1Gbps ethernet (so no wireless interfaces)

[router]---[switch1]---[switch2]---[host]

Host

  • Runs Ubuntu 16.04.5
  • 1x 1Gbps NIC configured with dhcp for management activities (em1)
  • 1x 1Gbps NIC configured as a bridge with a static IP address (br0 / eth1 slave)
  • Bridge connectivity tested successfully using SSH & ping from different host in same LAN network. I am explicitly disabling spanning-tree-protocol in the interfaces file, and brctl show command correctly displays my bridge with eth1 as the only interface.

Guest setup attempt with virt-install

install command

virt-install --name={guest-name} --vcpus=2 --memory=4096 --network bridge=br0
--cdrom={.iso-img-path} --disk size=20,path={diskimg-path} --os-variant=debian8
--graphics vnc,password={pass},listen=0.0.0.0 --noautoconsole

Observed behavior and troubleshooting so far

  • Connected to Guest VM VNC for install successfully

  • Debian 9 graphical installer reports a failure to autoconfigure the guests network interface for DHCP over the host bridge interface

  • command 'sudo tcpdump -i br0 | grep -i dhcp' (output below) only shows the requests being broadcasted, and no replies.

10:08:57.833669 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 52:54:00:8a:9e:69 (oui Unknown), length 300

I have a technical background but am unfamiliar with the low level details and how to troubleshoot DHCP properly.

UPDATE 1

I configured tshark on the Ubuntu host, and hooked up a laptop running Wireshark via ethernet to one of the switch ports on the router.

Note: As far as I can see my router doesn't support any port mirroring or port monitoring.

New Topology

[router]---[switch1]---[switch2]---[host-Tshark]

|

[Deb9.5-Wireshark]

Tshark Command & Results

[]$ tshark -w out.pcap -f "udp port 68 or port 67" -i any

[]$ tshark -r out.pcap -V | grep -e Frame -e Bootstrap -e User\ Datagram\ Protocol -e Bootp\ flags -e "Internet Protocol Version 4"
Frame 295: 344 bytes on wire (2752 bits), 344 bytes captured (2752 bits) on interface 0
    Frame Number: 295
    Frame Length: 344 bytes (2752 bits)
    [Frame is marked: False]
    [Frame is ignored: False]
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)
    Bootp flags: 0x0000 (Unicast)
  • I started the above capture command, and quickly moved over to my debian 9 install VNC and retried the DHCP autoconfiguration, and then quickly stopped capturing
  • I noticed that the capture count started going up after capturing started but BEFORE I told the graphical install to re-attempt DHCP configuration, and all the packets look the same as the output above. (some packets stuck in forwarding loop perhaps?)
  • I attempted to use Wireshark on the laptop prior to using Tshark on the host, so there are a few filters I'm going to try when I get home BUT during my testing with Wireshark I did not see any DHCP/BOOTP (Discover) packets come through.
  • The Bootp flags are marked for unicast, but the IP src/dst make it seem like it is trying to broadcast the Discover packet and wait for a response? Anyone more familiar with DHCP could help shed some light on this

Update 2 -- Plus resolution

So the next thing I tried was to use the MAC address in the DHCP Discover packets, and configure a static IP on my router GUI. This succeeded as it allowed me to get past the network configuration part of the installer, but it failed when I reached the point where I needed to configure the package manager by connecting to a mirror server.

Now ARP resolution IS something I am familiar with, and I found my ARP packets being sent out the bridge were not getting a reply, similar to the DHCP discover packets. When I inspected the packets from pinging the bridge interface from another host on the LAN I found the wrong MAC address in the source field for the reply. It turns out the Linux kernel treats an IP address as a "system object" in a loosely coupled system, so having multiple interfaces connected to the same network (192.168.1.0/24) won't guarantee which physical interface gets used for handling packets pertaining to a specific IP.

To resolve this, I just added every physical interface in the system to the bridge interface definition and everything is working now after reboot. Not my ideal setup, but it works, and DHCP works as expected.

Further info: https://lwn.net/Articles/45373/

  • Have you installed a DHCP server to answer those requests? – Rui F Ribeiro Aug 6 '18 at 19:11
  • My router has a DHCP server on it, and hasn't had any problems connecting any network devices via DHCP before. Static IP assignment feature in the DHCP server works as well (there is a static IP entry for the bridge interface on the host which works fine) – user3282129 Aug 6 '18 at 19:17
  • Your separate networks should not use the same IP subnet. – Michael Hampton Aug 9 '18 at 16:43
  • I need both interfaces and all VMs to be on the same LAN network. Which isn't possible with the Linux kernel implementation because it does not create a hard mapping between IP address and ethernet interface. – user3282129 Aug 9 '18 at 16:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.