0

What I have:

                        ________
   enp7s0f0------------>|router|   
   no IP addr           -------- 
      |                 
      |
      |
   Xen Dom0
      |
      |
      | 
    xenbr0
192.168.169.169/24
      |
      |                  _________________________
      |---vif1.0-------->|Old guest VM (ubuntu14)|  (eth0 192.168.169.69)
      |                  ------------------------
      |                  ________________________
      |---vif6.0-------->|New Guest VM (debian10)|  (eth0 192.168.169.42)
                         ------------------------

The back story: I built a Xen Dom0 many years ago and installed one VM, lets call it "old guest", running Ubuntu 14. Life is good.

It is now old, so I need a new one, but Xen Dom0 is too old to update also, so I moved "oldguest" off, installed a new Xen Dom0 using Debian 10, and moved the "oldguest" back on. Now I need to build a new VM, call it "newguest" and migrate data. I do so using:

xen-create-image --hostname=newguest --memory=4G --vcpus=2 --partition=/etc/xen-tools/partitions.d/newguest --dist=buster --dir=/home/xen --ip=192.168.169.42 --broadcast=192.168.169.255 --gateway=192.168.169.1 --nodhcp --netmask=255.255.255.0 --nameserver=9.9.9.9 

Seems to go ok, I then ssh into the new guest, create a user for myself, and do

apt-get update ; apt-get upgrade

Hrm, thats odd, it takes like, FOREVER AND A DAY for apt-get to come back.

Check networking: seems fine.

I decide to do a non-scientific test and grab a file, I think it could be the net? Ops, do not have wget:

apt-get install wget

THE ABOVE COMMAND TAKES LIKE 10 MINUTES. I ssh into the vm from another host. CPU load seems moderately high... lets try a different test: I have yet another host connected to the node called "router" in my diagram that is also running linux. I perform the wget:

65kB/s average download.

I go to "oldguest" and perform the same test:

11MB/s. OK, that is the correct speed to get.

Observations: If I reboot "newguest" and do NOT perform any action on "oldguest", I can get 11MB/s with the same wget test! Hrm, this makes me think it is networking on the Dom0. If I go to "oldguest" and perform the same test I get 11MB/s, BUT, if I go back to "newguest" and run the test again, it drops to around 50-60kB/s.

Note, I have even tried the download from "newguest" and then ssh into "oldguest" while it is running. The instant "oldguest" gets a packat, the speed on "newguest" falls to the floor.

One final note on speed tests:

If I copy/scp a file from "oldguest" to "newguest", I get 60MB/s. So this seems to be an issue with packets traversing Dom0 to "newguest", or a problem in my network configuration.

Steps taken to try to solve:

Rechecked all network configs. Check routing, all guests have the node marked "router" as the default gateway. Checked processes, it seems like I am not CPU bound anywhere.

Rechecked my config against this.

Have read this article on interfaces.

Have read this post, which is actually the opposite of my config.

My configs:

Dom0 /etc/network/interfaces:

auto enp7s0f0
iface enp7s0f0 inet manual  

auto xenbr0
iface xenbr0 inet static
    bridge_ports enp7s0f0
    address 192.168.169.169
    broadcast 192.168.169.255
    netmask 255.255.255.0
    gateway 192.168.169.1

Note, in /etc/xen/newguest we see:

vif         = [ 'ip=192.168.169.42 ,mac=00:16:3E:27:99:CD,bridge=xenbr0' ]

in /etc/xen/oldguest we see:

vif         = [ 'ip=192.168.169.69 ,mac=00:16:3E:B1:AC:98,bridge=xenbr0' ]

Newguest /etc/network/interfaces

auto eth0
iface eth0 inet static
    address 192.168.169.42
    gateway 192.168.169.1
    netmask 255.255.255.0
    broadcast 192.168.169.255

Old Guest /etc/network/interfaces

auto eth0
iface eth0 inet static
 address 192.168.169.69
 gateway 192.168.169.1
 netmask 255.255.255.0
 broadcast 192.168.169.255

Update, after reading this page, I have been doing top and network and xentop testing. It does not seem to be a Dom0 bottleneck. See this screenshot during file transfer, there does not see to be any real problem, and xentop shows 1% CPU utilization also.

top during file transfer

Any help in diagnosis would be great. I am thinking it is the hypervisor, but am not sure what to look at next.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.