Xen: Migrating LVM-based machines from Xen Opensource to Xen Cloud Platform

Question

we have several LVM-based VMs running on a Xen Opensource host. "LVM-based", in our case, means that there's a LVM volume for the root partition and another volume for the swap partition. The kernel and the initrd for the VM are outside of these volumes, as files in the Xen host. These VMs were created with xen-create-image, part of the xen-tools suite for Debian.

We want to migrate these VMs to a XCP host, and we've found a script to do it (http://www.xen.org/files/xva/xva.py). But this script assumes that the disks listed in the configuration file are "virtual disks", ie., with their own partition table and stuff. This is not the case, obviously, and it doesn't work.

What would you recommend to do this? Any homebrewn method? In the end, we could have to migrate the VMs by overwriting new ones with a big tar file from the old ones, but I'd like it to be our last resort.

I thought that maybe we could create a disk like the one this script expects by using something that does the reverse of kpartx: instead of splitting a virtual disk into devices for each partition, bundle several partitions (or in our case, LVM volumes) in a virtual disk. But I don't know anything like that.

Thanks in advance!

Answer 1

Ok, solved. This is how we did it:

Let's say that our VM is named "anansi" (I'm currently reading a book by Neil Gaiman; I'll let you guess which :-)). It's installed in two LVM volumes, /dev/vg0/anansi-root (root filesystem, 5GB) and /dev/vg0/anansi-swap (512MB).

First we created a new LVM volume for the virtual hard disk:

lvcreate -n anansiVHD -L 6GB vg0

Then we created two partitions on it, using cfdisk:

cfdisk /dev/vg0/anansiVHD

Then we used kpartx to mount the first partition of this disk:

kpartx -av /dev/vg0/anansiVHD
mkdir /tmp/anansiVHD_root
mount /dev/vg0/anansiVHD1 /tmp/anansiVHD_root

As the VM we wanted to migrate is running, we use a snapshot to copy its contents:

lvcreate -s -n anansi-snap -L 2G /dev/vg0/anansi-root
mkdir /tmp/anansi-snap
mount /dev/vg0/anansi-snap /tmp/anansi-snap
cd /tmp/anansi-snap
tar c . | (cd /tmp/anansiVHD_root ; tar xv)

We clean up:

umount /tmp/anansiVHD_root
umount /tmp/anansi-snap
lvremove -f /dev/vg0/anansi-snap
kpartx -d /dev/vg0/anansiVHD

We had to change the VM's config file ("anansi.cfg") to reflect the change to a virtual disk. It meant changing the line:

disk    = [ 'phy:/dev/vg0/anansi-root,xvda1,w', 'phy:/dev/vg0/anansi-swap,xvda2,w' ]

To this:

disk    = [ 'phy:/dev/vg0/anansiVHD,xvda,w' ]

And then we used the script previously linked to migrate the machine:

xva.py -c anansi.cfg -s <the IP of our XCP host> --username=root --password=<root password>

Afterwards we had to copy the kernel and initrd used to boot this VM to the XCP host, following the instructions of the script. The last step is to ssh into the XCP host and change the bootloader for this VM: if not, XCP will try to use pygrub, and as this expects the kernel and initrd inside the virtual hard disk, it won't work.

So we did this:

xe vm-param-set PV-bootloader="" uuid=0ab93325-bdb3-e5c9-e7ab-2173b612b02c

With the uuid for the new VM we got from "xe vm-list".

Hope it helps.

Answer 2

vi /tmp/tmpmigrateVHD_root/boot/grub/grub.conf :

• root=/dev/sda1 console=tty0 --> root=/dev/xvda1 console=xvc0

vi /tmp/tmpmigrateVHD_root/etc/fstab :

• /dev/sda1 --> /dev/xvda1
• /dev/sda2 --> /dev/xvda2

vi /tmp/tmpmigrateVHD_root/etc/inittab : (reverse comment out)

• co:2345:respawn:/sbin/agetty xvc0 9600 vt100-nav
• #1:2345:respawn:/sbin/mingetty tty1
• #2:2345:respawn:/sbin/mingetty tty2
• #3:2345:respawn:/sbin/mingetty tty3
• #4:2345:respawn:/sbin/mingetty tty4
• #5:2345:respawn:/sbin/mingetty tty5
• #6:2345:respawn:/sbin/mingetty tty6