4

I'm playing around with using Qemu external snapshots for KVM (because they are so fast!) on Ubuntu 16.04 and have the following setup:

  • All files for the VM are within a single directory (named after the guest).
  • There is a head.qcow2 symlink that always points to the top "head" qcow2 image. This way I don't have to keep updating the VM config. (virsh edit).
  • The original/bottom image is called base.qcow2

I use the following script to take a snapshot (create a new qcow2 image and make it the head). The script is only executed when the guest is not running.

#!/bin/bash
UNIX_TIMESTAMP=$(date +%s)
CURRENT_BACKING_FILE=`readlink head.qcow2`
NEW_HEAD_FILE="`pwd`/`echo $UNIX_TIMESTAMP`.qcow2"

# Create the new "head" disk image where all future changes are made.
sudo qemu-img create \
  -f qcow2 \
  -b $CURRENT_BACKING_FILE \
  $NEW_HEAD_FILE

# Update the symlink
rm head.qcow2
ln -s $NEW_HEAD_FILE head.qcow2

This will work fine when head.qcow2 points to base.qcow2, or I run the script once so there is only a single backing file. However, If there is a chain of backing files (e.g. a backing file also has a backing file), then I will get the following error:

error: Failed to start domain template-ubuntu-docker2
error: internal error: early end of file from monitor, possible problem: 2018-04-05T19:04:55.597373Z qemu-system-x86_64: -drive file=/media/ssd_storage2/kvm/vms/templates/template-ubuntu-docker2/head.qcow2,format=qcow2,if=none,id=drive-virtio-disk0: Could not open backing file: Could not open backing file: Could not open '/media/ssd_storage2/kvm/vms/templates/template-ubuntu-docker2/base.qcow2': Permission denied

The output of qemu-img --backing-chain head.qcow2 is:

image: head.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 196K
cluster_size: 65536
backing file: /media/ssd_storage2/kvm/vms/templates/template-ubuntu-docker2/1522954330.qcow2
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

image: /media/ssd_storage2/kvm/vms/templates/template-ubuntu-docker2/1522954330.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 4.1M
cluster_size: 65536
backing file: base.qcow2 (actual path: /media/ssd_storage2/kvm/vms/templates/template-ubuntu-docker2/base.qcow2)
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

image: /media/ssd_storage2/kvm/vms/templates/template-ubuntu-docker2/base.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 8.8G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: true
    refcount bits: 16
    corrupt: false

I've played around with deleting images or using qemu-img commit to merge them, and as soon as I go down to just one or two images the guest will start.

Since it looked like a permission error with "could not open" I tried setting permissions to 777 and I still get the same error message, so it shouldn't be a permissions issue.

What do I need to do?

4

It looks like it's related to AppArmor again (I had a similar issue some years ago on Ubuntu 14.04 that I forgot about). I was able to get it working by:

Installing apparmor-utils

sudo apt-get install apparmor-utils

Then find the libvirt ID in the most recent syslogs:

sudo cat /var/log/syslog | grep "apparmor" | grep "DENIED" | grep libvirt-

Then use that ID with the full path to set it to "complain mode".

sudo aa-complain \
/etc/apparmor.d/libvirt/libvirt-b0cd5bdf-6dfc-4f75-bc0f-e38adff1cdd2

Then start your guest.

This feels like a workaround at best and annoying if I have to do it for each guest individually. I would be happier if someone provided a better solution.

0

Got same problem on Debian Buster, the fact is that libvirt apparmor profiles are dynamically created and stored under /etc/apparmor.d/libvirt/ (thanks @Programster). In this directory you should found a TEMPLATE.qemu similar to this one:

#
# This profile is for the domain whose UUID matches this file.
#

#include <tunables/global>

profile LIBVIRT_TEMPLATE flags=(attach_disconnected) {
  #include <abstractions/libvirt-qemu>
}

You should add rules for the directory where you store your qcow2 in the profile:

#
# This profile is for the domain whose UUID matches this file.
#

#include <tunables/global>

profile LIBVIRT_TEMPLATE flags=(attach_disconnected) {
  #include <abstractions/libvirt-qemu>
  /media/ssd_storage2/kvm/vms/templates/template-ubuntu-docker2/**.qcow2 rk,
}

"r" is for read permission and "k" is for lock permissions.

You can track apparmor audit messages with "dmesg -w", and get similar messages depending which permissions are missing:

[ 4404.140981] audit: type=1400 audit(1566342100.488:254): apparmor="DENIED" operation="open" profile="libvirt-b24e7663-d71a-4970-9d6c-711d43629135" name="/home/libvirt/0.0.6-dev.19+20~811d1dc983b5.qcow2" pid=22384 comm="qemu-system-x86" requested_mask="r" denied_mask="r" fsuid=64055 ouid=64055
...
[ 4652.827860] audit: type=1400 audit(1566342349.172:263): apparmor="DENIED" operation="file_lock" profile="libvirt-b24e7663-d71a-4970-9d6c-711d43629135" name="/home/libvirt/0.0.6-dev.19+20~811d1dc983b5.qcow2" pid=23085 comm="qemu-system-x86" requested_mask="k" denied_mask="k" fsuid=64055 ouid=64055

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.