10

I'm using Arch Linux, both host and guest. For UEFI boot, I've installed edk2-ovmf and made the guest use the firmware /usr/share/edk2-ovmf/x64/OVMF_CODE.fd. I want to create snapshots of my VM but am getting the error:

Error creating snapshot: Operation not supported: internal snapshots of a VM with pflash based firmware are not supported

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 65, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/details/snapshots.py", line 237, in _do_create_snapshot
    self.vm.create_snapshot(xml)
  File "/usr/share/virt-manager/virtManager/object/domain.py", line 1124, in create_snapshot
    self._backend.snapshotCreateXML(xml, flags)
  File "/usr/lib/python3.9/site-packages/libvirt.py", line 3059, in snapshotCreateXML
    raise libvirtError('virDomainSnapshotCreateXML() failed')
libvirt.libvirtError: Operation not supported: internal snapshots of a VM with pflash based firmware are not supported

It seems my choice of firmware, which was required to use UEFI boot which was required to install Arch, doesn't support snapshots?

Is there a way I can create snapshots with my current setup?

1 Answer 1

11

There is no fix for internal snapshots even in year 2022. The only solution would be to use external snapshots. For more details visit KVM: creating and reverting libvirt external snapshots. All the credits for the external snapshots examples go to Fabien Lee.

External snapshot for simple guest domains

Creating external snapshot

If your guest domain is a simple VM where all the <disk> are of type="file", then creating an external snapshot of a powered-on VM’s state (excluding the RAM) looks like below. For this example, assume there is a hard drive at hda and CD-ROM at hdb.

# name of domain, snapshot, and target disk device
thedomain="mysimpledomain"
snapshotname="simple-test"
targetdisk="hda"

# look at '<disk>' types, should be just 'file' types
virsh dumpxml $thedomain | grep '<disk' -A5

# show block level devices and qcow2 paths (hda,hdb,..etc)
virsh domblklist $thedomain

# create snapshot in default pool location
# file name is $thedomain.$snapshotname
virsh snapshot-create-as $thedomain --name $snapshotname --disk-only

# list snapshot
virsh snapshot-list $thedomain

Reverting external snapshot

To see the mechanism underneath and prepare variables for the revert of the snapshot, execute the commands below.

# notice path to hda has now changed to snapshot file
virsh domblklist $thedomain

# <source> has changed to snapshot file
virsh dumpxml $thedomain | grep '<disk' -A5

# pull default pool path from xml 
pooldir=$(virsh pool-dumpxml default | grep -Po "(?<=path\>)[^<]+")
echo "default pool dir: $pooldir"

# should see two files starting with $thedomain
# the one named $thedomain.$snapshotname is the snapshot
cd $pooldir
ls -latr $thedomain*

# snapshot points to backing file, which is original disk
sudo qemu-img info $thedomain.$snapshotname -U --backing-chain

# capture original backing file name so we can revert
backingfile=$(qemu-img info $thedomain.$snapshotname -U | grep -Po 'backing file:\s\K(.*)')
echo "backing file: $backingfile"

To do the revert we need to modify the domain xml back to the original qcow2 file, delete the snapshot metadata, and finally the snapshot file.

# stop VM
virsh destroy $thedomain

# edit hda path back to original qcow2 disk
virt-xml $thedomain --edit target=$targetdisk --disk path=$backingfile --update

# validate that we are now pointing back at original qcow2 disk
virsh domblklist $thedomain

# delete snapshot metadata
virsh snapshot-delete --metadata $thedomain $snapshotname

# delete snapshot qcow2 file
sudo rm $pooldir/$thedomain.$snapshotname

# start guest domain
virsh start $thedomain

The guest domain should now be in the original state. For more details about external snapshots, visit the link above.

Why the internal snapshots have been disabled?

Why the internal snapshots have been disabled you can read below. One of thee best answer I found when searching this topis is this:

On Mon, Sep 04, 2017 at 08:30:23AM -0700, ovirt@xxxxxxxxxxxxxxxxx wrote:

o/s: Fedora 26 + Virt-Manager v1.42 vm: Win 10 UEFI + Q35

Error creating snapshot: Operation not supported: Internal snapshots of a VM with pflash based firmware are not supported.

Is there a fix?

Hi, unfortunately no, and it will probably take some time to support snapshots for guests with UEFI.

The issue is that internal snapshots are obsolete and misses a lot of features that external snapshots have. They are basically usable only for guests with only one disk, where everything including the guest state is stored in one file.

In order to get this fixed and to get snapshots for guests with UEFI virt-manager will have to switch to use external snapshots, there is already a BUG for it 3. However, it cannot be currently done because libvirt still lacks some mandatory features for external snapshots, for example you cannot revert to external snapshot which makes them unusable. There is also a BUG for libvirt to implement this 4.

Pavel

3 https://bugzilla.redhat.com/show_bug.cgi?id=1403951 4 https://bugzilla.redhat.com/show_bug.cgi?id=1402581

The reason for this is here:

Re: [libvirt] [PATCH v2] qemu: snapshot: Forbid internal snapshots with pflash firmware

From: Laszlo Ersek <lersek redhat com>
To: Peter Krempa <pkrempa redhat com>
Cc: libvir-list redhat com
Subject: Re: [libvirt] [PATCH v2] qemu: snapshot: Forbid internal snapshots with pflash firmware
Date: Thu, 23 Mar 2017 17:49:56 +0100

On 03/23/17 15:07, Peter Krempa wrote:

On Thu, Mar 23, 2017 at 11:03:02 +0100, Laszlo Ersek wrote:

On 03/23/17 10:54, Peter Krempa wrote:

On Thu, Mar 23, 2017 at 10:48:01 +0100, Laszlo Ersek wrote:

On 03/23/17 10:29, Peter Krempa wrote:

If the variable store () file is raw qemu can't do a snapshot of it and thus the snapshot would be incomplete. QEMU does no reject such snapshot.

Additionally allowing to use a qcow2 variable store backing file would solve this issue but then it would become eligible to become target of the memory dump.

Offline internal snapshot would be incomplete too with either storage format since libvirt does not handle the pflash file in this case.

Forbid such snapshot so that we can avoid problems.

[...]

@@ -13873,8 +13873,14 @@ qemuDomainSnapshotPrepare(virConnectPtr conn, goto cleanup; }

  • /* Internal snapshots don't work with VMs with OVMF loader since qemu does
  • * not snapshot the variable store */
    
  • /* internal snapshots + pflash based loader have the following problems:
  • * - if the variable store is raw, the snapshot is incomplete
    
  • * - alowing a qcow2 image as the varstore would make it eligible to receive
    
  • *   the vmstate dump, which would make it huge
    
  • * - offline snapshot would not snapshot the varstore at all
    
  • *
    
  • * Avoid the issues by forbidding this completely.
    
  • */
    

I thought about this a bit more and I think that while there are the above problems we still can have users of snapshots + OVMF which use it successfully. Forbiding it would create a regression for them since they did not observe anything bad despite the problems mentioned above:

The reasons are following:

  1. internal snapshots are the default in virt-manager
  2. guests usually don't re-write the varstore very often, usually only at install
  3. OSes usually don't modify anything besides the boot entry
  4. snapshot of an online VM carries the varstore in the memory image
  5. OSes are pretty good at restoring the boot entry if it fails

Due to the facts above I think that there are users that legitimately think that snapshots with pflash loaders work as expected. It's mostly due to the fact that the data are pretty static and OSes don't store anything important there and are able to self-heal some of the problems.

I think we should not disallow this to avoid usability regressions. We can add documentation that states that it's unsafe to do snapshots. Additionally we will need to add support for external snapshots, which currently have similar kind of problems, although fixable.

The tradeoff is between a seemingly working, but inherently unsafe operation, and a clear error message that keeps things safe.

The UEFI variable store is used for more and different things than you mention, such as (in roughly decreasing order of importance):

  • Some UEFI variables (the authenticated ones) have security impact. This covers the standardized ones used for Secure Boot (Platform Key, Key Exchange Keys, white-listed certificates and signatures (DB) and black-listed certificates and signatures (DBX)).

  • To my knowledge, it also includes some similar security-related variables used by shim / MokManager (where "MOK" is short for Machine Owner Key); that is, non-volatile variables to which shim delegates the EFI binaries' verification, from the standardized Secure Boot interfaces.

  • UEFI variables can serve as the backend for the linux "pstore" (persistent store) file system. Pstore in turn can be used to save the last part of dmesg on a crash. The messages can be re-read at a new boot.

  • Firmware uses (reads/writes) a number of variables internally at each boot. These may not be critical. One example is a variable that helps reduce UEFI memory map fragmentation over a series of boots.

  • OVMF manipulates UEFI boot options on each boot, according to the bootindex properties (or more directly, according to the "bootorder" fw_cfg file). Although, admittedly, this is likely the least risky category of varstore contents.

While I have myself successfully used -- offline only -- internal snapshots with OVMF guests, hand-waving away the knowledge that the varstore was never actually snapshotted, I feel real uncomfortable about silently performing an inherently lossy operation, especially when the varstore may well have security impact.

Users will not read the documentation (they never do), and I would rather not field future bug reports about obscure Secure Boot misbehavior.

It is ultimately up to libvirt developers, but IMO, if we continue to allow this unsafe operation, then the minimum would be:

  • in virt-manager, pop up an extra confirmation dialog, with clear indication that the operation will be lossy and could have security impact,

  • in virsh, reject the operation with a similar error message, unless "--force" or something similar were specified.

  • And, because there are other (independent) libvirt client applications, this would likely require a new flag on the libvirt API level, so that libvirt itself can reject unsafe snapshotting requests, regardless of the client application that submits it.

I agree that usability regressions are not nice and will likely generate bug reports; however, if they are in direct conflict with security improvements, then security improvements trump usability regressions. I guess we can allow users to ignore security, but they need to be informed on the spot, and they have to opt in.

I would prefer if we went ahead with this patch; but, again, it's up to you in the end.

Thanks! Laszlo

0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .