2

I have a Ubuntu 14.04 application appliance that makes heavy use of BTRFS snapshots. The application is meant to be hypervisor agnostic and the snapshots need to be stored with the virtual machine image in case we need to troubleshoot an issue. Using the hypervisors built in snapshotting methods instead of BTRFS snapshotting would be more work than it's worth since it would require API access to the hypervisor from the VM which we don't want for security reasons. I can also remotely access the BTRFS snapshot filesystem subvolumes directly from the appliances command line via ssh without having to power down the machine or access a hypervisor API.

In the past I've only had to deal with deploying this application to vmware based hypervisors. I always used thin provisioning on my vmware disks and never noticed any performance issues. I use thin provisioning because I do a large amount of testing on this appliance and I tend to deploy many appliances at a time to run different tests in parallel.

I'm very careful about not over committing the storage by using scripts that ensure the disk growth doesn't run out of control. The I/O dip that happens when the thin provisioned disk needs to grow isn't all that noticeable either.

Now I need to support KVM and would very much like to keep Thin/sparse provisioning my disks, however I've read a few things that state mixing a CoW filesystem with another CoW filesystem is a bad idea due to over-redundant writes and disk fragmentation among other things. The common example given was running a VM with a qcow2 formatted disk stored on a BTRFS formatted volume. My situation would be the opposite. I want to have a BTRFS formatted filesystem in a VM running from a qcow2 image. I haven't found much on the particulars of the performance impact of BTRFS snapshots on top of a qcow2 image.

Question: Are there any other sparse file formats that grow with the disk size that KVM likes to use?

I've explored with using sparse raw files, but I can't seem to get them to stay sparse through a cp, download untar/gunzip, ect. It seems that if you want to use a sparse raw file, you can't move it around at all which would make distribution a pain in the ass.

2

The thing is, KVM doesn't support disk images at all. Unlike vmware which is an all-in-one type of solution, KVM only provides the virtualization capabilities, upon which a virtulization solution can be built.

...It consists of a loadable kernel module, kvm.ko, that provides the core virtualization infrastructure and a processor specific module, kvm-intel.ko or kvm-amd.ko. - http://www.linux-kvm.org/page/Main%5FPage

Typically, a KVM-based solution involves the combination of:

  • KVM - for access to the CPU's virtualization capabilities
  • QEMU - which provides the emulation bits, such as BIOS/UEFI
  • additional libraries and tools - such as libvirt and Virtual Machine Manager

QEMU

QEMU supports two native disk formats (there are other formats as well, like VDI):

Raw

Raw disk image format (default). This format has the advantage of
being simple and easily exportable to all other emulators. If your
file system supports holes (for example in ext2 or ext3 on Linux or NTFS on Windows), then only the written sectors will reserve space. - man qemu-img

Qcow/2

QEMU image format, the most versatile format. Use it to have
smaller images (useful if your filesystem does not supports holes,
for example on Windows), optional AES encryption, zlib based
compression and support of multiple VM snapshots. - man qemu-img

libvirt

libvirt provides a number of ways to store disk images:

  • Directory backend
  • Local filesystem backend
  • Network filesystem backend
  • Logical backend - LVM but without thin-provisioning :(
  • Disk backend
  • iSCSI backend
  • SCSI backend
  • Multipath backend
  • RBD (RADOS Block Device) backend
  • Sheepdog backend
  • Gluster backend
  • ZFS backend
  • Virtuozzo storage backend

Take a look at those and see which, if any, meet your requirements. See https://libvirt.org/storage.html, and better yet https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Administration_Guide/chap-Virtualization_Administration_Guide-Storage_Pools-Storage_Pools.html

1

My personal experience is that you can work with raw files on a sparse-capable host filesystem.

I have raw files on an ext4 host file-system. With the vm shut down I can use "zerofree" on the raw files (mounted as loop device) and using cp --sparse=always oldfile newfile I get reduced file sizes. I dont know if the files keep sparse during use as I only use this method to "archieve" images of test systems.

BTW, btrfs with heavy snapshotting seems to be a bad idea: https://btrfs.wiki.kernel.org/index.php/Gotchas#Having_many_subvolumes_can_be_very_slow

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.