49

Should LVM be used for the partitions when creating VM images (e.g., KVM images)? It seems like it adds complexity if you want to, say, mount a qcow2 image in the host if the image has LVM partitions.

On the other hand, it doesn't seem like the advantages of LVM partitions are as significant on a VM image, since it's much easier to take a VM offline and resize partitions than it is for a physical system.

  • Do you ask about using LVs as whole disks or as partitions that in turn make up a disk in a VM? – Nils Mar 5 '12 at 20:57
  • @Nils I'm talking about LVs as partitions that make up a disk. For example, having the "/" partition and the swap partition as logical volumes inside of a volume group. – Lorin Hochstein Mar 5 '12 at 21:08
  • Just to clarify, it seems like you were asking about using LVM on the guest side. It's much easier to manage if you use LVM on the host side, pass one disk or two, and use them without any partitioning on the guest. – Tobu Oct 3 '13 at 8:06
  • The LVM overhead is in the range of 10e-9 second. – Emmanuel Nov 13 '17 at 19:13
23

"It depends."

If you are on an environment that you control (vmware or kvm or whatever), and can make your own decisions about disk performance QoS, then I'd recommend not using LVM inside your VMs. It doesn't buy you much flexibility that you couldn't get at the hypervisor level.

Remember, the hypervisor is already effectively performing these tasks. If you want to be able to arbitrarily resize file systems (a fine idea), just create a separate virtual disk for each filesystem.

One thing you might think of as you go down this road. You don't even necessarily need to put partitions on your virtual disks this way. For example, you can create a virtual disk for /home; it is /dev/vdc inside your vm. When creating the filesystem, just do something like mke2fs -j /dev/vdc instead of specifying a partition.

This is a fine idea, but...most tools (and other admins who come after you) will expect to see partitions on every disk. I'd recommend just putting a single partition on the disk and be done with it. It does mean one more step when resizing the filesystem, though. And don't forget to properly align your partitions - starting the first partition at 1MB is a good rule of thumb.

All that said - Doing this all at the hypervisor level means that you probably have to reboot the VM to resize partitions. Using LVM would allow you to hot-add a virtual disk (presuming your hypervisor/OS combination allows this), and expand the filesystem without a reboot. This is definitely a plus.


Meanwhile, if you are using a cloud provider, it's more subtle.

I don't know much about Azure, GCP, or any of the smaller players, so I can't help there.

With AWS you can follow my advice above and you'll often be just fine. You can (now) increase the size of EBS volumes (virtual disks) on-the-fly, and resize partitions, etc.

However, in the general case, it might make sense to put everything on a single big EBS volume, and use LVM (or, I suppose, plain partitions). Amazon gives you an IOPS limit on each volume. By default, this limit scales with the size of the volume. e.g., for gp2 volumes you get 3 IOPS per GiB (minimum of 100 IOPS). See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html

For most workloads, you will want all your available IOPS to be available to any filesystem, depending on the need at the moment. So it makes sense to make one big EBS volume, get all your IOPS in one bucket, and partition/LVM it up.

Example:

3 disks with independent filesystems/swap areas, each 100GB in size. Each gets 300 IOPS. Performance is limited to 300 IOPS on each disk.

1 disk, 300GB in size. LVM partitions on the disk of 100GB each. The disk gets 900 IOPS. Any of the partitions can use all 900 IOPS.

| improve this answer | |
8

Logical volumes are easier to create on the fly, resize, delete.
The "to LVM or not" question always has the same answer, it depends :)
It makes sense if you need the flexibility at the disk(s), partition(s) level.
It doesn't make much sense if you do not need the flexibility provided by LVM or do not want to take advantage of other LVM features

| improve this answer | |
7

I actually like using LVs because they are not easily accessible from the virt-server. Thus these files can not easyly be destroyed/moved by chance.

Other important features of LVs:

  • You can make snapshots
  • You can analyze disk IO based on LV (iostat)
  • Easy to resize
  • By using snapshots you can make a consistent clone of running systems

To reduce complexity I use a LV as disk (not as partition). The drawback is that I can only easyly resize the last partition of the "disk" - but my standard-VM-disk-layout takes that into account (so the last partition contains the important application data).

| improve this answer | |
2

In addition to flexibility, LVM based VM images have potentially less overhead, because they are not accessed through a filesystem. On the other hand, it takes away the means of easily moving images around, like you would do with a file. Not impossible, but a bit more complicated

| improve this answer | |
  • 1
    That's an answer to a different question. The question here was about using LVM in the guests, not using LVM on the host (LVs to store the VM disks). – Stéphane Chazelas Oct 29 '15 at 17:13
  • 1
    No, the question was about using LVM FOR the guests, not IN the guests. – HDave Nov 13 '17 at 14:11
1

I actually only use LVM's for the backing storage at the hypervisor level, image files are for the birds. I would also recommend using them at the guest level as well. It's true that you won't benefit from pooling disparate storage sources or find it easier to increase total disk space available (since you can get that just as easily by resizing what the hypervisor is presenting), but sometime you allocate too much to one filesystem. You might like an easy way to take 1 gig from /opt and give it to /var (for example). If you're doing regular partitions inside the VM itself then it makes the resizing aspect just that much harder.

| improve this answer | |
1

In addition to the other good answers here, the only really good reason to use LVM inside a VM is you want a test environment to experiment with and get some practical hands-on experience with LVM.

You can go through various HOWTOs and tutorials, practice common (and not-so-common) LVM administration taks, set up various failure scenarios, and learn how to deal with them.

i.e. as a self-teaching aid.

| improve this answer | |
1

My own experience....

I wanted to use a logical volume (lv) with lvm2 for an ext4 file system; not as a disk, or rather, simply as a non-partitioned raw disk for the fs.

What I found was that startup of the VM would be stopped at the initrd stage; if I commented out the /etc/fstab entry, the machine would boot. Leaving the /etc/fstab entry commented was not a solution that I was happy to live with. So, I created a normal disk image (still a logical volume), partitioned it with one partition using fdisk and created the file system on that. No further issues.

The relevant mounts in my /etc/fstab were using UUID.

I thought about using file or filesystem, but decided against it.

In my case, I am using a Debian Jessie based system Devuan

| improve this answer | |
0

Edit: The below is no longer true. The value of using thin provisioning provided by LVM for VM disk images is probably situational;

Are you running Development VM's on a laptop? then you're probably better off with QCow2.

Managing a farm of VM's that can possibly use vast amounts of storage across multiple disks? LVM is likely a good way to manage that storage.


one reason not to use lvm is you cannot overcommit storage using lvm. If you create an 10 VM's with 100GB of storage, you need 1000 GB of actual disk, even if 9 out of the ten VM's will only ever use 20 GB of their filesystems. Sparse Disk images or qcow2 format images can mean only the storage actually used by the guests need be allocated to them.

Whether this is actually useful to you depends on what you need out of your storage.

| improve this answer | |
  • 2
    You actually can overcommit storage using lvm snapshots. It has overhead, though. – derobert Sep 13 '12 at 15:53
  • 3
    and actually, newer versions of LVM support a "thin pool", which is overcommit w/o any snapshot sillyness. – derobert Sep 13 '12 at 15:56
  • It's a valid point, albeit incorrect; the fact that the "standard"/on-the-fly way of using LVM partitions/disks results in the outlined scenario is certainly worth pointing out. Though, the answer could be changed to reflect that it would add even more complexity to the learning path. – ILMostro_7 Feb 11 '16 at 20:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.