Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.
Sign up to join this community
Should LVM be used for the partitions when creating VM images (e.g., KVM images)? It seems like it adds complexity if you want to, say, mount a qcow2 image in the host if the image has LVM partitions.
On the other hand, it doesn't seem like the advantages of LVM partitions are as significant on a VM image, since it's much easier to take a VM offline and resize partitions than it is for a physical system.
"It depends."
If you are on an environment that you control (vmware or kvm or whatever), and can make your own decisions about disk performance QoS, then I'd recommend not using LVM inside your VMs. It doesn't buy you much flexibility that you couldn't get at the hypervisor level.
Remember, the hypervisor is already effectively performing these tasks. If you want to be able to arbitrarily resize file systems (a fine idea), just create a separate virtual disk for each filesystem.
One thing you might think of as you go down this road. You don't even necessarily need to put partitions on your virtual disks this way. For example, you can create a virtual disk for /home; it is /dev/vdc inside your vm. When creating the filesystem, just do something like mke2fs -j /dev/vdc instead of specifying a partition.
This is a fine idea, but...most tools (and other admins who come after you) will expect to see partitions on every disk. I'd recommend just putting a single partition on the disk and be done with it. It does mean one more step when resizing the filesystem, though. And don't forget to properly align your partitions - starting the first partition at 1MB is a good rule of thumb.
All that said - Doing this all at the hypervisor level means that you probably have to reboot the VM to resize partitions. Using LVM would allow you to hot-add a virtual disk (presuming your hypervisor/OS combination allows this), and expand the filesystem without a reboot. This is definitely a plus.
Meanwhile, if you are using a cloud provider, it's more subtle.
I don't know much about Azure, GCP, or any of the smaller players, so I can't help there.
With AWS you can follow my advice above and you'll often be just fine. You can (now) increase the size of EBS volumes (virtual disks) on-the-fly, and resize partitions, etc.
However, in the general case, it might make sense to put everything on a single big EBS volume, and use LVM (or, I suppose, plain partitions). Amazon gives you an IOPS limit on each volume. By default, this limit scales with the size of the volume. e.g., for gp2 volumes you get 3 IOPS per GiB (minimum of 100 IOPS). See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html
For most workloads, you will want all your available IOPS to be available to any filesystem, depending on the need at the moment. So it makes sense to make one big EBS volume, get all your IOPS in one bucket, and partition/LVM it up.
Example:
3 disks with independent filesystems/swap areas, each 100GB in size. Each gets 300 IOPS. Performance is limited to 300 IOPS on each disk.
1 disk, 300GB in size. LVM partitions on the disk of 100GB each. The disk gets 900 IOPS. Any of the partitions can use all 900 IOPS.
Logical volumes are easier to create on the fly, resize, delete.
The "to LVM or not" question always has the same answer, it depends :)
It makes sense if you need the flexibility at the disk(s), partition(s) level.
It doesn't make much sense if you do not need the flexibility provided by LVM or do not want to take advantage of other LVM features
I actually like using LVs because they are not easily accessible from the virt-server. Thus these files can not easyly be destroyed/moved by chance.
Other important features of LVs:
iostat)To reduce complexity I use a LV as disk (not as partition). The drawback is that I can only easyly resize the last partition of the "disk" - but my standard-VM-disk-layout takes that into account (so the last partition contains the important application data).
In addition to the other good answers here, the only really good reason to use LVM inside a VM is you want a test environment to experiment with and get some practical hands-on experience with LVM.
You can go through various HOWTOs and tutorials, practice common (and not-so-common) LVM administration taks, set up various failure scenarios, and learn how to deal with them.
i.e. as a self-teaching aid.
I actually only use LVM's for the backing storage at the hypervisor level, image files are for the birds. I would also recommend using them at the guest level as well. It's true that you won't benefit from pooling disparate storage sources or find it easier to increase total disk space available (since you can get that just as easily by resizing what the hypervisor is presenting), but sometime you allocate too much to one filesystem. You might like an easy way to take 1 gig from /opt and give it to /var (for example). If you're doing regular partitions inside the VM itself then it makes the resizing aspect just that much harder.
My own experience....
I wanted to use a logical volume (lv) with lvm2 for an ext4 file system; not as a disk, or rather, simply as a non-partitioned raw disk for the fs.
What I found was that startup of the VM would be stopped at the initrd stage; if I commented out the /etc/fstab entry, the machine would boot. Leaving the /etc/fstab entry commented was not a solution that I was happy to live with. So, I created a normal disk image (still a logical volume), partitioned it with one partition using fdisk and created the file system on that. No further issues.
The relevant mounts in my /etc/fstab were using UUID.
I thought about using file or filesystem, but decided against it.
In my case, I am using a Debian Jessie based system Devuan
In addition to flexibility, LVM based VM images have potentially less overhead, because they are not accessed through a filesystem. On the other hand, it takes away the means of easily moving images around, like you would do with a file. Not impossible, but a bit more complicated
Edit: The below is no longer true. The value of using thin provisioning provided by LVM for VM disk images is probably situational;
Are you running Development VM's on a laptop? then you're probably better off with QCow2.
Managing a farm of VM's that can possibly use vast amounts of storage across multiple disks? LVM is likely a good way to manage that storage.
one reason not to use lvm is you cannot overcommit storage using lvm. If you create an 10 VM's with 100GB of storage, you need 1000 GB of actual disk, even if 9 out of the ten VM's will only ever use 20 GB of their filesystems. Sparse Disk images or qcow2 format images can mean only the storage actually used by the guests need be allocated to them.
Whether this is actually useful to you depends on what you need out of your storage.
