Move data from regular partition to lvm

Question

I've read a bit about lvm and decided that I want to switch to using it. It seems like actually setting up lvm can be done from the debian installer, so I suppose that'll be easiest. What I'm not sure about is how to copy all of my files to the new system.

I would like to copy my whole system, not just my home directory or just my packages, I've messed around with a lot of stuff, to the point where I have no idea exactly where all of my config files live. The guides I've found on the internet seem to glaze over this step, just mentioning tools that can do it, not how to use them, i.e. use rsync. Instinct tells me I can't just rsync the root of my backup to the root of my lvm system, I'll copy over something important, like the lvm config files, but I'm not sure what exactly I need to exclude, while being sure not to exclude anything I might have changed. Any tips?

My planned steps, for clarity: I have a 1tb hdd and a 250gb ssd. I want to shrink the partition I have on the hdd, use the free space (I have enough) to make two new partitions, and dd my ssd and hdd partitions to those partitions as a backup. Then I want to wipe the ssd partition and hdd partition I'm currently using, and set up lvm on them (ideally with the debian installer for simplicity's sake, but maybe manually). And then I want to copy everything from the backup partitions to the new lvm partitions, somehow. And then wipe the backup partitions and either resize the existing lvm partitions to include the remaining space, or just make the remaining space another physical volume on lvm and add it that way.

Answer 1

Don't install a new system unless you want to redo your system configuration from scratch. Note that you may want to do that — copy your home directory over, and progressively restore the system configuration, taking care to 1. install etckeeper as soon as you've rebooted after the system installer and 2. not make changes outside of /etc and your home directory. But if you want to keep your existing installation, moving it to a different partition scheme is a lot easier than moving it on top of another installation.

You can't “LVMize” a partition with official tools. There is a third-party tool called blocks to-lvm (originally known as lvmify) which can do this. I've never tried it. You may wish to try it, but make very sure your backups are up-to-date first.

Assuming you don't use blocks-to-lvm, you'll need to make some room on the disk, create some space for the LVM volumes, move data over, and repeat until all the data is moved. Many filesystems, including ext4 (the default for Debian), can't be resized while they're mounted, so you'll need to do all this from a live system such as SystemRescueCD or GParted Live.

1. Shrink one of your existing filesystems and the containing partition. Let's say you're working on sda and currently have partition number 1. Shrink your filesystem to half of what it was minus some change if possible, because that lets you move it over in one go. If you can't do that, shrink the filesystem to have little free space left. There are two ways to do this:

   • Use parted /dev/sda (text mode) or gparted (GUI, select sda) and its resizepart 1/“Resize” command.
   • Use resize2fs /dev/sda1 to shrink the filesystem, then fdisk /dev/sda or some other partition utility. Note that fdisk has no command to resize a partition, you need to delete the partition then create a partition with the same characteristics except for the size. This is not difficult, but it is error-prone; miscopying a number or mixing up units can destroy your data.

   At this point, delete your swap partition, too, to make a little more room.

2. In the now free space, create a partition. Declare it of type 8e (MBR) or “Linux LVM” (GPT). Let's say the new partition is number 2, so its device path is /dev/sda2.

3. Run pvcreate /dev/sda2 to make the new partition an LVM physical volume. This allocates /dev/sda2 for use by LVM, but doesn't yet use the space for anything.
4. Run vgcreate drowhd /dev/sda2, where drowhd is the name of your choice for the volume group. This creates some working space on LVM: a volume group.
5. Run lvcreate -n root -L 42g drowhd where root is the name you're giving to your first logical volume and 42g is the size. A logical volume is where you can put a filesystem or swap space.

6. Move some data.

   • If the logical volume is at least as large as your existing root filesystem now is, you can copy it wholesale.

     cat /dev/sda1 >/dev/mapper/drowhd-root
     

   • If you didn't have enough room for that, you'll need to move the files in pieces. Create a filesystem on the new volume, then move some files over.

     mkfs.ext4 /dev/mapper/drowhd-root
     mount /dev/sda1 /media/old-root
     mount /dev/mapper/drowhd-root /media/new-root
     mkdir /media/new-root/usr
     mv /media/old-root/usr/lib /media/new-root/usr
     umount /media/old-root
     

     Now repeat steps 1–3 to shrink the old root filesystem and partition some more and create a new physical volume (say /dev/sda3) on the now-free space. Add the new physical volume to the existing volume group and enlarge your logical volume:

     pvcreate /dev/sda3
     vgextend drowhd /dev/sda3
     lvresize -L 99g drowhd/root
     

     Move some more files as above. Repeat until you've moved everything.

7. When you've moved everything, turn the former data partition into a physical volume and add it to the volume group:

   pvcreate /dev/sda1
   vgextend drowhd /dev/sda1
   

   You can use this space to extend existing logical volumes or create new ones. Don't forget to create a logical volume for swap and to initialize it with mkswap. You may want to leave some space free to create snapshots, if you intend to use this feature.

8. Unless you have a separate /boot partition and you didn't modify it, you need to update your bootloader configuration. For Grub:

   1. Mount the new system partition somewhere:

      mount /dev/mapper/drowhd-root /media/new-root
      

   2. Make some in-memory filesystems available under the new root:

      mount --rbind /dev /media/new-root/dev
      mount --bind /proc /media/new-root/proc
      mount --bind /sys /media/new-root/sys
      mount --bind /run /media/new-root/run
      

   3. Get a command line inside your system:

      chroot /media/new-root
      

   4. Run update-grub
   5. Exit the chroot and unmount all the stuff.

9. You probably need to update /etc/fstab (unless you did only whole-filesystem move with cat and your fstab uses only UUID= or LABEL= to identify partitions). Edit the file to reflect the new partition locations.

LVM itself doesn't keep any configuration about your disks in /etc, they're all stored on the physical volumes themselves. The configuration files under /etc/lvm are configuration for the LVM tools (scanning rules, logging preferences, etc.) and a cache of the volume arrangement that's updated automatically. You can access LVM volumes from any system where you plug in the disk containing them.

Use one volume group per underlying block device. There are few circumstances where it's wise to do otherwise. An underlying block device is typically be a disk, or a RAID array, or an encrypted volume. In your case, create a VG on the hard disk and a VG on the SSD, unless you set up some RAID.