I want to move all my 14.5 TB of media drives (not OS) to a combined LVM file system due to constant problems arranging things to fit into multiple smaller file systems.

My question is if after setup any of the 6 drives moves to a different location (/dev/sd*), is that going to be a problem? I have always mounted them based on UUID, but I don't know LVM enough to know how it works with multiple drives. I know I can still mount the file system based on UUID, but I want to make sure LVM is not going to be messed up finding the individual parts of the system.

I have to ask this since, for some reason, if I reboot with USB drives inserted they get lower sd* letters than some of the media drives and it causes those media drives to be rearranged for that boot only.

PS. I maintain off site backups of my media so I'm not to worried about if one drive fails breaking stuff. Only mentioned since my Google searches of LVM always have someone trying to talk the person out of it because one problem loses everything.

  • It scans all the block devices according to rules defined in /etc/lvm/lvm.conf – Stéphane Chazelas Jan 14 '14 at 10:01
up vote 2 down vote accepted

Each LVM object (physical volume, volume group, logical volume) has a UUID. LVM doesn't care where physical volumes are located and will assemble them as long as it can find them.

By default, LVM (specifically vgscan, invoked from an init script) scans all likely-looking block devices at boot time. You can define filters in /etc/lvm.conf. As long as you don't define restrictive filters, it doesn't matter how you connect your drives. You can even move partitions around while the system isn't running and LVM will still know how to assemble them.

You hardly ever need to interact with LVM's UUIDs. Usually you would refer to a volume group by name and to a logical volume by its name inside its containing volume group.

If you use LVM for all your volumes, the only thing that may be affected by shuffling disks around is your bootloader.

Bit of a rant about RAID

NOTE #1: Use caution when setting up LVM. In older versions RAID was not supported and so one would have to set up RAID (mdadm) together with LVM. Setting up your disks using just LVM to pool them puts your data at risk! It's much wiser to create a RAID using the disks, and then layer LVM on top of the RAID array.

NOTE #2: In more recent versions of LVM there is now actual RAID functionality baked into LVM.

excerpt from LVM wikipedia page

features:

  • Create RAID logical volumes (since recent LVM implementations, such as Red Hat Enterprise Linux v6 1): RAID1, RAID5, RAID6, etc.
  • Stripe whole or parts of logical volumes across multiple PVs, in a fashion similar to RAID 0.
  • Mirror whole or parts of logical volumes, in a fashion similar to RAID 1.

Example

From the lvcreate man page.

  • Creates a 5GiB RAID10 logical volume "vg00/my_lv", with 2 stripes on 2 2-way mirrors. Note that the -i and -m arguments behave differently. The -i specifies the number of stripes. The -m specifies the number of additional copies:

    $ sudo lvcreate --type raid10 -L 5G -i 2 -m 1 -n my_lv vg00
    
  • The following command creates a RAID6 array (3 stripes + 2 implicit parity drives) named my_lv in the volume group my_vg that is 1G in size.

    $ sudo lvcreate --type raid6 -i 3 -L 1G -n my_lv my_vg
    

Your question about UUIDs

If you take a look at your system's lvm.conf file, often stored here, /etc/lvm/lvm.conf you'll notice there are rules that specify which block devices to scan.

excerpt from my Fedora 19 system

# By default we accept every block device:
filter = [ "a/.*/" ]

# Exclude the cdrom drive
# filter = [ "r|/dev/cdrom|" ]

# When testing I like to work with just loopback devices:
# filter = [ "a/loop/", "r/.*/" ]

# Or maybe all loops and ide drives except hdc:
# filter =[ "a|loop|", "r|/dev/hdc|", "a|/dev/ide|", "r|.*|" ]

# Use anchors if you want to be really specific
# filter = [ "a|^/dev/hda8$|", "r/.*/" ]

The rules that start with a| add devices, the rules that start with r| remove devices from consideration.

NOTE: You can also dump the config using the command sudo lvm dumpconfig. This will show you the running configuration.

$ sudo lvm dumpconfig | head -10
devices {
    dir="/dev"
    scan="/dev"
    obtain_device_list_from_udev=1
    preferred_names=["^/dev/mpath/", "^/dev/mapper/mpath", "^/dev/[hs]d"]
    filter="a/.*/"
    cache_dir="/etc/lvm/cache"
    cache_file_prefix=""
    write_cache_state=1
    sysfs_scan=1
...

A walk through

So when your system first boots, there's a init/systemd/etc. service that starts up and invokes vgscan. This application uses the lvm.conf file to determine which block devices are in play and which are not.

You can run vgscan with additional switches to see what it's doing. It's safe to run this command afterwards.

Example

To get more verbosity simply add additional -v switches to get more or less.

$ sudo vgscan -vvv |& less

step #1 - find devices

As this runs you can see as it goes through all the block devices like so:

Wiping cache of LVM-capable devices
    /dev/sda: Added to device cache
    /dev/disk/by-id/ata-ST95005620AS_5YX1K0Q5: Aliased to /dev/sda in device cache
    /dev/disk/by-id/wwn-0x5000c50044b84d12: Aliased to /dev/sda in device cache
    /dev/sda1: Added to device cache
    /dev/disk/by-id/ata-ST95005620AS_5YX1K0Q5-part1: Aliased to /dev/sda1 in device cache
    /dev/disk/by-id/wwn-0x5000c50044b84d12-part1: Aliased to /dev/sda1 in device cache
    /dev/disk/by-uuid/190ba40f-0c79-462c-b592-0b3d6450f40a: Aliased to /dev/sda1 in device cache
    /dev/sda2: Added to device cache

Eventually it will get to several devices that are part of an LVM.

TH7Tn397qrQAETbHjeSaO0AwZk7zbm7r7NnkhRk9ndQ65S0K: Aliased to /dev/disk/by-id/dm-name-fedora_greeneggs-swap in device cache
        /dev/disk/by-uuid/5c863357-78a9-4e8d-a66a-c3bc25695ce8: Aliased to /dev/disk/by-id/dm-name-fedora_greeneggs-swap in device cache
        /dev/fedora_greeneggs/swap: Aliased to /dev/disk/by-id/dm-name-fedora_greeneggs-swap in device cache (preferred name)
        /dev/mapper/fedora_greeneggs-swap: Aliased to /dev/fedora_greeneggs/swap in device cache
        /dev/dm-1: Added to device cache
        /dev/disk/by-id/dm-name-fedora_greeneggs-root: Aliased to /dev/dm-1 in device cache (preferred name)
        /dev/disk/by-id/dm-uuid-LVM-UHlsLRdlvpobBVb5TH7Tn397qrQAETbHH5YiQx0WJaOqsUAs7hyr5FK8r0i0fR8D: Aliased to /dev/disk/by-id/dm-name-fedora_greeneggs-root in device cache

step #2 - find VG's

It will then find out what VG (Volume Groups) are present:

  Setting name to fedora_greeneggs
  Setting metadata/format to lvm2
  Setting id to 6Cf2tN-ayiy-PIO7-44We-20vs-oTEt-QWS3JR

step #3 - identify LV's

Once it's identified the VG, it will look to identify the LV (Logical Volumes).

        Allocated VG fedora_greeneggs at 0x206a1d0.
        /dev/sda2 0:      0   1956: swap(0:0)
        /dev/sda2 1:   1956 104353: home(0:0)
        /dev/sda2 2: 106309  12800: root(0:0)
  Found volume group "fedora_greeneggs" using metadata type lvm2

At this point LVM (and your system) now have identified which block devices are members of which VG's.

UUIDs

If you're still unconvinced that these are used throughout the LVM stack to identify physical volumes, volume groups, and logical volumes you can scan the vgscan output looking for strings that are UUID's.

For example, one of the LV's in my LVM VG has this bit in its UUID, 5ece678c. Looking for this string in blkid output as well as vgscan turns up this:

$ blkid | grep 5ece678c
/dev/mapper/fedora_greeneggs-home: UUID="5ece678c-0382-4015-94e8-88c06ecdd872" TYPE="ext4" 

$ sudo vgscan -vvv |& grep 5ece678c
        /dev/disk/by-uuid/5ece678c-0382-4015-94e8-88c06ecdd872: Aliased to /dev/disk/by-id/dm-name-fedora_greeneggs-home in device cache

The physical disk that is part of that VG has this bit in its UUID, 6Cf2tN:

$ blkid | grep 6Cf2tN
/dev/sda2: UUID="6Cf2tN-ayiy-PIO7-44We-20vs-oTEt-QWS3JR" TYPE="LVM2_member" 

$ sudo vgscan -vvv |& grep 6Cf2tN
      Setting id to 6Cf2tN-ayiy-PIO7-44We-20vs-oTEt-QWS3JR

The VG's UUID has this bit in it, UHlsLR. This won't show up in the output of blkid, but you can use vgs to find the UUID's of all the VG's.

$ sudo vgs -v |& grep UHlsLR
  fedora_greeneggs wz--n- 4.00m   1   3   0 465.27g    0  UHlsLR-dlvp-obBV-b5TH-7Tn3-97qr-QAETbH

$ sudo vgscan -vvv |& grep UHlsLR
        /dev/disk/by-id/dm-uuid-LVM-UHlsLRdlvpobBVb5TH7Tn397qrQAETbHjeSaO0AwZk7zbm7r7NnkhRk9ndQ65S0K: Aliased to /dev/disk/by-id/dm-name-fedora_greeneggs-swap in device cache
        /dev/disk/by-id/dm-uuid-LVM-UHlsLRdlvpobBVb5TH7Tn397qrQAETbHH5YiQx0WJaOqsUAs7hyr5FK8r0i0fR8D: Aliased to /dev/disk/by-id/dm-name-fedora_greeneggs-root in device cache
        /dev/disk/by-id/dm-uuid-LVM-UHlsLRdlvpobBVb5TH7Tn397qrQAETbHF4nEeJnAPJdZG6gNAs8DRGW7FegWi5JI: Aliased to /dev/disk/by-id/dm-name-fedora_greeneggs-home in device cache
        lvmcache: /dev/sda2: setting fedora_greeneggs VGID to UHlsLRdlvpobBVb5TH7Tn397qrQAETbH

This last set of commands is interesting because it shows the VG UUID (VGID) along with the LV's that are within it. Notice how the UUID for the VG gets incorporated into the /dev/disk/by-id/... device ID's?

Going further

If you're the curious type, I encourage you to take a look at the output of vgscan -vvv to at least familiarize yourself with the content. If you later want to explore things further than you can use the command line tools listed in the man lvm man page page to interact with LVM's various levels.

  • pvs - physical volumes
  • vgs - volume groups
  • lvs - logical volumes

There are a whole host of other tools, these are some of the more common ones.

References

  • re: your RAID advice, I thought LVM could automatically do RAID. (maybe I'm thinking of btrfs.) – strugee Jan 15 '14 at 3:20
  • 1
    In the newer versions it's purported to be able to do RAID. See the LVM Wikipedia page: en.wikipedia.org/wiki/Logical_Volume_Manager_(Linux). I'll add this bit to the A. – slm Jan 15 '14 at 3:28
  • @strugee - added to the top. LMK what you think. – slm Jan 15 '14 at 3:54
  • I wish I could do raid but my motherboard only has 8 SATA ports all full with me already forced to choose leave stuff unplugged. And any future space upgrades being replacing the smallest with a higher capacity. – qawsedrf Jan 15 '14 at 5:23
  • @qawsedrf - from your Q I figured you weren't considering RAID. Understood, just use caution since the probability of a failure goes up w/ each disk you pool. – slm Jan 15 '14 at 5:33

I don't have time to write a full answer right now, but essentially, you have almost nothing to worry about if you mount based on UUID. Things should generally work (this is really because files in /dev/mapper/ are symlinked into /dev/disk/by-uuid/, just like regular /dev/sdx* files).

However, you must ensure that your LVM devices are set up during the initrd. Otherwise, your boot will fail, because it won't be able to find the root.

  • I shouldn't have to worry about initrd since I'm not doing the OS drive just my media which will be mounted under $HOME/media – qawsedrf Jan 14 '14 at 4:34

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.