1

Given 2 SSDs and 4 HDDs with their corresponding PVs (/dev/ssd1, /dev/ssd2, /dev/hdd1 to /dev/hdd4).

I know, identifiers are not correct, they are just for easier keeping track of what device I'm talking about.

What I want

  • all the HDDs to build up a RAID10 array
  • have a LVM thin pool on this RAID10
  • all the SSDs to build up a RAID1 array
  • have the metadata for then thin pool being located on this RAID1

What I did

  1. Create a VG with all physical PVs included
    vgcreate appdata /dev/sdd1 /dev/sdd2 /dev/hdd1 /dev/hdd2 /dev/hdd3 /dev/hdd4
    
  2. Create 2 LVs - one for metadata, one for data
    lvcreate --type raid1 -m 1 -n ThinMetaAD -L 15G appdata /dev/ssd1 /dev/ssd2
    lvcreate --type raid10 -n ThinAD -L 465G appdata /dev/hdd1 /dev/hdd2 /dev/hdd3 /dev/hdd4
    
  3. Convert according to lvmthin manual
    lvconvert --type thin-pool --poolmetadata appdata/ThinMetaAD appdata/ThinAD
    

What I got

$ lsblk

NAME                              MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
ssd                                 8:0    0 223.6G  0 disk
└─ssd1                              8:1    0    16G  0 part
  ├─appdata-ThinAD_tmeta_rmeta_0  252:2    0     4M  0 lvm
  │ └─appdata-ThinAD_tmeta        252:6    0    15G  0 lvm
  │   └─appdata-ThinAD            252:16   0   464G  0 lvm
  └─appdata-ThinAD_tmeta_rimage_0 252:3    0    15G  0 lvm
    └─appdata-ThinAD_tmeta        252:6    0    15G  0 lvm
      └─appdata-ThinAD            252:16   0   464G  0 lvm
hdd                                 8:16   0 465.8G  0 disk
└─hdd1                              8:17   0 465.8G  0 part
  ├─appdata-ThinAD_tdata_rmeta_0  252:7    0     4M  0 lvm
  │ └─appdata-ThinAD_tdata        252:15   0   464G  0 lvm
  │   └─appdata-ThinAD            252:16   0   464G  0 lvm
  └─appdata-ThinAD_tdata_rimage_0 252:8    0   232G  0 lvm
    └─appdata-ThinAD_tdata        252:15   0   464G  0 lvm
      └─appdata-ThinAD            252:16   0   464G  0 lvm
hdd                                 8:32   0 465.8G  0 disk
└─hdd2                              8:33   0 465.8G  0 part
  ├─appdata-ThinAD_tdata_rmeta_1  252:9    0     4M  0 lvm
  │ └─appdata-ThinAD_tdata        252:15   0   464G  0 lvm
  │   └─appdata-ThinAD            252:16   0   464G  0 lvm
  └─appdata-ThinAD_tdata_rimage_1 252:10   0   232G  0 lvm
    └─appdata-ThinAD_tdata        252:15   0   464G  0 lvm
      └─appdata-ThinAD            252:16   0   464G  0 lvm
hdd                                 8:48   0 465.8G  0 disk
└─hdd3                              8:49   0 465.8G  0 part
  ├─appdata-ThinAD_tdata_rmeta_2  252:11   0     4M  0 lvm
  │ └─appdata-ThinAD_tdata        252:15   0   464G  0 lvm
  │   └─appdata-ThinAD            252:16   0   464G  0 lvm
  └─appdata-ThinAD_tdata_rimage_2 252:12   0   232G  0 lvm
    └─appdata-ThinAD_tdata        252:15   0   464G  0 lvm
      └─appdata-ThinAD            252:16   0   464G  0 lvm
hdd                                 8:64   0 931.5G  0 disk
└─hdd4                              8:65   0   466G  0 part
  ├─appdata-ThinAD_tdata_rmeta_3  252:13   0     4M  0 lvm
  │ └─appdata-ThinAD_tdata        252:15   0   464G  0 lvm
  │   └─appdata-ThinAD            252:16   0   464G  0 lvm
  └─appdata-ThinAD_tdata_rimage_3 252:14   0   232G  0 lvm
    └─appdata-ThinAD_tdata        252:15   0   464G  0 lvm
      └─appdata-ThinAD            252:16   0   464G  0 lvm
ssd                           
└─ssd2                       259:5    0    16G  0 part
  ├─appdata-ThinAD_tmeta_rmeta_1  252:4    0     4M  0 lvm
  │ └─appdata-ThinAD_tmeta        252:6    0    15G  0 lvm
  │   └─appdata-ThinAD            252:16   0   464G  0 lvm
  └─appdata-ThinAD_tmeta_rimage_1 252:5    0    15G  0 lvm
    └─appdata-ThinAD_tmeta        252:6    0    15G  0 lvm
      └─appdata-ThinAD            252:16   0   464G  0 lvm

Question

I'm wondering if the above setup is correct in the sense of what I initially wanted, in particular I'm worried about the 3 levels below partition level, like here:

hdd                                 8:32   0 465.8G  0 disk
└─hdd2                              8:33   0 465.8G  0 part
>>├─appdata-ThinAD_tdata_rmeta_1  252:9    0     4M  0 lvm
>>│ └─appdata-ThinAD_tdata        252:15   0   464G  0 lvm
>>│   └─appdata-ThinAD            252:16   0   464G  0 lvm
  └─appdata-ThinAD_tdata_rimage_1 252:10   0   232G  0 lvm
    └─appdata-ThinAD_tdata        252:15   0   464G  0 lvm
      └─appdata-ThinAD            252:16   0   464G  0 lvm
  1. Is my setup correct?
  2. If not, what steps are needed to get the desired output in What I want?
0

1 Answer 1

0

lsblk shows it this way due to general device dependencies. For complex setups, this may not be directly related to physical storage, thus it looks "wrong".

Maybe LVM tools give a clearer picture:

# lvs --all --segments --options +seg_le_ranges appdata
  LV                      VG      Attr       #Str Type      SSize  LE Ranges                       
  ThinAD                  appdata twi-a-tz--    1 thin-pool  9.00g [ThinAD_tdata]:0-2303           
  [ThinAD_tdata]          appdata rwi-aor---    2 raid1      9.00g [ThinAD_tdata_rimage_0]:0-2303,[ThinAD_tdata_rimage_1]:0-2303
  [ThinAD_tdata_rimage_0] appdata iwi-aor---    1 linear     9.00g /dev/loop2001:1-2304            
  [ThinAD_tdata_rimage_1] appdata iwi-aor---    1 linear     9.00g /dev/loop2002:1-2304            
  [ThinAD_tdata_rmeta_0]  appdata ewi-aor---    1 linear     4.00m /dev/loop2001:0-0               
  [ThinAD_tdata_rmeta_1]  appdata ewi-aor---    1 linear     4.00m /dev/loop2002:0-0               
  [ThinAD_tmeta]          appdata ewi-aor---    2 raid1     92.00m [ThinAD_tmeta_rimage_0]:0-22,[ThinAD_tmeta_rimage_1]:0-22
  [ThinAD_tmeta_rimage_0] appdata iwi-aor---    1 linear    92.00m /dev/loop1001:1-23              
  [ThinAD_tmeta_rimage_1] appdata iwi-aor---    1 linear    92.00m /dev/loop1002:1-23              
  [ThinAD_tmeta_rmeta_0]  appdata ewi-aor---    1 linear     4.00m /dev/loop1001:0-0               
  [ThinAD_tmeta_rmeta_1]  appdata ewi-aor---    1 linear     4.00m /dev/loop1002:0-0               
  [lvol0_pmspare]         appdata ewi-------    1 linear    92.00m /dev/loop2001:2305-2327         

I tried to approximate your setup with loop devices, /dev/loop100x for meta, /dev/loop200x for data. Your output should list other devices and sizes.

You can see all tmeta stuff is on /dev/loop1xxx and all tdata stuff is on /dev/loop2xxx so it should be as you intended.

The exception is the pmspare (pool metadata spare) volume. This is described in the lvmthin manpage.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .