Grow LVM RAID5 by identical disk, not enough extents

Question

I have an existing LVM RAID5 array on a CentOS 8 box made up of 3x4TB drives. That array is beginning to run low on space, so I have an identical 4TB drive I'd like to add into the array to increase the total space. However, when I run lvextend /dev/storage/raidarray /dev/sda, I get the following output:

Converted 100%PVS into 953861 physical extents.
Using stripesize of last segment 64.00 KiB
Archiving volume group "storage" metadata (seqno 35).
Extending logical volume storage/raidarray to <10.92 TiB
Insufficient free space: 1430790 extents needed, but only 953861 available

Here is the output of pvs:

PV         VG      Fmt  Attr PSize   PFree
/dev/sda   storage lvm2 a--   <3.64t  <3.64t
/dev/sdb3  cl      lvm2 a--  221.98g      0
/dev/sdc   storage lvm2 a--   <3.64t      0
/dev/sdd   storage lvm2 a--   <3.64t      0
/dev/sde   storage lvm2 a--   <3.64t      0
/dev/sdf           lvm2 ---  119.24g 119.24g

lvs -o +devices:

LV        VG      Attr       LSize    Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices
home      cl      -wi-a----- <164.11g                                                     /dev/sdb3(12800)
root      cl      -wi-ao----   50.00g                                                     /dev/sdb3(0)
swap      cl      -wi-ao----   <7.88g                                                     /dev/sdb3(54811)
raidarray storage rwi-aor---   <7.28t                                    100.00           raidarray_rimage_0(0),raidarray_rimage_1(0),raidarray_rimage_2(0)

pvdisplay:

--- Physical volume ---
PV Name               /dev/sdb3
VG Name               cl
PV Size               221.98 GiB / not usable 3.00 MiB
Allocatable           yes (but full)
PE Size               4.00 MiB
Total PE              56827
Free PE               0
Allocated PE          56827
PV UUID               MM6j63-1V3E-YWXl-61ro-f3bB-7ysd-c1DGQv

--- Physical volume ---
PV Name               /dev/sdc
VG Name               storage
PV Size               <3.64 TiB / not usable <3.84 MiB
Allocatable           yes (but full)
PE Size               4.00 MiB
Total PE              953861
Free PE               0
Allocated PE          953861
PV UUID               rmqBBu-DD8U-d7WW-yzKW-R97b-1M4r-RYb1Qx

--- Physical volume ---
PV Name               /dev/sdd
VG Name               storage
PV Size               <3.64 TiB / not usable <3.84 MiB
Allocatable           yes (but full)
PE Size               4.00 MiB
Total PE              953861
Free PE               0
Allocated PE          953861
PV UUID               TBn2He-cRTU-eybT-fuBM-REbO-YNfr-Ca86gU

--- Physical volume ---
PV Name               /dev/sde
VG Name               storage
PV Size               <3.64 TiB / not usable <3.84 MiB
Allocatable           yes (but full)
PE Size               4.00 MiB
Total PE              953861
Free PE               0
Allocated PE          953861
PV UUID               wHZOf0-KTK9-2qLW-USl9-Gkgz-6MjV-D3gWrH

--- Physical volume ---
PV Name               /dev/sdf
VG Name               storage
PV Size               119.24 GiB / not usable <4.34 MiB
Allocatable           yes
PE Size               4.00 MiB
Total PE              30525
Free PE               30525
Allocated PE          0
PV UUID               MWWaUJ-UC2h-YT29-bMol-fWoQ-5Chl-uKBB4O

--- Physical volume ---
PV Name               /dev/sda
VG Name               storage
PV Size               <3.64 TiB / not usable <3.84 MiB
Allocatable           yes
PE Size               4.00 MiB
Total PE              953861
Free PE               953861
Allocated PE          0
PV UUID               vzGHi9-TF42-EFx9-uLch-EioJ-DI35-RuZuJt

and lsblk:

NAME                         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                            8:0    0   3.7T  0 disk
sdb                            8:16   0 223.6G  0 disk
├─sdb1                         8:17   0   600M  0 part /boot/efi
├─sdb2                         8:18   0     1G  0 part /boot
└─sdb3                         8:19   0   222G  0 part
  ├─cl-root                  253:0    0    50G  0 lvm  /
  └─cl-swap                  253:1    0   7.9G  0 lvm  [SWAP]
sdc                            8:32   0   3.7T  0 disk
├─storage-raidarray_rmeta_0  253:7    0     4M  0 lvm
│ └─storage-raidarray        253:14   0   7.3T  0 lvm  /home
└─storage-raidarray_rimage_0 253:8    0   3.7T  0 lvm
  └─storage-raidarray        253:14   0   7.3T  0 lvm  /home
sdd                            8:48   0   3.7T  0 disk
├─storage-raidarray_rmeta_1  253:9    0     4M  0 lvm
│ └─storage-raidarray        253:14   0   7.3T  0 lvm  /home
└─storage-raidarray_rimage_1 253:10   0   3.7T  0 lvm
  └─storage-raidarray        253:14   0   7.3T  0 lvm  /home
sde                            8:64   0   3.7T  0 disk
├─storage-raidarray_rmeta_2  253:11   0     4M  0 lvm
│ └─storage-raidarray        253:14   0   7.3T  0 lvm  /home
└─storage-raidarray_rimage_2 253:12   0   3.7T  0 lvm
  └─storage-raidarray        253:14   0   7.3T  0 lvm  /home
sdf                            8:80   0 119.2G  0 disk
sdg                            8:96   1  14.8G  0 disk
└─sdg1                         8:97   1  14.8G  0 part

I've been searching around for answers to this question, but can find very little written about LVM RAID; only mdadm. Is anyone aware of a way that I can extend the RAID array without purchasing additional drives and without data loss?

Answer 1

I don't normally use LVM RAID so excuse me if I reproduce your situation a bit imperfectly. So the numbers will be a bit weird.

Given what would be a 3 device RAID 5 in mdadm. In LVM terms, this is called a raid5 with 2 stripes (the parity is not counted).

# lvs -o +devices HDD/raidtest
  LV       VG  Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices                                                       
  raidtest HDD rwi-a-r--- 256.00m                                    100.00           raidtest_rimage_0(0),raidtest_rimage_1(0),raidtest_rimage_2(0)

Growing it by one more stripe works like this:

# lvconvert --stripes 3 HDD/raidtest
  Using default stripesize 64.00 KiB.
  WARNING: Adding stripes to active logical volume HDD/raidtest will grow it from 4 to 6 extents!
  Run "lvresize -l4 HDD/raidtest" to shrink it or use the additional capacity.
Are you sure you want to add 1 images to raid5 LV HDD/raidtest? [y/n]: maybe
[... this takes a while ...]
  Logical volume HDD/raidtest successfully converted.

Things to look out for: the WARNING message should clearly state that the device is growing, not shrinking.

Also I did not specify which PV to use for the extension so LVM picked it on its own. In your case this is also optional and should just work (as there are no other eligible PV) but feel free to keep specifying it so there will be no surprises.

Result:

# lvs -o +devices HDD/raidtest
  LV       VG  Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices                                                                            
  raidtest HDD rwi-a-r--- 384.00m                                    100.00           raidtest_rimage_0(0),raidtest_rimage_1(0),raidtest_rimage_2(0),raidtest_rimage_3(0)

The filesystem will not grow along in this case, you're given the choice to either do that separately or use lvresize to shrink the LV back to what it was before (just distributed to more drives now). I guess that is useful when using multiple RAID LVs side by side, instead of giving the entire disk to a single one as you seem to be doing.