2

Due to my increasing amount of important data, I need a software RAID5 array for redundancy, flexibility and cost effectiveness.

ZFS is not an option for me, since my distro (OpenWrt) does not support it. Btrfs RAID5 is broken for so long and it is unlikely the developers can fix it. And from what I've heard, 𝐋𝐕𝐌's RAID5 is nowhere comparable to that of 𝐦𝐝𝐚𝐝𝐦.

Now my idea is classically simple: 𝐫𝐞𝐚𝐥 𝐝𝐞𝐯𝐢𝐜𝐞𝐬 → 𝐦𝐝𝐚𝐝𝐦 → 𝐋𝐔𝐊𝐒 (optional) → 𝐥𝐯𝐦𝟐 → 𝐟𝐢𝐥𝐞𝐬𝐲𝐬𝐭𝐞𝐦 (I'd like to go with Btrfs for block checksum and transparent zstd compression).

What would happen when I add/remove HDDs to grow or shrink my 𝐦𝐝𝐚𝐝𝐦 RAID5 device? Can the 𝐋𝐔𝐊𝐒 and 𝐋𝐕𝐌 devices be grown or shrunk accordingly or it is not possible?

enter image description here

Improve this question
1
  • It's possible to grow/shrink both LVM and LUKS and using LVM on top of mdadm is quite normal. Instead of RAID 0 you could do RAID 5 across 500GB partitions. I find that to be easier to handle and if /dev/sdb goes bad you don't lose /dev/sda too... but it's a matter of taste / opinion.
    – frostschutz
    Commented Mar 27, 2020 at 12:26

2 Answers 2

Reset to default
4

Your MD device ends up being a PV like any other from the point of view of LVM, nothing changes on the LV side so all your LVs are resizable as usual. The MD device itself is resizable too, you can grow your RAID device if you change your drives (or add more) and then resize the PV.

I don’t use RAID 5 so I don’t know what LVM’s support for that is like nowadays compared to MD’s; but I have switched to LVM only for RAID 1 and that works just fine. (On the kernel side, most if not all of the replication code is shared nowadays; I don’t know what the tooling is like now for recovery on replicated LVM compared to replicated MD, on RAID 5.)

Improve this answer
3
  • When you resize a partition, the filesystem on top of it needs to be resized too. I know the mdadm device is resizeable, but when it is, does the LUKS and Volume Group on top of it need to be resized manually?
    – Livy
    Commented Mar 27, 2020 at 14:25
  • The LUKS mapping can be resized with cryptsetup resize <name>, and the LVM PV with pvresize. And yes, both can be done while the thing in question is active and in use. For shrinking, both commands will accept an option to specify the new size; without the size option they'll check the current size of the underlying device and expand to fill it if needed.
    – telcoM
    Commented Mar 27, 2020 at 14:32
  • Resizing a PV is independent of the LVs it hosts. You can resize a PV without needing to do anything to the LVs contained in the VG it’s a part of.
    – Stephen Kitt
    Commented Mar 27, 2020 at 15:18
3

I run LVM on LUKS on mdadm (RAID 6). When you increase the effective disk space at the physical layer you just need to update the available space in order, layer by layer

  • Physical
  • RAID
  • LUKS
  • PV for VG/LVM

If you get the order of operations wrong nothing bad will happen - you'll just get no change.

Note that if you are shrinking it is absolutely essential to get the order of operations correct. Start with the top-most layer - typically the filesystem - and work down the layers. Personally the few times I've needed to do this (a) I take a backup, (b) I shrink the filesystem more then I need to, and for each layer I allow a little safety margin. At the end of that I resize everything back up again to fill the margins.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .