5

my dedicated server has a RAID1 array with 2x1TB HDD It is filling up

i want add new hard disk (It has already been physically added)

I have no idea how to do it.

I'm afraid of losing data

I found this tutorial

 [root@server ~]# fdisk -l

 Disk /dev/nvme2n1: 1024.2 GB, 1024209543168 bytes, 2000409264 sectors
 Units = sectors of 1 * 512 = 512 bytes
 Sector size (logical/physical): 512 bytes / 512 bytes
 I/O size (minimum/optimal): 512 bytes / 512 bytes


 Disk /dev/nvme1n1: 1024.2 GB, 1024209543168 bytes, 2000409264 sectors
 Units = sectors of 1 * 512 = 512 bytes
 Sector size (logical/physical): 512 bytes / 512 bytes
 I/O size (minimum/optimal): 512 bytes / 512 bytes
 Disk label type: dos
 Disk identifier: 0x844cebe6

Device Boot      Start         End      Blocks   Id  System
 /dev/nvme1n1p1            2048    67110911    33554432   fd  Linux raid autodetect
 /dev/nvme1n1p2        67110912    68159487      524288   fd  Linux raid autodetect
 /dev/nvme1n1p3        68159488  2000407215   966123864   fd  Linux raid autodetect

 Disk /dev/nvme0n1: 1024.2 GB, 1024209543168 bytes, 2000409264 sectors
 Units = sectors of 1 * 512 = 512 bytes
 Sector size (logical/physical): 512 bytes / 512 bytes
 I/O size (minimum/optimal): 512 bytes / 512 bytes
 Disk label type: dos
 Disk identifier: 0xcb3fcb0e

    Device Boot      Start         End      Blocks   Id  System
 /dev/nvme0n1p1            2048    67110911    33554432   fd  Linux raid autodetect
 /dev/nvme0n1p2        67110912    68159487      524288   fd  Linux raid autodetect
 /dev/nvme0n1p3        68159488  2000407215   966123864   fd  Linux raid autodetect

 Disk /dev/md1: 535 MB, 535822336 bytes, 1046528 sectors
 Units = sectors of 1 * 512 = 512 bytes
 Sector size (logical/physical): 512 bytes / 512 bytes
 I/O size (minimum/optimal): 512 bytes / 512 bytes


 Disk /dev/md2: 989.2 GB, 989175545856 bytes, 1931983488 sectors
 Units = sectors of 1 * 512 = 512 bytes
 Sector size (logical/physical): 512 bytes / 512 bytes
 I/O size (minimum/optimal): 512 bytes / 512 bytes


 Disk /dev/md0: 34.3 GB, 34325135360 bytes, 67041280 sectors
 Units = sectors of 1 * 512 = 512 bytes
 Sector size (logical/physical): 512 bytes / 512 bytes
 I/O size (minimum/optimal): 512 bytes / 512 bytes
1
  • 4
    When you said "It is filling up" it gives impression that you want more space by adding that new disk? However, adding it to RAID1 array will not give you any space, as it will simply mirror what other disks have. Maybe you want to simply partition the disk? Aug 19, 2021 at 21:42

4 Answers 4

5

You can grow a RAID1... if you replace both disks. I just did, it worked like this:

  • Started with 2x4T drives (sda1 and sdb1) in a working RAID1 array (md0)
  • Bought two new 16T drives and plugged them in (sdc and sdd)
  • partition them with full drive partitions (sdc1 and sdd1)
  • Added them both to the RAID1: mdadm /dev/md0 --add /dev/sdc1 /dev/sdd1
  • fail one of the old drives: mdadm /dev/md0 -f /dev/sda1
  • wait for sync to finish: watch cat /proc/mdstat
  • fail the other old drive: mdadm /dev/md0 -f /dev/sdb1
  • wait for sync to finish: watch cat /proc/mdstat
  • remove the old drives from the array: mdadm /dev/md0 -r /dev/sda1 /dev/sdb1
  • grow the array: mdadm --grow /dev/md0 --size max
  • grow the filesystem: resize2fs /dev/md0

..and now I have a 14TB RAID1 instead of the initial 4TB.

3
  • IF you are going to use the WHOLE drive as part of an LVM, then there is no reason to partition it. My raidset consists of /dev/sdb /dev/sdc /dev/sdd /dev/sdk3 /dev/sda Only /dev/sdk has a partition table, it is twice the size of the others and so has some scratch partitions on it.
    – user516667
    Aug 9, 2022 at 8:02
  • Agreed, but OP already had partitions, so I kept them in the example for clarity/simplicity.
    – pjz
    Nov 8, 2022 at 21:37
  • 1
    Instead of failing a device prematurely you should grow the raid 1 by the number of added drives mdadm /dev/md0 --grow --raid-devices=4. This ensures that we always have redundancy. After recovery is complete, we can fail the old devices and then shrink the raid again mdadm /dev/md0 --grow --raid-devices=2. Bonus: Recovery is performed on both new devices simultaneously.
    – Weishaupt
    Mar 29, 2023 at 15:18
2

With mdadm, there is a migration path to go from raid1 to raid5. So all you need is a new drive, partition it accordingly, and then grow your existing raid.

For 2 disks, mdadm's raid5 uses the same on-disk layout as raid1, so converting to raid5 (in name only) is instant:

mdadm --grow /dev/md2 --level=5

Growing a 2 disk raid5 to 3 disk raid5 then reshapes all data:

mdadm --grow /dev/md2 --raid-devices=3 --add /dev/newXp3

Once the reshape is finished, the additional storage capacity is available and you can grow the filesystem accordingly using resize2fs or similar.


Note:

It's not entirely clear but you seem to be using raid1 for your boot partitions as well. You can't use raid5 if your boot loader is not raid aware, i.e. if you are using 0.90 or 1.0 version metadata, where metadata is located at the end of partition, so bootloaders see a regular filesystem instead of raid.

This only works with raid1 so you should probably stick to raid1 for your /dev/md0 and/or /dev/md1 devices. You can still grow them to 3 disks like above, just without the level change.

Like already stated, growing raid1 to 3 disks will not add capacity (you need raid5 for that), but it will add redundancy (3-way mirror raid1).

1

That tutorial looks pretty useless because it's about "growing RAID-1" but doesn't even mention the single most important thing about doing that, which is that you can't (not without rebuilding it with larger drives) - the most you can do is increase the redundancy by adding another drive to the array - but the array will still have the same storage capacity. Growing a RAID-1 means your data will be safer because there will be more copies of it, but you won't get any more storage capacity.

RAID-1 mirrors the same data across all devices in the array, the capacity of a RAID-1 array with any number of same-sized drives is the capacity of just one of those drives (and the capacity of a RAID-1 of differently-sized drives is the capacity of the smallest of them).

To increase your storage, you're going to have to boot into rescue mode (or into a rescue CD/DVD/USB/etc) and do a complete backup and restore of your existing RAID-1 array to a new RAID-5 array. Alternatively, you could use LVM's RAID-5 capabilities instead of mdadm's - LVM is very well supported and documented on RHEL and Centos and is pretty much the default environment for those distros. Or, see below, maybe use ZFS instead.

If you have another 1TB drive (say a mechanical HDD rather than another nvme drive), then you can use that as the backup target. Then create a new RAID-5 array with all three of your nvme drives and restore everything to it.

Otherwise, you'll have to use the new nvme drive as the backup target, backup everything to it, then use your original two nvme drives to create a degraded RAID-5 array, restore everything to it, then (once you have tested that the system boots and works on the RAID-5 array), add the new nvme to the array.

The new RAID-5 array will have 2TB (because the capacity of a RAID-5 array with N drives is N-1...3 x 1TB drives in RAID-5 will have 2 x 1TB total storage capacity).

I prefer rsync to do the backup and restore because if a crash or power-failure or something interrupts the process, you can just restart it and it will pick up where it left off, instead of having to start from scratch and copy the same files it already copied before.

There are numerous step-by-step examples of how to do this on this site, search for questions migrating data to a new drive, paying particular attention to questions and answers involving mdadm and lvm (or ZFS). Ignore any that talk about using dd (because you want to do a file-based backup and restore, not a bit copy of the disk images). I know I've written a few answers on this topic over the years (e.g. How to migrate to LVM? and Is it possible to integrate existing partitions with filesystem?), and other people have too.

Once you've done your reading, write out a step-by-step plan of what you need to do. Migration disasters can be avoided with thorough planning. Not having a plan will almost certainly result in disaster, especially if you've never done this before. I've done hundreds of such migrations with physical and virtual machines over the years, and I still take the time to write down a plan in point form because I know that if I don't, I'll forget some important step.

If you can run VMs on your system, then I highly recommend setting up a VM with a minimal install of Centos on a RAID-1 array and an extra disk to practice the migration on before trying it on your real drives. The virtual disks for the VM don't need to be large, a few hundred MB each should be plenty...just enough for a minimal OS (and mdadm, lvm, rsync, and any other tools you'll need). This will make planning the real migration much easier, it's hard to plan something you've never done before.

Practice this several times until you understand the process, and know what kinds of things might go wrong (and how to recover from them). You can also practice different kinds of migration - e.g. from mdadm RAID-1 to mdadm RAID-5, and mdadm RAID-1 to LVM RAID-5 or to ZFS to help you decide which will work best for you. Make it easier on yourself by making a copy of the two RAID-1 virtual disks after you finish setting it up for the first time....that way you can quickly and easily restore the VM to its "starting condition" for another practice run.


Another alternative is to use ZFS RAIDZ instead of RAID-5, which would give roughly the same storage capacity but ZFS supports transparent compression so highly compressible files like text (e.g. config files, log files, source code, markdown or TeX files, HTML, XML, json, etc), would use less disk space. Already compressed files (e.g. PDFs, and most video, image, audio formats) would use the same amount of space as before. There are other benefits to ZFS too, include error detection and correction, snapshots, and the ability to make separate datasets for different kinds of data with different attributes (e.g. compression type, encryption, record size, etc).

You'd have to install the ZFS dkms module and the ZFS utilities. That's pretty easy and straight-forward on Debian and Ubuntu, but I've never done it on Centos, so have no idea how difficult that would be. I'm sure that there will be blog posts and tutorials about it.

0

Was only going to comment, but I have to post as an answer because I lack the reputation to comment.

Adding another disk to raid1 will add another mirror with no more space. Raid5/6 are falling out of favor [citation needed] due to the computation for calculating parity. Depending on the application, of course.

More similar to a raid1 but can grow in size with more disks is a raid10 (was originally nested raid1/0 (requiring an even number of drives), but raid10 can work with an odd number of drives), it stores copies instead of parity. A 2 drive raid10 is a raid1, so you can change for free:

 mdadm <device> --grow --raid-level=10

Default layout is near 2, which should be sufficient. You get 2 copies of each block distributed across the drives (currently 2, hence still a raid 1).
Then add the new drive:

mdadm <device> --add /dev/new-disk
mdadm <device> --grow --raid-devices=3

Now our raid has more space available on it, but the filesystem is still the original size, so finally:

resize2fs <device>

Not the answer you're looking for? Browse other questions tagged .