1

I currently have 2 3TB HDDs, one that is always almost full, and another with ~200GB free. I would like to purchase an extra 3TB drive and setup a RAID 5 array, but I am concerned about losing the existing data.

I have found that mdadm will be used to create the array, with a command similar to mdadm --create --verbose /dev/md0 --level=5 --raid-devices=2 /dev/sdb2 /dev/sdc2 --spare-devices=1 /dev/sdd2, where /dev/sdb2 and /dev/sdc2 are my existing drives (that have data) and /dev/sdd2 is a new 3TB drive with no data on it. Will this cause me to lose the data on /dev/sdb2 and /dev/sdc2?

My other idea was to somehow create a 2x3TB RAID 5 array without a spare device, where one of the drives in the array is empty and the other has data. Then I could copy my files over from the existing drive to the new (6TB) array, wipe the now redundant drive, and then add it as the spare drive for the array. Although I doubt this would work?

If neither of the above options will work, is there another way to create a RAID 5 array with 2 drives that already have data and 1 that is empty? What about if I were add 2 new empty drives at once, would that open up new options?

I am using Ubuntu Server 16.04.2 with mdadm version 3.3.

3

You have two 3TB disks storing 6TB data. You want to install one new 3TB disk.

This will allow you to convert the three 3TB disks into a RAID5 array storing 6TB data. However, the process is rather fiddly and the opportunities for losing data somewhere along the route are fairly large.

Steps to achieve the requirement

Let's declare the disks as sda (contains data), sdb (contains data), sdc (new).

  1. If at all possible take a backup of all your data, even if you have to borrow a disk from a friend for a few days
  2. Create a RAID1 array on the new disk sdc. It should have two members of which one is missing
  3. Create a filesystem on this RAID1 array
  4. Copy the data from sdb to the new RAID1 array
  5. Verify that you have correctly copied the data
  6. Add sdb to the RAID1 array
  7. Wait for the synchronisation to complete
  8. Reboot
  9. Grow the RAID1 array to RAID5. It should have three members of which one is missing. to do this you will need 128K of temporary space on an additional disk. You might want to use a USB thumbstick for this. Do not use a RAM disk.
  10. Copy the data from sda to the new RAID5 array
  11. Verify that you have correctly copied the data
  12. Add sda to the RAID5 array
  13. Reboot

Worked example

Here is a worked example using three files as disk images.

# Prepare the demonstration
#
dd if=/dev/zero bs=1M count=100 of=sda.img
dd if=/dev/zero bs=1M count=100 of=sdb.img
( echo n; echo p; echo 1; echo; echo; echo w ) | fdisk sda.img  # One primary partition
( echo n; echo p; echo 1; echo; echo; echo w ) | fdisk sdb.img  # One primary partition
losetup --show --find --partscan sda.img
losetup --show --find --partscan sdb.img
mkfs -t ext4 -L sda /dev/loop0p1
mkfs -t ext4 -L sdb /dev/loop1p1
mkdir -p /mnt/sda1 /mnt/sdb1
mount /dev/loop0p1 /mnt/sda1
mount /dev/loop1p1 /mnt/sdb1
cp -a /usr/local/man/. /mnt/sda1/u.l.man/
mkdir /mnt/sdb1/u.l.etc
cp -a /usr/local/bin/. /mnt/sdb1/u.l.bin/
df -h | grep mnt
umount /mnt/sda1
umount /mnt/sdb1


# Create the third disk
#
dd if=/dev/zero bs=1M count=100 of=sdc.img
( echo n; echo p; echo 1; echo; echo; echo w ) | fdisk sdc.img  # One primary partition
losetup --show --find --partscan sdc.img


# Create the RAID1 array and its filesystem
#
mdadm --create /dev/md1 --level=1 --raid-devices=2 --metadata=default /dev/loop2p1 missing
mkfs -t ext4 -L md1 /dev/md1
mkdir -p /mnt/md1


# Copy the data from sdb to md1
#
mount /dev/loop1p1 /mnt/sdb1
mount /dev/md1 /mnt/md1
cp -a /mnt/sdb1/. /mnt/md1/
umount /mnt/sdb1
umount /mnt/md1


# Complete the RAID1 array
#
mdadm --manage /dev/md1 --add /dev/loop1p1


# Grow the RAID1 array to RAID5
#
mdadm --grow /dev/md1 --level=5 --raid-devices=3 --backup-file=/root/workarea.dat --force
e2fsck -f /dev/md1
resize2fs /dev/md1


# Copy the data from sda to md1
#
mount /dev/loop0p1 /mnt/sda1
mount /dev/md1 /mnt/md1
cp -a /mnt/sda1/. /mnt/md1/
umount /mnt/sda1
umount /mnt/md1


# Add the remaning disk to the RAID5 array
#
mdadm --manage /dev/md1 --add /dev/loop0p1


# All done
#
mdadm --stop /dev/md1
losetup -d /dev/loop0
losetup -d /dev/loop1
losetup -d /dev/loop2
rm sda.img sdb.img sdc.img

You really should ensure you understand the worked example BEFORE touching the live data on your disks. Needless to say, it's your responsibility and I really would recommend a backup before you change your live system.

  • Wow, thank you very much! Currently my drives are formatted using NTFS, will that be a problem given that the above commands use ext4? Is it more likely that one of the drives would lose data over the other? One of them has data that I am more willing to lose, if I were to lose any data (which of course I do not want). I have ordered a new drive so I shall we doing this tomorrow. Again, thank you! – Joseph Duffy Mar 3 '17 at 23:38
  • @Joseph, I'd suggest you test it out through my example but with NTFS. It's your data. – roaima Mar 4 '17 at 0:41
  • I have bought 2 new empty drives, which I hope will reduce the risk. I also think it's a good idea to move over to ext4 so my plan is create the RAID 1 array with the 2 new empty drives, transfer data over, then continue the steps as described, which should get me a 3x3TB RAID 5 array with ~3TB of data (6TB usable). Then I'll transfer the remaining ~3TB, wipe the final drive, and add it to the array, leaving me with 4x3TB RAID 5 (9TB usable, 6TB used). – Joseph Duffy Mar 4 '17 at 11:16
  • @Joseph if you can put both new drives in straightaway, consider creating a RAID5 array with one missing member. That would be more sense than all this conversion I described. – roaima Mar 4 '17 at 11:30
  • brilliant, the above solution scared me a little so I'm glad an extra drive makes it easier/safer. So I'll be running mdadm --create --verbose /dev/md0 --level=5 --raid-devices=2 /dev/sdb1 /dev/sdc1, transfer my data, wipe old drives, then mdadm --add /dev/md0 /dev/sdd1; mdadm --add /dev/md0 /dev/sde1; mdadm --grow --raid-devices=3 --spare-devices=1 /dev/md0? If so that feels fairly painless 😄 – Joseph Duffy Mar 4 '17 at 11:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.