1

My current idea is to create one software array, class RAID-6, with 4 member drives, using mdadm.

Specifically, the drives would be 1 TB HDDs on SATA in a small server Dell T20.

Operating System is GNU/Linux Debian 8.6 (later upgraded: JessieStretchBuster)

That would make 2 TB of disk space with 2 TB of parity.


I would also like to have it with GPT partition table, for that to work, I am unsure how to proceed specifically supposing I would prefer to do this purely over the terminal.

As I never created a RAID array, could you guide me on how I should proceed?


Notes:

  • This array will serve for the sole data only. No boot or OS on it.

  • I opted for RAID6 due to the purpose of this array. Two drive failures the array must be able to survive. Since I am limited by hardware to 4 drives, there is no alternative to RAID6 that I know of. (However ugly the RAID6 slowdown may seem, it does not matter in this array.)

6

In this answer, let it be clear that all data will be destroyed on all of the array members (drives), so back it up first!


Open terminal and become root (su); if you have sudo enabled, you may also do for example sudo -i; see man sudo for all options):

sudo -i

First, we should erase the drives, if there was any data and filesystems before, that is. Suppose we have 4 members: sdi, sdj, sdk, sdl. For the purpose of having feedback of this process visually, the pv (pipe viewer) I used here:

pv < /dev/zero > /dev/sdi
pv < /dev/zero > /dev/sdj
pv < /dev/zero > /dev/sdk
pv < /dev/zero > /dev/sdl

Alternatively, to just check if there is nothing left behind, you may peek with GParted on all of the drives, and if there is any partition with or without any filesystem, wiping it could be enough, though I myself prefer the above zeroing all of the drives involved, remember to un-mount all partitions before doing so, it could be done similar to these one-liners:

umount /dev/sdi?; wipefs --all --force /dev/sdi?; wipefs --all --force /dev/sdi
umount /dev/sdj?; wipefs --all --force /dev/sdj?; wipefs --all --force /dev/sdj
umount /dev/sdk?; wipefs --all --force /dev/sdk?; wipefs --all --force /dev/sdk
umount /dev/sdl?; wipefs --all --force /dev/sdl?; wipefs --all --force /dev/sdl

Then, we initialize all drives with GUID partition table (GPT), and we need to partition all of the drives, but don't do this with GParted, because it would create a filesystem in the process, which we don't want, use gdisk instead:

gdisk /dev/sdi
gdisk /dev/sdj
gdisk /dev/sdk
gdisk /dev/sdl

In all cases use the following:

o Enter for new empty GUID partition table (GPT)
y Enter to confirm your decision
n Enter for new partition
Enter for default of first partition
Enter for default of the first sector
Enter for default of the last sector
fd00 Enter for Linux RAID type
w Enter to write changes
y Enter to confirm your decision


You can examine the drives now:

mdadm --examine /dev/sdi /dev/sdj /dev/sdk /dev/sdl

It should say:

(type ee)

If it does, we now examine the partitions:

mdadm --examine /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1

It should say:

No md superblock detected

If it does, we can create the RAID6 array:

mdadm --create /dev/md0 --level=6 --raid-devices=4 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1

We shall wait until the array is fully created, this process we may watch with:

watch -n 1 cat /proc/mdstat

After the creation of the array, we should look at its detail:

mdadm --detail /dev/md0

It should say:

          State : clean
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0

Now we create a filesystem on the array, if you use ext4, the below hidden command is better to be avoided, because of ext4lazyinit would take noticeable amount of time in case of a large array, hence the name, "lazyinit", therefore I recommend you to avoid this one:

mkfs.ext4 /dev/md0

Instead, you should force a full instant initialization (with 0% reserved for root as it is a data array):

mkfs.ext4 -m 0 -E lazy_itable_init=0,lazy_journal_init=0 /dev/md0

By specifying these options, the inodes and the journal will be initialized immediately during creation, useful for larger arrays.

If you chose to take a shortcut and created the ext4 filesystem with the "better avoided command", note that ext4lazyinit will take noticeable amount of time to initialize all of the inodes, you may watch it until it is done, e.g. with:

iotop

Either way you choose to make the file system initialization, you should mount it after it has finished its initialization:


We now create some directory for this RAID6 array:

mkdir -p /mnt/raid6

And simply mount it:

mount /dev/md0 /mnt/raid6

Since we are essentially done, we may use GParted again to quickly check if it shows linux-raid filesystem, together with the raid flag on all of the drives.

If it does, we properly created the RAID6 array with GPT partitions and can now copy files on it.

See what UUID the md filesystem has:

blkid /dev/md0

Copy the UUID to clipboard.

Now we need to edit fstab, with your favorite text editor, I used nano, though sudoedit might better be used:

nano /etc/fstab

And add add an entry to it:

UUID=<the UUID you have in the clipboard>    /mnt/raid6    ext4    defaults 0 0

I myself do not recommend using defaults set of flags, I merely wanted the line not to be overly complex.

Here is what mount flags I use on a UPS backed-up data RAID (instead of defaults): nofail,nosuid,nodev,noexec,rw,auto,nouser,async,noatime,data=writeback,errors=remount-ro


You may check if it is correct after you save the changes:

mount -av | grep raid6

It should say:

already mounted

If it does, we save the array configuration; in case you don't have any md device yet created, you can do simply:

mdadm --detail --scan >> /etc/mdadm/mdadm.conf

In case there are arrays already existent, just run the previous command without redirection to the conf file:

mdadm --detail --scan

and add the new array to the conf file manually.

In the end, don't forget to update your initramfs:

update-initramfs -u -k all

Check if you did everything according to plan, and if so, you may restart:

reboot --reboot
| improve this answer | |
0

If you create the RAID arrays over 4 raw block devices instead of 2 x 4 partitions, that means all RAID recovery operations may necessarily operate on the whole devices, and vice versa.

So for example if you expect the disks to eventually start developing I/O errors on the latter half of them, with the arrays over partitions that would mean that only one of the arrays would notice, while the other would continue untouched, at least until the damage spreads into its half. This might provide you with some temporary flexibility, or an absence of I/O slowdown.

On the other hand, as soon as you start intervening, you have to take out the whole physical disk for replacement, so you necessarily have to degrade both arrays at some point. Also, with SSDs, it's apparently become more common that the whole disk fails, so both arrays might be affected by such events anyway.

There is nothing to be said about the details of the process that can't be found in typical partitioning and mdadm documentation.

| improve this answer | |
  • @VlastimilBurian please ask specific questions, rather than generic "show me how to do everything" questions. Cf. meta.stackexchange.com/questions/18242/… – Josip Rodin Oct 23 '16 at 9:03
  • You can replace disks without degrade using mdadm --replace. You just need a free sata port for 5 disks in the system for the duration of the resync. – frostschutz Nov 1 '16 at 16:49
  • @frostschutz surely that assumes the array didn't get into a degraded state automatically by way of unrecoverable errors? – Josip Rodin Nov 2 '16 at 15:58
  • That's what you described... multiple partitions per disk, only one degraded. Or did I misunderstand you? In that situation you don't "necessarily have to degrade both". You can --replace the not-degraded one. Same if the errors turn up in SMART selftest but RAID doesn't know about it yet. – frostschutz Nov 2 '16 at 16:11
  • @frostschutz that's true, but the original question talked of a small server, and specifically about four disks, so I had assumed that those are constraints, not just arbitrary variables. – Josip Rodin Nov 2 '16 at 20:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.