20

Can someone please explain to me, what the difference is between creating mdadm array using partitions or the whole disks directly? Supposing I intend to use the whole drives. Note, that they are with GPT.

Imagine RAID6 created in two ways, either:

mdadm --create /dev/md0 --level=6 --raid-devices=4 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1

or:

mdadm --create /dev/md0 --level=6 --raid-devices=4 /dev/sda /dev/sdb /dev/sdc /dev/sdd

Is there a difference?

EDIT1: I mean in for example reliability or manageability or recovery operations on such arrays, etc.

EDIT2: I do use different manufacturer's drives. But that is not the question really. They all have the exact same size, I do get the point, that there may be manufacturers making drives slightly different sizes. But for the sake of this question, let's assume they are the very same size.

EDIT3: Sort of disclaimer: So far I am always partitioning the drives.

  • 2
    I've always been curious about this but never asked anyone. One difference I see is using the partitioned disks (sda1, etc) you get a little bit of less disk space (some KB) as GPT reserves some space for partition information. – GMaster Oct 31 '16 at 16:11
20
+50

The most important difference is that it allows you to increase the flexibility for disk replacement. It is better detailed below along with a number of other recommendations.

One should consider to use a partition instead of the entire disk. This should be under the general recommendations for setting up an array and may certainly spare you some headaches in the future when further disk replacements get necessary.

The most important arguments is:

Disks from different manufacturers (or even different models of the "same" capacity from the same manufacturer) don't necessarily have the exact same disk size and, even the smallest size difference, will prevent you from replacing a failed disk with a newer one if the second is smaller than the first. Partitioning allows you to workaround this;

Side note on why to use different manufacturers disks: Disks will fail, this is not a matter of a "if" but a "when". Disks of the same manufacturer and the same model have similar properties, and so, higher chances of failing together under the same conditions and time of use. The suggestion so is to use disks from different manufacturers, different models and, in special, that do not belong to the same batch (consider buying from different stores if you are buying disks of the same manufacturer and model). This is not uncommon that a second disk fail happen during a resotre after a disk replacement when disks of the same batch are used. You certainly don't want this to happen to you.

So the recommendations:

1) Partition the disks that will be used with a slightly smaller capacity than the overall disk space (e.g, I have a RAID5 array of 2TB disks and I intentionally partitioned them wasting about 100MB in each). Then, use /dev/sd?1 of each one for composing the array - This will add a safety margin in case a new replacing disk has less space than the original ones used to assemble the array when it was created;

2) Use disks from different manufacturers;

3) Use disks of different models if different manufacturers are not an option for you;

4) Use disks from different batches;

5) Proactively replace disks before they fail and not all at the same time. This may be a little paranoid and really depends on the criticity of the data you have. I use to have disks that have 6 months differences in age from each other;

6) Make regular backups (always, regardless if you use an array or not). Raid doesn't serve the same purpose of backups. Arrays assure you high availability, Backups allow you to restore lost files (including the ones that get accidentally deleted or are damaged by viruses, some examples of something that using arrays will not protect you from).

OBS: Except for all the non-neglectable rational above, there aren't much further technical differences between using /dev/sd? vs /dev/sd?#.

Good luck

  • That's mostly not related... – user1133275 Nov 15 '16 at 23:48
  • Yet, I guess @Kalavan warning refers to use multiple partitions in the same disk as components of an array. That certainly wouldn't make much sense as it removes the redundancy. We know, though, that this was not the intention of your question, His warning makes sense anyway. – Marcelo Nov 16 '16 at 23:16
  • If managing the partitions manually seems to be a hassle, one should use sfdisk -d src | sfdisk tgtor sgdisk --backup && sgdisk --load-backup – andyn Jan 3 at 10:15
5

Another important argument is that some mainboards may delete your RAID superblocks if you use whole disk devices and are not super careful with wiping them when adding disks to a RAID array that once were GPT devices.

I learned this the hard way, when my ASRock mainboard's UEFI rendered my RAID unusable:

To protect yourself against this, always run sgdisk --zap on a formerly GPT-formatted disk that you want to use for a whole-device RAID.

wipefs -a may also work to wipe all kinds of leftover partitioning info (not only GPT) before adding the device to the disk.


That said, just using partitions seems like a best practice to make this less likely to happen.

I never knew what the real arguments were for this best practice (beyond @Marcelo's answer explaining that it can deal with disks of slightly different sizes). Now I know.

By the way, if this happens to you, your data is not lost. You can most likely just sgdisk --zap the device, and then recreate the RAID with e.g. mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdc /dev/sdd (mdadm will tell you that it already detects past data, and asks you if you want to continue re-using that data). I tried this mutliple times and it worked, but I still recommend taking a backup before you do it.

2

There are couple of posts citing problem with using the whole device as a RAID member instead of partition, one here. I can confirm that to be the case.

So, if you create RAID using the whole device (say, /dev/sda, /dev/sdb, ... instead of on /dev/sda1, /dev/sdb1, ...) the RAID device doesn't reassemble after a reboot even if you have configuration saved under mdadm.conf. There is also a possibility of mainboard firmware overwriting or removing the RAID superblock. In my setup, running mdadm --assemble --scan --verbose doesn't get me back my RAID setup and it seems to be the case of RAID superblock not being available.

  • I tested RAID1 setup using 2x10GB drives inside VirtualBox. RAID was created on the whole device and it is reassembling fine after reboot. The problem could be applicable only to physical devices. – Lungten Jan 1 at 5:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.