3

I have two mdadm arrays on my system:

alex@fileserver:~$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid5 sdc1[0] md0[3] sde1[1]
      7759197184 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

md0 : active raid0 sdd1[1] sdb1[0]
      3907019776 blocks super 1.2 512k chunks

unused devices: <none>

If you look real close, you'll see that md1 uses md0 in its array. (I had two 4 TB disks and two 2 TB disks, and this is how I wanted it working together...)

This works great, except on boot. Even though my mdadm.conf has these lines:

ARRAY /dev/md/0 metadata=1.2 UUID=ba271258:3f18f660:1d5d19aa:2bd0d8e3 name=fileserver:0
ARRAY /dev/md/1 metadata=1.2 UUID=262a6823:3fafb521:75e315ab:f2de4329 name=fileserver:1

it manages to assembly md0 just fine, but md1 is missing md0 in its array, and ends up sitting there inactive, with only sdc1 and sde1 as spares. I'm guessing mdadm tries to assemble its arrays in parallel or something?

If, after boot, I do mdadm --stop /dev/md1 and then mdadm --assemble --scan, the array is correctly put together.

My scenario is probably not very standard, but is there some simple solution to this issue? E.g. some way to tell mdadm.conf that md1 depends on md0? I could always just write a small init script that does the stop/assemble commands for me after mdadm, but I'd rather avoid such hacks if a proper solution exists, especially since I have system services that rely on md1 being present and mounted to work properly.

  • Not sure if these will help: use /dev/md0 instead of /dev/md/0, remove metadata= name= (only UUID is enough), check UUID for typos, try to reverse line order, remember to update initramfs when changing mdadm.conf. If you don't mind completely reworking your setup, make two RAID5 of 2TB each. It should perform better and more reliably than md on md. – frostschutz Sep 2 '17 at 10:56
  • You can also specify DEVICE and for each array entry devices= but I doubt it would help, as long as you don't set these at all it should work. Can you show dmesg log of the boot process and failed assembly attempt? – frostschutz Sep 2 '17 at 11:00
  • There is probably not a simple solution for nested MDs. But there are standard solutions to make sure that your mdadm command is executed (if necessary) before those system services are started. Write a script which checks whether the RAID is already there and creates it if not, and call this script via a unit file (or SysV init script) on which those servies (i.e. their unit file or init script) depend on. – Hauke Laging Sep 2 '17 at 14:22
  • It could just be sheer luck, but I just upgraded my system from Ubuntu 14.04 to 16.04 (yeah I know, I'm a little late...) and after the reboot for that, everything worked as it was supposed to. I'm reluctant to reboot again to test my luck, so for now, I'll just stick with this. But thanks, all of you, for your input and suggestions. – Alex Sep 2 '17 at 14:25
0

This is an Ubuntu bug that showed up in 12.04 (kernel ~3.2.0.40) and existed until fixed in 16.04. The same problem didn't appear in the Debian builds.

From conversation in the Ubuntu bug tracker:

(user Tarantoga):

It looks like [Debian] Wheezy disables udev for md and uses an rc-script for assembling at boot; therefore it will not automatically assemble arrays after boot. Trusty assembles real devices during and after boot, but seems to ignore virtual devices like /dev/md0.

Tarantoga found a simple solution by adding a single-line udev rules file:

/etc/udev/rules.d/85-mdadm.rules:

SUBSYSTEM=="block", ACTION=="add|change", ENV{ID_FS_TYPE}=="linux_raid_member", DEVPATH=="*/virtual/*", RUN+="/sbin/mdadm --incremental $tempnode"

This solution just worked for me. I have a mirror of two stripes and only the stripes would assemble at boot time. Putting this rule in place caused everything to assemble properly.

(And, yes, I have a specific use case where a mirror of stripes is better than a stripe of mirrors :-) )

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.