0

I had a working raid 5 array that had a whole debian install on it (I set it up during a debian install). I'd been using it a couple of days, all was fine, but since I created it during debian install, the raid array itself had 2 partitions on it, one for GPT and the big ext4 with the OS and everything on it.

After a lot of reading, it seems having your OS on the mdadm managed raid array is a bad idea. Since if there is a drive failure, you will have no OS to fix it. So I unplugged all my drives, grabbed an old one from the closet, and installed debian on that as the sole disk in the machine.

Once installed, I plugged my raid drives back in. Ran mdadm --assemble --scan and setup my /etc/mdadm/mdadm.conf so it assembles on boot. All was pretty good, I could mount -t ext4 /dev/md0p2 /mnt and see all my data. Happy days.

So I added it to /etc/fstab and rebooted.

But after reboot, /dev/md0p2 (and md0p1) don't exist anymore. Only /dev/md0 is there. Not sure where the 2 partitions have gone!

Completely weird that I can re-assemble the array with no worries, it doesn't complain one bit. But the 2 partitions that were there are gone. I'm not even sure what /dev/md0p1 was for. It concerned me when I first set it up, fdisk told me it was of type GPT.

Some command outputs to help:

$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sdf1[0] sdb1[4] sdc1[5] sdd1[6] sde1[7] sdi1[3] sdh1[2] sdg1[1]
      5860147200 blocks super 1.2 level 5, 512k chunk, algorithm 2 [8/8] [UUUUUUUU]
      bitmap: 0/15 pages [0KB], 65536KB chunk

unused devices: <none>


$ sudo mdadm -D /dev/md0
/dev/md0:
        Version : 1.2
  Creation Time : Fri May 23 14:05:16 2014
     Raid Level : raid5
     Array Size : 5860147200 (5588.67 GiB 6000.79 GB)
  Used Dev Size : 1953382400 (1862.89 GiB 2000.26 GB)
   Raid Devices : 8
  Total Devices : 8
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Sun May 25 18:15:26 2014
          State : active
 Active Devices : 8
Working Devices : 8
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 512K

           Name : doc:0  (local to host doc)
           UUID : 11d6a8a7:d05a8c58:41fab15a:0b177f69
         Events : 25704

    Number   Major   Minor   RaidDevice State
       0       8       81        0      active sync   /dev/sdf1
       1       8       97        1      active sync   /dev/sdg1
       2       8      113        2      active sync   /dev/sdh1
       3       8      129        3      active sync   /dev/sdi1
       7       8       65        4      active sync   /dev/sde1
       6       8       49        5      active sync   /dev/sdd1
       5       8       33        6      active sync   /dev/sdc1
       4       8       17        7      active sync   /dev/sdb1

UPDATE: I just realise the order of those disks in the array aren't what they used to be. It should start at /dev/sdb1. I've done a mdadm --create --level=5 --num-devices=8 /dev/md0 /dev/sd[b-i]1 to re-create the array. This has inadvertantly put /dev/sdi1 into recovery. Yuck.

Now that I think back, I'm not entirely sure what the order should be. It's either alphabetical, or it's e,d,c,b,f,g,h,i which is odd. First 4 in reverse alphabetical, next for in normal alphabetical. I may have moved sata cables around when I plugged them back in, not sure.

  • 4
    Never ever do a mdadm --create to fix things. drive letters may change, drive order in mdadm metadata does not. Furthermore, mdadm defaults like offsets changed over time, so chances are high that you won't see your old data with the newly created array, even if you got the drive order right. – frostschutz May 25 '14 at 14:29
  • 2
    The metadata you posted above says Fri May 23 14:05:16 2014 for creation time, so unless you actually created this thing yesterday, you lost your partition data and such to a botched mdadm --create. Additional or complete data loss through metadata/sync/recovery is possible. – frostschutz May 25 '14 at 14:32
  • 1
    I created it on Friday, those times are correct. It's now Sunday here in AU. I've found many stackexchange responses to suggest mdadm --create over the top is perfectly fine providing the metadata version is the same and you don't write anything to the new array (e.g. create a new partition). – tigris May 25 '14 at 14:38
  • 1
    When you --create, you must know all the variables the raid was originally created with, order, offset, chunk size, raid level, ... fixing things with --create is a bad idea because the ways you can fail with it are not obvious and a bad sync can kill everything. – frostschutz May 25 '14 at 14:40
  • 2
    You started with needing to add --auto=mdp to a mdadm line somewhere (to get md0pX instead of md0), and now you instead get to figure out your array geometry. Note the 'infinite wisdom' in the Serverfault question is sarcastic; the person asking that question realizes it was a bad idea. You have the Serverfault answer about figuring out the geometry to use now... And do note its bold line up top about a backup. In the future, DO NOT USE --create TO REPAIR YOUR ARRAY. – derobert May 25 '14 at 15:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.