Error setting up Raid 1 with mdadm at RaspberryPi 2

Question

I was setting up a RAID 1 with mdadm at my RaspberryPi with two drives (both 2 TB and formatted with exFAT, both with independent power supplies), but I ran into an error.

Unfortunately I am not an expert at Linux und commands.

Here is what I've done:

• installed mdadm with apt-get install mdadm
• found both devices with sudo fdisk -l (as /dev/sda and /dev/sdb)
• set up RAID 1 to /dev/md0 with sudo mdadm -Cv /dev/md0 -l1 -n2 /dev/sd[ab]1
• formatted /dev/md0 with sudo mkfs /dev/md0 -t ext4
• mounted /dev/md0 to /media/nas with sudo mount /dev/md0 /media/nas
• edited /etc/fstab with /dev/md0 /media/nas ext4 4 0 0
• added AUTOSTART=true to /etc/default/mdadm
• enabled samba in /etc/samba/smb.conf

Everything went well and I could upload my files to /media/nas with WinSCP.

Now my Problem: The next day my uploads failed with error code 4 (no further error text). When I run sudo fdisk -l I found both devices sda and sdb. Also there is /dev/md0 with size of 2000.3 GB, but there is also /dev/md127 with size 2000.3 GB.

When I run sudo mdadm --detail /dev/md0 I get the following:

/dev/md0: Version : 1.2 Creation Time : Fri Jan 5 12:23:50 2018 Raid Level : raid1 Array Size : 1953371712 (1862.88 GiB 2000.25 GB) Used Dev Size : 1953371712 (1862.88 GiB 2000.25 GB) Raid Devices : 2 Total Devices : 1 Persistence : Superblock is persistent

Update Time : Sun Jan  7 14:37:23 2018
      State : clean, degraded

Active Devices : 1 Working Devices : 1 Failed Devices : 0 Spare Devices : 0

       Name : raspberrypi:0  (local to host raspberrypi)
       UUID : 926bc124:2945e335:1e79ab6c:06b12095
     Events : 21

Number   Major   Minor   RaidDevice State
   0       0        0        0      removed
   1       8        1        1      active sync   /dev/sda1

The output of sudo mdadm --detail /dev/md127 is:

/dev/md127: Version : 1.2 Creation Time : Fri Jan 5 12:23:50 2018 Raid Level : raid1 Array Size : 1953371712 (1862.88 GiB 2000.25 GB) Used Dev Size : 1953371712 (1862.88 GiB 2000.25 GB) Raid Devices : 2 Total Devices : 1 Persistence : Superblock is persistent

Update Time : Sun Jan  7 14:38:47 2018
      State : clean, degraded

Active Devices : 1 Working Devices : 1 Failed Devices : 0 Spare Devices : 0

       Name : raspberrypi:0  (local to host raspberrypi)
       UUID : 926bc124:2945e335:1e79ab6c:06b12095
     Events : 27

Number   Major   Minor   RaidDevice State
   0       8       17        0      active sync   /dev/sdb1
   1       0        0        1      removed

When I try to set up the RAID again with sudo mdadm -Cv /dev/md0 -l1 -n2 /dev/sd[ab]1, I get the error:

mdadm: super1.x cannot open /dev/sda1: Device or resource busy mdadm: failed container membership check mdadm: cannot open /dev/sda1: Device or resource busy

How can I set the RAID up again, where does md127 come from and what causes this error?

(rebooting doesn't do anything)

Thanks in advance!

Answer 1

What happened is your RAID array fell apart. Based on the fragmentary output you've provided, I suspect /dev/sdb1 suffered a transient fault (most likely a hiccup of the Pi's USB system) and was marked as failed. When it came back online, the Linux md subsystem saw it as a new RAID volume not belonging to any known array, and set it up as /dev/md127.

When you ran sudo mdadm -Cv /dev/md0 -l1 -n2 /dev/sd[ab]1, you got lucky: it failed. Running mdadm --create is almost never the solution to a RAID problem. It's far more likely to destroy your data than recover it.

At this point, your best option is probably to destroy the /dev/md127 array and re-add /dev/sdb1 to /dev/md0.

1. Make sure that /dev/md0 really is the live copy of your data. Inspect the output of mount to verify that it's mounted on /media/nas, and run ls /media/nas to make sure your data is there.
2. Remove /dev/sdb1 from /dev/md127: mdadm /dev/md127 --fail /dev/sdb1, followed by mdadm /dev/md127 --remove /dev/sdb1.
3. Make /dev/sdb1 not look like a RAID member anymore: wipefs -a /dev/sdb1.
4. Put it back into /dev/md0: mdadm /dev/md0 --add /dev/sdb1.
5. Let the computer rebuild the array, copying everything from /dev/sda1.

To prevent this from happening in the future, set up entries describing your array in /etc/mdadm/mdadm.conf. If a drive temporarily falls out of the array, mdadm will see it in the configuration file, and will sit there waiting for you to perform step (4).

If your setup has a habit of producing temporary failures, consider adding a write-intent bitmap to your array: mdadm --grow /dev/md0 --bitmap=internal. This will slow down writes somewhat, as the bitmap needs to be updated, but will greatly speed up recovery, as only the changes need to be copied from one disk to the other. If you add a write-intent bitmap, you put temporarily-failed disks back into the array with --re-add rather than --add.