Hot answers tagged

48

Using mdadm 3.3+ Since mdadm 3.3 (released 2013, Sep 3), if you have a 3.2+ kernel, you can proceed as follows: # mdadm /dev/md0 --add /dev/sdc1 # mdadm /dev/md0 --replace /dev/sdd1 --with /dev/sdc1 sdd1 is the device you want to replace, sdc1 is the preferred device to do so and must be declared as a spare on your array. The --with option is optional, if ...


38

The most important difference is that it allows you to increase the flexibility for disk replacement. It is better detailed below along with a number of other recommendations. One should consider to use a partition instead of the entire disk. This should be under the general recommendations for setting up an array and may certainly spare you some ...


30

When an array is initially assembled, it is placed in "auto-read-only" mode. I quickly tested, with my kernel (3.10.x) and mdadm (3.3), this doesn't happen on create—but you must be running different versions. However, auto-read-only isn't an error, nor is it anything to worry about. The basic idea behind it is to make --assemble (and, apparently now, even -...


27

The bitmap line in /proc/mdstat indicates how much memory is being used to cache the write-intent bitmap. Basically, in RAID setups with redundant devices, mdadm can use a "bitmap" to keep track of which blocks may be out of sync (because they've been written to). When a block is written to the mdadm device, it is flagged in the bitmap, and then written to ...


26

The first one reports the UUID of the ext4 filesystem on the md block device. It helps the system identify the file system uniquely among the filesystems available on the system. That is stored in the structure of the filesystem, that is in the data stored on the md device. The second one is the UUID of the RAID device. It helps the md subsystem identify ...


18

If you know the array UUID, then mdadm --assemble /dev/md0 --uuid <uuid> (note the slight difference in parameter order) will do what you want: scan all unused volumes for ones that have md metadata for the given UUID. Other options: mdadm --assemble /dev/md0 --name <name> (does the same thing as --uuid, but with an array name instead of a UUID....


16

It's because the device nodes no longer exist on your system (probably udev removed them when the drive died). You should be able to remove them by using the keyword failed or detached instead: mdadm -r /dev/md0 failed # all failed devices mdadm -r /dev/md0 detached # failed ones that aren't in /dev anymore If your version of mdadm is too old to do ...


15

You need to assemble the (degraded) RAID array, using something like: mdadm --assemble --readonly /dev/md0 /dev/sdb2 Of course, pick a number besides md0 if that's already in use. Then you can mount /dev/md0 (or, if it is actually LVM, etc., continue down the chain). You can, in the case of RAID1, also do this using loopback devices & an offset, but ...


14

the Debian and Ubuntu 'mdadm' package contains the file /etc/cron.d/mdadm which in turns the first sunday of each month will run the command /usr/share/mdadm/checkarray --cron --all --idle --quiet that will check all your arrays for consistency (unless you set AUTOCHECK to false in /etc/default/mdadm ). A report will be sent to the 'root' user (make ...


13

There is nothing wrong with --create - if you know what you are doing. The only problem is: You don't know. When you create a RAID, the command is usually something short, like: mdadm --create /dev/md42 --level=5 --raid-devices=3 /dev/sdx1 /dev/sdy1 /dev/sdz1 Dead simple, right? Except it isn't, really. RAID has a lot more variables. There's a data ...


12

Two issues spring to mind You've got duplicate array definitions in mdadm.conf. Replace (or comment out) the block of three ARRAY lines following # definitions of existing MD arrays so that each array is declared only by your most recent scan. A typical scenario for RAID arrays that fail to build on boot is that either they have not been updated in the ...


12

That's a bad idea because you're deliberately degrading your RAID and Resyncs might fail unexpectedly. It's better to hook the new disk up to the system (so you have n+1 disks) and then use mdadm --replace to sync it in. That way the RAID never degrades in between. You don't have to fail / remove drives to find out which is which. You can see a device's ...


12

Another important argument is that some mainboards may delete your RAID superblocks if you use whole disk devices and are not super careful with wiping them when adding disks to a RAID array that once were GPT devices. I learned this the hard way, when my ASRock mainboard's UEFI rendered my RAID unusable: http://forum.asrock.com/forum_posts.asp?TID=10174 ...


11

Your arrays are not properly started. Remove them from your running config with this: mdadm --stop /dev/md12[567] Now try using the autoscan and assemble feature. mdadm --assemble --scan Assuming that works, save your config (assuming Debian derivative) with (and this will overwrite your config so we make a backup first): mv /etc/mdadm/mdadm.conf /etc/...


11

This would seem to be indicating that the syncing between the 2 members of the RAID are not staying in sync with each other. 1. Investigate logs I'd investigate your dmesg logs and see if there are any messages stating that either of the physical HDDs that make up this array are having hardware failures. 2. Check mdadm You can also consult mdadm using ...


11

Check your cron files, many distros do a scheduled resync/re-check once a week. On CentOS 7.1 it's in /etc/cron.d/raid-check # Run system wide raid-check once a week on Sunday at 1am by default 0 1 * * Sun root /usr/sbin/raid-check To configure the behaviour edit /etc/sysconfig/raid-check


11

The cause was an erroneous spares=1 option in the mdadm.conf: # definitions of existing MD arrays ARRAY /dev/md0 UUID=621d5f15:cce75825:60273c48:78a7dac7 spares=1 I'm not sure how this ended up there, but I suppose it happened when a device failed and was replaced. Removing the spares=1 option or just recreating the mdadm.conf from scratch fixes the ...


11

EFI knows how to access FAT and FAT32 filesystems. This is why your EFI boot partition has to be FAT or FAT32 formatted. EFI however does not know how to read a software RAID 1 partition, even if it is formatted using FAT32. There is a pretty simple away around this, at least using Arch Linux. When installing the system, you set the boot partition up as ...


10

The dracut documentation implies that any md raid arrays should be automatically assembled, and that the rd.md.uuid parameter should only be used if you only want certain arrays assembled as part of the boot process. It seems that in reality, the arrays are not assembled automatically, and are in fact only assembled when the rd.md.uuid parameter is set (for ...


10

The UUID seen in mdadm.conf are related to the MD drivers. The UUID used in fstab are related to filesytems. What you need are the filesystem UUID numbers. You can get them with a command line sudo dumpe2fs /dev/md0 | grep UUID So in my case: $ grep md/0 /etc/mdadm/mdadm.conf ARRAY /dev/md/0 metadata=1.2 ...


10

In this answer, let it be clear that all data will be destroyed on all of the array members (drives), so back it up first! Open terminal and become root (su); if you have sudo enabled, you may also do for example sudo -i; see man sudo for all options): sudo -i First, we should erase the drives, if there was any data and filesystems before, that is. Suppose ...


9

I managed to assemble my raid in the end. This is how you do it: mdadm --assemble --update=devicesize /dev/md2


9

All you should have done was your step one mdadm --manage /dev/md0 --fail /dev/sdc At this point your RAID 5 array is running in degraded mode and you can replace the disk with a new one. Unfortunately it looks like you have truncated your array's effective size from 2TB to 1TB, destroying the second half of your filesystem. Fortunately you say you have ...


8

fdisk is the wrong tool for disks >2TB. Use parted or gdisk instead. It appears that /dev/sdc1 and /dev/sdd1 are 2TB partitions, so that's what limits your array size. For the other disks, they have GPT so I assume they are 3TB already, but you should check. Basically you have to stop the array, enlarge each partition to 3TB (without changing the starting ...


8

This won’t explain why your array ended up in read-only mode, but mdadm --readwrite /dev/md0 should return it to normal. In your case it doesn’t, and the reason isn’t entirely obvious: if constituent devices are themselves read-only, the RAID array is read-only (which is what matches the behaviour you’re seeing, and the code-paths used when you try to re-...


8

The --write-mostly, --write-behind is handled by the md driver internally. md keeps metadata, like the write-intent bitmap (which is mandatory for the write-behind feature) that basically logs which data has been written yet vs. which data is still missing. This is necessary in case there is a power loss event, when the data hasn't reached the write-mostly ...


8

Unmounting (filesystems) is not sufficient. You'd have to stop the array then re-assemble it afterwards: mdadm --stop /dev/md0 # re-arrange / hotplug drives mdadm --stop /dev/md0 # (*) mdadm --assemble /dev/md0 It makes sense to check journalctl / dmesg, and/or cat /proc/partitions / lsblk, to make sure the drives got re-detected fine before attempting to ...


7

In this scenario, each disk would claim that the other disk failed. The result depends on how exactly you're assembling disks, but essentially, it will go with one disk and ignore the other; or it might assemble the other as a separate raid, which gives you a split brain. I've done an experiment with loop devices, where I first changed loop1 and then ...


7

What I ended up doing was using mknod like @derobert suggested to create the devices that mdadm was looking for. I tried the major/minor numbers mdadm was telling me it couldn't find with the different drive letters I was trying to remove until it worked. mknod /dev/sde1 b 8 17 Then I had to use the --force option to get it remove the component. mdadm /...


7

Most init systems probably won't expect this situation so they won't be prepared to first assemble LVM, then MD, then LVM again. If you put LVM below MD you have to be very careful. MD is supposed to be used with physical disks, but with LVs as RAID members you could easily end up with two members sharing the same disks. That is if using MD as a standalone ...


Only top voted, non community-wiki answers of a minimum length are eligible