Hot answers tagged

56

The point of RAID with redundancy is that it will keep going as long as it can, but obviously it will detect errors that put it into a degraded mode, such as a failing disk. You can show the current status of an array with mdadm -D: # mdadm -D /dev/md0 <snip> 0 8 5 0 active sync /dev/sda5 1 8 23 ...


36

Using mdadm 3.3 Since mdadm 3.3 (released 2013, Sep 3), if you have a 3.2+ kernel, you can proceed as follows: # mdadm /dev/md0 --add /dev/sdc1 # mdadm /dev/md0 --replace /dev/sdd1 --with /dev/sdc1 sdd1 is the device you want to replace, sdc1 is the preferred device to do so and must be declared as a spare on your array. The --with option is optional, ...


28

The simple answer to the question in the title is "Yes". But what you really want to do is the next step, which is getting the existing data mirrored. It's possible to convert the existing disk, but it's risky, as mentioned, due the the metadata location. Much better to create an empty (broken) mirror with the new disk and copy the existing data onto it. ...


25

When an array is initially assembled, it is placed in "auto-read-only" mode. I quickly tested, with my kernel (3.10.x) and mdadm (3.3), this doesn't happen on create—but you must be running different versions. However, auto-read-only isn't an error, nor is it anything to worry about. The basic idea behind it is to make --assemble (and, apparently now, even -...


25

The most important difference is that it allows you to increase the flexibility for disk replacement. It is better detailed below along with a number of other recommendations. One should consider to use a partition instead of the entire disk. This should be under the general recommendations for setting up an array and may certainly spare you some ...


23

The first one reports the UUID of the ext4 filesystem on the md block device. It helps the system identify the file system uniquely among the filesystems available on the system. That is stored in the structure of the filesystem, that is in the data stored on the md device. The second one is the UUID of the RAID device. It helps the md subsystem identify ...


19

You can force a check of the entire array while it's online. For example, to check the array on /dev/md0, run as root: echo check > /sys/block/md0/md/sync_action I also have a cron job that runs the following command once a month: tar c /dir/of/raid/filesystem > /dev/null It’s not a thorough check of the drive itself, but it does force the system ...


18

And here's the fix, I'm not entirely clear on why it works, but it does! After updating your mdadm file following the RAID array creation - usually with something like mdadm --detail --scan >> /etc/mdadm/mdadm.conf Simply run the following command update-initramfs -u Then reboot, and everything will work as expected. Voila ;)


17

From Documentation/md.txt in the Linux kernel documentation: clean - no pending writes, but otherwise active. When written to inactive array, starts without resync If a write request arrives then if metadata is known, mark 'dirty' and switch to 'active'. if not known, block and switch to write-pending If written to an active array ...


17

Basic Boot Process Grub Grub reads its disk, md, filesystem, etc. code from the MBR. Grub finds its /boot partition, and reads the rest of itself out of it. Including the config, and any modules the config specifies need loading. Grub follows the instructions in the config, which typically tell it to load a kernel and initramfs into memory, and execute the ...


16

It's because the device nodes no longer exist on your system (probably udev removed them when the drive died). You should be able to remove them by using the keyword failed or detached instead: mdadm -r /dev/md0 failed # all failed devices mdadm -r /dev/md0 detached # failed ones that aren't in /dev anymore If your version of mdadm is too old to do ...


13

If you know the array UUID, then mdadm --assemble /dev/md0 --uuid <uuid> (note the slight difference in parameter order) will do what you want: scan all unused volumes for ones that have md metadata for the given UUID. Other options: mdadm --assemble /dev/md0 --name <name> (does the same thing as --uuid, but with an array name instead of a UUID....


12

mdraid always allows you to move disks around freely in the machine, regardless of how you add the disk to the array. It tracks the disks by the RAID metadata (superblocks) stored on the disk. Note that this assumes mdadm can find the disks when its assembling the arrays. The default (specified in /etc/mdadm/mdadm.conf) is normally DEVICE partitions, which ...


12

The bitmap line in /proc/mdstat indicates how much memory is being used to cache the write-intent bitmap. Basically, in RAID setups with redundant devices, mdadm can use a "bitmap" to keep track of which blocks may be out of sync (because they've been written to). When a block is written to the mdadm device, it is flagged in the bitmap, and then written to ...


11

the Debian and Ubuntu 'mdadm' package contains the file /etc/cron.d/mdadm which in turns the first sunday of each month will run the command /usr/share/mdadm/checkarray --cron --all --idle --quiet that will check all your arrays for consistency (unless you set AUTOCHECK to false in /etc/default/mdadm ). A report will be sent to the 'root' user (make ...


11

There is nothing wrong with --create - if you know what you are doing. The only problem is: You don't know. When you create a RAID, the command is usually something short, like: mdadm --create /dev/md42 --level=5 --raid-devices=3 /dev/sdx1 /dev/sdy1 /dev/sdz1 Dead simple, right? Except it isn't, really. RAID has a lot more variables. There's a data ...


11

That's a bad idea because you're deliberately degrading your RAID and Resyncs might fail unexpectedly. It's better to hook the new disk up to the system (so you have n+1 disks) and then use mdadm --replace to sync it in. That way the RAID never degrades in between. You don't have to fail / remove drives to find out which is which. You can see a device's ...


10

Two issues spring to mind You've got duplicate array definitions in mdadm.conf. Replace (or comment out) the block of three lines following # definitions of existing MD arrays so that each array is declared by your most recent scan. A typical scenario for RAID arrays that fail to build on boot is that either they have not been updated in the initramfs or ...


9

The right thing to do is something like mdadm --add /dev/md0 /dev/sdb1. Use the correct array in place of md0 and the correct partition in place of sdb1. The key thing is the array is running. Its completely unambiguous which data to copy: the data that is currently running. If you have bitmaps enabled, the resync will be fairly fast as it'll only copy what ...


9

That file isn't typically included in the mdadm package. $ rpm -ql mdadm | grep "mdadm.conf" /usr/share/doc/mdadm-2.6.9/mdadm.conf-example /usr/share/man/man5/mdadm.conf.5.gz You can either use the sample one included or generate it your self. My file shows this in the header: # mdadm.conf written out by anaconda So it was likely built by some choices I ...


8

This is an attempt to summarize from the chat troubleshooting session. The setup turns out to be physical disk -> mdraid raid1 -> LVM. So there are several layers to work through. The old setup was (due to unfortunate prior recovery efforts) not available. However, the NAS gui had been used to create another volume on a different disk, and thankfully the ...


8

I use this simple function to check /proc/mdstat: #Health of RAID array raid() { awk '/^md/ {printf "%s: ", $1}; /blocks/ {print $NF}' /proc/mdstat; }


8

If it's RAID 1, and if you know the data offset (e.g. 2048 sectors, you can usually find out the exact data offset by mdadm --examine /dev/sdb1), then you can create a read-only (to be safe) loop device like so: # losetup --find --show --read-only --offset $((2048*512)) /dev/sdb1 /dev/loop7 And then try to check then mount the printed loop device: # fsck....


8

fdisk is the wrong tool for disks >2TB. Use parted or gdisk instead. It appears that /dev/sdc1 and /dev/sdd1 are 2TB partitions, so that's what limits your array size. For the other disks, they have GPT so I assume they are 3TB already, but you should check. Basically you have to stop the array, enlarge each partition to 3TB (without changing the starting ...


8

The cause was an erroneous spares=1 option in the mdadm.conf: # definitions of existing MD arrays ARRAY /dev/md0 UUID=621d5f15:cce75825:60273c48:78a7dac7 spares=1 I'm not sure how this ended up there, but I suppose it happened when a device failed and was replaced. Removing the spares=1 option or just recreating the mdadm.conf from scratch fixes the ...


8

All you should have done was your step one mdadm --manage /dev/md0 --fail /dev/sdc At this point your RAID 5 array is running in degraded mode and you can replace the disk with a new one. Unfortunately it looks like you have truncated your array's effective size from 2TB to 1TB, destroying the second half of your filesystem. Fortunately you say you have ...


7

I hate to be the bearer of bad news, but... Q: I'm new to mdadm, did I do everything correctly? A: No. In fact, you did just about everything in the most destructive way possible. You used --create to destroy the array metadata, instead of using --assemble which probably would have allowed you to read the data (at least, to the extent the disk is ...


7

Your arrays are not properly started. Remove them from your running config with this: mdadm --stop /dev/md12[567] Now try using the autoscan and assemble feature. mdadm --assemble --scan Assuming that works, save your config (assuming Debian derivative) with (and this will overwrite your config so we make a backup first): mv /etc/mdadm/mdadm.conf /etc/...


7

This would seem to be indicating that the syncing between the 2 members of the RAID are not staying in sync with each other. 1. Investigate logs I'd investigate your dmesg logs and see if there are any messages stating that either of the physical HDDs that make up this array are having hardware failures. 2. Check mdadm You can also consult mdadm using ...


7

Check your cron files, many distros do a scheduled resync/re-check once a week. On CentOS 7.1 it's in /etc/cron.d/raid-check # Run system wide raid-check once a week on Sunday at 1am by default 0 1 * * Sun root /usr/sbin/raid-check To configure the behaviour edit /etc/sysconfig/raid-check


Only top voted, non community-wiki answers of a minimum length are eligible