61

How mature and featureful is LVM RAID? LVM-RAID is actually mdraid under the covers. It basically works by creating two logical volumes per RAID device (one for data, called "rimage"; one for metadata, called "rmeta"). It then passes those off to the existing mdraid drivers. So things like handling disk read errors, I/O load balancing, etc. should be fairly ...


38

I just had a similar problem to what you describe, though for me it happened when I was attempting to install the new Ubuntu 11.10 Oneiric Ozelot to an LVM volume. I had done the following to set lvm up on a live boot system (the logical volumes I needed were already present): apt-get install lvm2 vgscan --mknodes -v Now lvscan -v showed my volumes but ...


36

Using mdadm 3.3 Since mdadm 3.3 (released 2013, Sep 3), if you have a 3.2+ kernel, you can proceed as follows: # mdadm /dev/md0 --add /dev/sdc1 # mdadm /dev/md0 --replace /dev/sdd1 --with /dev/sdc1 sdd1 is the device you want to replace, sdc1 is the preferred device to do so and must be declared as a spare on your array. The --with option is optional, ...


25

The most important difference is that it allows you to increase the flexibility for disk replacement. It is better detailed below along with a number of other recommendations. One should consider to use a partition instead of the entire disk. This should be under the general recommendations for setting up an array and may certainly spare you some ...


23

Hardware and Software RAID are two different worlds. Since you mention "Server" most likely there is Hardware RAID present. to find out use: lspci -vv | grep -i raid If Hardware RAID is present the output should be something like: Subsystem: abcdefg RAID Controller To find out more about your Hardware RAID configuration, this is only possible using ...


20

All these "poke the sector" answers are, quite frankly, insane. They risk (possibly hidden) filesystem corruption. If the data were already gone, because that disk stored the only copy, it'd be reasonable. But there is a perfectly good copy on the mirror. You just need to have mdraid scrub the mirror. It'll notice the bad sector, and rewrite it ...


17

Q#1: Can you set up a RAID system using USB sticks as the storage media You should be able to use any block storage devices in a RAID. Any standard directions for setting up a RAID using SATA HDD's should be applicable when using USB storage as well. You'll have to set it up so that the USB devices are assembled as members of the RAID array. Q#2: What ...


16

It's because the device nodes no longer exist on your system (probably udev removed them when the drive died). You should be able to remove them by using the keyword failed or detached instead: mdadm -r /dev/md0 failed # all failed devices mdadm -r /dev/md0 detached # failed ones that aren't in /dev anymore If your version of mdadm is too old to do ...


16

If the two disks are /dev/sda and /dev/sdb, run both grub-install /dev/sda and grub-install /dev/sdb. Then both drives will be able to boot alone. Make sure that your Grub configuration doesn't hard-code disks like (hd0), but instead searches for the boot and root filesystems' UUIDs. I'm not aware of support in Grub to declare two disks as being in a RAID-...


14

Not sure if the installer does it by default, but if (once the system is booted) you run: # dpkg-reconfigure -plow grub-pc just hit enter until you're prompted for "GRUB install devices", then you can select the MBRs of both drives. Here is a screenshot of the relevant screen in Debian Jessie (8.3), with GRUB2 version 2.02~beta2-22+deb8u1. This shows the ...


13

The bitmap line in /proc/mdstat indicates how much memory is being used to cache the write-intent bitmap. Basically, in RAID setups with redundant devices, mdadm can use a "bitmap" to keep track of which blocks may be out of sync (because they've been written to). When a block is written to the mdadm device, it is flagged in the bitmap, and then written to ...


11

This is working excellently in Ubuntu 14.04: sudo -i mdadm --assemble --scan You will get: mdadm: /dev/md/1 has been started with 1 drive (out of 2) Then mount and see your files: cd /mnt && mkdir to-restore-md1 && mount /dev/md1 to-restore-md1 ls -la to-restore-md1


10

Yes grub2 is fully raid ( and LVM ) aware. In fact you do not need a separate /boot partition at all; you can just put everything on the raid5. Ideally you want to not install with a /boot partition at all, but removing it after the fact simply means copying all of the files to the root partition, and reinstalling grub, like this: umount /boot mount /dev/[...


10

The right thing to do is something like mdadm --add /dev/md0 /dev/sdb1. Use the correct array in place of md0 and the correct partition in place of sdb1. The key thing is the array is running. Its completely unambiguous which data to copy: the data that is currently running. If you have bitmaps enabled, the resync will be fairly fast as it'll only copy what ...


10

The warning messages about the file descriptor leaking are indeed harmless, and are being tracked in bug 592834. You might want to contribute your findings there; I'm not sure Colin (the maintainer) has been able to reproduce the bug in this instance...


9

That file isn't typically included in the mdadm package. $ rpm -ql mdadm | grep "mdadm.conf" /usr/share/doc/mdadm-2.6.9/mdadm.conf-example /usr/share/man/man5/mdadm.conf.5.gz You can either use the sample one included or generate it your self. My file shows this in the header: # mdadm.conf written out by anaconda So it was likely built by some choices I ...


8

If it's RAID 1, and if you know the data offset (e.g. 2048 sectors, you can usually find out the exact data offset by mdadm --examine /dev/sdb1), then you can create a read-only (to be safe) loop device like so: # losetup --find --show --read-only --offset $((2048*512)) /dev/sdb1 /dev/loop7 And then try to check then mount the printed loop device: # fsck....


8

fdisk is the wrong tool for disks >2TB. Use parted or gdisk instead. It appears that /dev/sdc1 and /dev/sdd1 are 2TB partitions, so that's what limits your array size. For the other disks, they have GPT so I assume they are 3TB already, but you should check. Basically you have to stop the array, enlarge each partition to 3TB (without changing the starting ...


8

The cause was an erroneous spares=1 option in the mdadm.conf: # definitions of existing MD arrays ARRAY /dev/md0 UUID=621d5f15:cce75825:60273c48:78a7dac7 spares=1 I'm not sure how this ended up there, but I suppose it happened when a device failed and was replaced. Removing the spares=1 option or just recreating the mdadm.conf from scratch fixes the ...


7

Linux mdraid has several metadata formats. Formats 0.9 and 1.0 put the metadata at the end of the containing device, and the payload (the filesystem) starts at the beginning of the device and can be accessed directly without going through the raid layer. Formats 1.1 and 1.2 put the metadata at the middle and beginning of the containing device respectively, ...


7

I didn't know LVM could do RAID either. Personally, I would stick with mdadm since it's a much more mature software that does the same thing. If something breaks with LVM RAID, you're probably not going to be able to get as much support than if you had gone with mdadm. Additionally, I wouldn't trust LVM RAID since LVM has historically shown to not be the ...


7

My answer had 2 parts: investigation of the block device driver; and optimisation worth looking at with your use case. But I removed last part as it was reported that it can lead to data loss. See comments. Investigation of Hardware I understood that for the same application but on 2 different sets of hardware the performance is very different and you ...


7

Another important argument is that some mainboards may delete your RAID superblocks if you use whole disk devices and are not super careful with wiping them when adding disks to a RAID array that once were GPT devices. I learned this the hard way, when my ASRock mainboard's UEFI rendered my RAID unusable: http://forum.asrock.com/forum_posts.asp?TID=10174 ...


7

The --write-mostly, --write-behind is handled by the md driver internally. md keeps metadata, like the write-intent bitmap (which is mandatory for the write-behind feature) that basically logs which data has been written yet vs. which data is still missing. This is necessary in case there is a power loss event, when the data hasn't reached the write-mostly ...


6

First, you create the raid array. Assuming the new drives are sdc, sdd, and sde, and you don't already have any raid arrays, and you have created a single raid partition on each, do: sudo -s mdadm --create /dev/md0 -n 3 -l raid5 /dev/sd[cde]1 mdadm --detail --scan >> /etc/mdadm/mdadm.conf Then you add it to the vg, move the logical volumes over, and ...


6

Simply run mdadm --build -l1 -n2 /dev/md0 /dev/dm-10 /dev/dm11 to get your data back. Verify that the devices are correct (or use the aliases from /dev/mapper) before doing this! Also, if your kernel has already used (or is using) one of these devices, data will be inconsistent. You should set up the second device as a degenerate 1.2 array, copy the data ...


6

We have configured all of our servers to run regular selftests with the following line in /etc/smartd.conf: # DEVICESCAN matches all hard disks found in /dev/ and applies the following # options to them. # # Default options from Debian: # -d removable don't exit when the device when a device vanishes # -n standby don't wake a device up that is ...


6

Partition start/alignment Make them start at 1 MiB boundaries, for example using parted and unit mib. That way you won't have an issue with today's 4k sector disks, and not with tomorrow's 8k or 16k disks... and you only waste 1MiB per disk. You can verify the partition alignment of any given disk using parted /dev/disk unit b print free. It prints ...


6

What I ended up doing was using mknod like @derobert suggested to create the devices that mdadm was looking for. I tried the major/minor numbers mdadm was telling me it couldn't find with the different drive letters I was trying to remove until it worked. mknod /dev/sde1 b 8 17 Then I had to use the --force option to get it remove the component. mdadm /...


6

RAID always needs a device (at least if you use md). There are two ways. The probably better one is to use a network block device: https://en.wikipedia.org/wiki/Network_block_device You can tell md to use such a device for writing only (because it's too slow for reading) with mdadm ... --write-mostly. The other option is to create a file on a network ...


Only top voted, non community-wiki answers of a minimum length are eligible