Grub2 RAID /boot

Question

I'm using Gentoo on my HomeServer. My HDDs are raid1 mirrored with mdadm.

My problem is how to boot the /boot partition correct?

/dev/md1 consists of /dev/sdc2 & /dev/sdd2 mounted on /boot

In my grub.cfg I have the uuid of /dev/md1 but at boot time the array is of course not ready.

simplest way would be to un-raid the /boot partitions, but that is not what I want to do.

so, how do i find out the UUID of the raid members? ls -l /dev/disk/by-uuid only shows me the uuid of /dev/md1. And when I unraid the disks, I cannot mount them to get their UUID mount: unknown filesystem type 'linux_raid_member'

so how can I let Grub2 boot those disks, will it be able to determine the contents before they are in raid? (they are in fact ext2).

I should maybe add that I use gpt partition tables, therefore I have a small partition /dev/sdc1 and /dev/sdd1 on both drives to host the bios_grub partition. those are not raided atm.

As pointed out in the comments below Gilles' answer my problem persists. Grub won't find the device, telling me

error: no such device: 9f81a-(device uuid)-5580.
entering rescue mode...
grub rescue> 

afterwards I added the lines

insmod part_msdos
insmod part_gpt
insmod raid
insmod mdraid

to my grub.cfg, just above the line where root is defined root='(md0)' I still get the same error.

I did some more research and i really guess its an error related to the uuids:

# grub-probe -d --target=fs_uuid /dev/md1
9f81a35d-0813-481f-9ae0-e4fba57c5580
# blkid
/dev/sdc2: UUID="11adb545-0e80-61d1-61f6-565f18e8c3f0" UUID_SUB="88826c5e-d12b-307d-6e54-556d1ebb2458" LABEL="livecd:1" TYPE="linux_raid_member"
/dev/sdd2: UUID="11adb545-0e80-61d1-61f6-565f18e8c3f0" UUID_SUB="15150fb2-5066-edba-d39b-08b63219453c" LABEL="livecd:1" TYPE="linux_raid_member"
/dev/md1: UUID="9f81a35d-0813-481f-9ae0-e4fba57c5580" TYPE="ext2"
(I cut away the other drives here)

as you can see both partitions have the same UUID, so I edited the UUID of the drives in my grub.cfg and did grub-install /dev/sdx again, but I still get the error with the "old" /dev/md1 UUID.

Answer 1

The Gentoo Wiki has information that discusses both Grub 1 and Grub 2. Note that the situation is completely different in Grub 1 and Grub 2: Grub 1 doesn't know anything about RAID, but can fake reading from a RAID-1 device by reading one of the disks (this requires a 0.9 or 1.0 mdraid superblock format, not 1.1 or 1.2); Grub 2 supports Linux mdraid volumes natively.

Grub 2 works in three stages (see here for more details):

1. The boot sector.
2. A core image, generated by install-grub in /boot/grub/core.img, capable of showing the rescue prompt and loading modules but not much else.
3. A full system with loadable modules, which normally (through the normal module) shows a boot menu described by /boot/grub/grub.cfg.

Generating the core image

Run grub-install /dev/sda to generate the core image and populate /boot/grub with module files. grub-install also installs a boot sector on the indicated device. Run both grub-install /dev/sdc and grub-install /dev/sdd if you want to have a boot sector on both drives.

If grub-install doesn't detect your drives properly, create the following script as /usr/local/sbin/grub-probe-verbose and run grub-install --grub-probe=/usr/local/sbin/grub-probe-verbose to see what's going on.

#!/bin/sh
exec /usr/sbin/grub-probe "$@" | tee /dev/stderr

From your transcript from bash -x /sbin/grub-install /dev/sdc, it appears that grub-probe isn't detecting /dev/md1 as a RAID array (/sbin/grub-probe --device-map=/boot/grub/device.map --target=partmap --device /dev/md1 and /sbin/grub-probe --device-map=/boot/grub/device.map --target=abstraction --device /dev/md1 return nothing). In case the issue is only in grub-probe and not in the boot code, try overriding its decision:

grub-install --modules='biosdisk ext2 msdos raid mdraid' /dev/sdc

Support for mdraid 1.x superblocks is still recent; it's not in Grub 1.98, you need at least Bazaar revision 2550 dated 2010-07-20 (the code was on a branch for a few months before). Debian squeeze and Ubuntu 10.10 ship with 1.x superblock support; I would expect Gentoo to have 1.99 by now.

Generating grub.cfg

Run grub-mkconfig -o /boot/grub/grub.cfg.grub-mkconfig to produce a tentative Grub configuration file. Review it, and if looks correct, rename it to /boot/grub/grub.cfg.

Device map

You may need to tweak /boot/grub/device.map to get the output of grub-mkconfig right. This file will be generated automatically by grub-mkdevicemap if it doesn't exist, but setups with many disks tend to confuse it. I would expect it to contain something like

(md0) /dev/md1
(hd0) /dev/sdc
(hd1) /dev/sdd

where md0 is the Grub volume name and /dev/md1 is the name under Linux. Grub needs to access /boot/grub for two things:

• Grub's boot sector reads the core image at a fixed location on a disk supported by the BIOS (it doesn't have much choice). With a PC BIOS, (hd0) is the disk that the BIOS reads the boot sector from (it's the disk that contains the boot sector), and (hd1) is some other disk (you don't always get to choose). It's simpler if (hd0) is the disk (or a disk, with a RAID-1 array) that contains /boot/grub.
• The core image loads modules and grub.cfg and the Linux kernel from a filesystem location determined by grub-install, typically /grub or /boot/grub from a device that can be a RAID array or any other volume type supported by Grub.

If you change device.map, you need to run grub-install again. You may need to run grub-mkconfig again if you're not relying on the search command for everything.

Answer 2

To answer the question about the UUID: Use blkid

Answer 3

livecd ~ # mdadm --create /dev/md1 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdc2 /dev/sdd2

livecd ~ # grub

grub> root (hd0,1)

Filesystem type is ext2fs, partition type 0xfd

grub> setup (hd0)

Checking if "/boot/grub/stage1" exists... yes  
Checking if "/boot/grub/stage2" exists... yes  Checking if
"/boot/grub/e2fs_stage1_5" exists... yes  Running "embed
/boot/grub/e2fs_stage1_5 (hd0)"...  16 sectors are embedded. succeeded
Running "install /boot/grub/stage1 (hd0) (hd0)1+16 p
(hd0,0)/boot/grub/stage2 /boot/ grub/menu.lst"... succeeded Done.