Context: while trying to install Arch Linux with an LVM setup including RAID1 for my root volume, I encountered the problem that I described here on the Arch Linux fora. I never found a solution to this issue, but it disappeared after I reinstalled the entire system from scratch except for the swapfile (I created a swap partition instead) so I forgot about it.
After accidentally turning the power to my system (the 'reinstalled-from-scratch' system from above, plus a few months of use) off at the plug while the system was suspended, I rebooted my system to be presented with the same error again.
The problem is just like that described in some detail in the link above. To make the Stack Exchange into a self-complete knowledge base, I will include the contents of the post here:
I'm installing Arch Linux using LVM. About my third or fourth time installing Arch, but my first time trying out LVM. Here's my disk layout:
I have two physical storage media, /dev/sda (an SSD, capacity about 100GB) and /dev/sdb (an HDD, capacity about 1TB).
Using LVM, I created a volume group volumegroup0, which contains:
- a system volume mirrored across the SSD and the HDD using raid1, which takes up most of the SSD's capacity (volumegroup0/systemvolume_raid1), with ext4 filesystem (which contains a swapfile of 16GB)
- a boot volume on the SSD about 500MB (volumegroup0/boot), with btrfs filesystem
- a home volume on the HDD, which has a 10GB cachepool on the SSD for faster access (volumegroup0/home), with btrfs filesystem
- a var volume on the HDD (volumegroup0/var), which has a 2GB cachepool on the SSD, like the home volume, with ext4 filesystem
- a snapshot volume for making backups of the systemvolume, called volumegroup0/systemvolume_snapshot.
Now, the system could boot last night, and I thought that I had successfully completed installation. But on booting this morning, the system fails to mount the root volume, and drops me into an emergency shell.
Booting with the default GRUB parameter (I'm using GRUB), "quiet", off, this is what the screen looks like after failure to boot:
:: running early hook [udev] Starting version 242.29-2-arch :: running early hook [lmv2] :: running hook [udev] :: Triggering uevents...
This stays on the screen for what feels like 30 seconds or so before the following appears:
Waiting 10 seconds for device /dev/mapper/volumegroup0-systemvolume_raid1 ...
Then, after 10 seconds:
ERROR: device '/dev/mappervolumegroup0-systemvolume_raid1' not found. Skipping fsck. :: mounting '/dev/mappervolumegroup0-systemvolume_raid1' on real root mount: /new_root: no filesystem type specified. You are now being dropped into an emergency shell. sh: can't access tty; job control turned off. [rootfs ]#
While looking for a solution to the issue by booting from an archiso live USB, I've noticed something odd. Directly after booting, if I
ls /dev/mapper
or
ls /dev/volumegroup0
, the output contains every logical volume that I would expect -- except for systemvolume_raid1. Even volumegroup0-systemvolume_raid1_rimage0 and 1 and volumegroup0-systemvolume_raid1_rmeta0 and 1 are present in /dev/mapper, but the root volume proper is absent. But if I run
vgscan
(with OR without --mknodes then list either of those directories again, systemvolume_raid1 is present and correct as if nothing's wrong. And I can mount it without any problems -- which I have done, in order to try regenerating the initramfs after checking that the necessary modules and hooks were present in mkinitcpio.conf (they were) and regenerating /etc/fstab (using genfstab -U) without the swapfile (since regenerating /etc/fstab to include a line for automatically activating swap was the last thing I did).
[The contents of /etc/fstab were included in the original forum post, but those contents are not quite identical since ; see below for the correct version]
UPDATE:
If I run
lvs -a
when systemvolume_raid1 is not visible, I get a lot of lines of the following output above the table containing the proper output of lvs -a:
Expected raid segment type but got NULL instead.
Notes:
The LVM article on the Arch Wiki says that you should ensure the kernel parameter "root" points to the mapped device, "e.g. /dev/[italic]vg-name[/italic]/[italic]lv-name[/italic]". In my case, the parameter points to /dev/mapper/volumegroup0-systemvolume_raid1, but changing it to a form as recommended by the wiki doesn't change anything except the device name in the error messages at boot (it becomes the new value of "root", as one might expect).
I'm using GRUB, generating configuration with grub-mkconfig, if that's relevant.
My problem appears to be similar to this unanswered problem on Unix & Linux Stack Exchange.
Here are the correct contents of /etc/fstab which are different to the version in the linked forum post above:
#/dev/mapper/volumegroup0-systemvolume_raid1
UUID=... / ext4 rw,relatime 0 1
#/dev/mapper/volumegroup0-var
UUID=... /var ext4 rw,relatime,stripe=16 0 2
#/dev/mapper/volumegroup0-boot
UUID=... /boot btrfs rw,relatime,ssd,space_cache,subvolid=5,subvol=/ 0 0
#/dev/mapper/volumegroup0-home
UUID=... /home btrfs rw,relatime,ssd,space_cache,subvolid=5,subvol=/ 0 0
#/dev/mapper/volumegroup0-swap
UUID=... none swap defaults 0 0
I'm sure that the issue has something to do with a failure to 'activate' the root volume, or to check for root volumes at all, at system boot, but I do not know in what way.
A relevant point may be that the output of lvscan after booting from a live USB is:
ACTIVE '/dev/volumegroup0/home' [500.00 GiB] inherit
ACTIVE '/dev/volumegroup0/var' [50.00 GiB] inherit
ACTIVE Original '/dev/volumegroup0/systemvolume_raid1' [83.50 GiB] inherit
inactive '/dev/volumegroup0/boot' [500.00 MiB] inherit
ACTIVE Snapshot '/dev/volumegroup0/systemvolume_snapshot' [40.00 GiB] inherit
inactive '/dev/volumegroup0/swap' [16.00 GiB] inherit
inactive Snapshot '/dev/volumegroup0/systemvolume_pre_dwarf' [3.00 GiB] inherit
After running a command such as pvscan or vgscan (I say "such as" because other commands may have the same effect but I have not tested every possible LVM command), all the volumes marked above as inactive become active. This seems to support my hypothesis that something isn't being activated properly at boot, but I can't say anything for sure.
Why will the system not boot? How can I solve this?
