1

I'm trying to install NixOS 19.09 on a notebook. My disk setup is a 30GB SSD and a 700GB HDD. I decided that I want to use the SSD as cache for the HDD, and also that I want the entire root partition to be encrypted.

My partition setup therefore looks like this:

  • /dev/sda - SSD
    • /dev/sda1 - boot partition, UEFI bootable flag, formatted with FAT, unencrypted
    • /dev/sda2 - to be used for the cache
  • /dev/sdb - HDD, unpartitioned, to be used as the main disk with LVM

I formatted /dev/sda1 with mkfs.fat, I set up a volume group vg that included /dev/sda2 and /dev/sdb, and I then followed the guide that you can find in the lvmcache manual page. After creating a single cached volume called hdd0, I ran cryptsetup on it, so that I could achieve a LUKS-on-LVM scheme. The entire cached hdd0 is encrypted with LUKS2.
Then I unlocked hdd0 again using cryptsetup, mapping it to /dev/mapper/root. I mounted /mnt on it, created /mnt/boot and mounted /dev/sda1 there. After that, I generated a NixOS config and basically left it at default, with the notable changes of adding dm-cache to boot.initrd.kernelModules, and setting boot.initrd.luks.devices."root".preLVM to false. Then, I ran nixos-install successfully and rebooted.

The system managed to successfully run GRUB, but after that failed to find the root volume. It provided an error message of:

device-mapper: table: 254:3: cache: Error creating cache's policy  
device-mapper: reload ioctl on (254:3) failed: Invalid argument

I'm assuming it has a problem with detecting the entire hdd0. I tried booting it from the live installation USB stick, and there you have to enable dm-cache using modprobe and then run

# lvchange -ay vg

to get hdd0 to be active and visible.

Did anyone try a similar setup? How can I get it to boot automatically? Or is the setup flawed conceptually and can never work?

PS: I hope the question is detailed enough, it's my first time asking here.

2
  • Have you tried adding dm-cache to boot.initrd.availableKernelModules? The modules listed here are added to the initrd, making them available before the root fs is mounted. Mar 25, 2020 at 5:01
  • I tried, didn't fix the issue.
    – fkmjec
    Mar 28, 2020 at 17:43

1 Answer 1

0

Here's my conf that I got to work, today, after some headbangery:

boot.initrd.availableKernelModules = [ "nvme" "xhci_pci" "ahci" "ehci_pci" "usb_storage" "usbhid" "sd_mod" "sr_mod" ];
  boot.initrd.kernelModules = [ "dm-cache" "dm-cache-smq" "dm-cache-mq" "dm-cache-cleaner" ];
  boot.kernelModules = [ "kvm-amd" "dm-cache" "dm-cache-smq" "dm-persistent-data" "dm-bio-prison" "dm-clone" "dm-crypt" "dm-writecache" "dm-mirror" "dm-snapshot"];

  boot.initrd.luks.devices = {
    "decrypted" = {
      device = "/dev/mapper/vg-crypt";
      allowDiscards = true;
      preLVM = false;
    };
  };

The main trick was to have "dm-cache-smq" in modules - without it, I was getting a similar error message like what you're getting.

You'll have an ugly warning message about cache_check missing - to get rid of it, add this too:

services.lvm.boot.thin.enable = true;

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.