5

The Arch Linux ZFS wiki page explains grub-compatible pool creation, as does this page about booting Fedora, but I have not been able to create a pool that is readable by Grub. The Arch Linux wiki page about Installing Arch Linux on ZFS highlights certain bugs but doesn't really explain how to overcome them.

The linked pages explain that Grub supports a subset of zpool features and cannot read a pool that uses features that it doesn't support. They go on to explain how to configure a suitable pool but I have been unable to make it work. The supported feature subset does not appear to be documented anywhere.

I am using a virtual machine to test with Grub 2.02 and Arch Linux kernel 4.16.13-1-ARCH which is the most recent and is compatible with the current zfs-linux package version (zfs-linux-0.7.9.4.16.13.1-1). I am not (yet) trying to make a bootable system, only to prove that Grub can read the zpool. Here is what I have tried:

First, like the arch wiki page suggests, by disabling unwanted features:

# zpool create \
    -o feature@multi_vdev_crash_dump=disabled \
    -o feature@large_dnode=disabled           \
    -o feature@sha512=disabled                \
    -o feature@skein=disabled                 \
    -o feature@edonr=disabled                 \
    testpool mirror \
    /dev/disk/by-id/ata-VBOX_HARDDISK_VB{5f2d4170-647f16b7,f38966d8-57bff7df}

which results in these features:

testpool  feature@async_destroy          enabled                       local
testpool  feature@empty_bpobj            active                        local
testpool  feature@lz4_compress           active                        local
testpool  feature@multi_vdev_crash_dump  disabled                      local
testpool  feature@spacemap_histogram     active                        local
testpool  feature@enabled_txg            active                        local
testpool  feature@hole_birth             active                        local
testpool  feature@extensible_dataset     active                        local
testpool  feature@embedded_data          active                        local
testpool  feature@bookmarks              enabled                       local
testpool  feature@filesystem_limits      enabled                       local
testpool  feature@large_blocks           enabled                       local
testpool  feature@large_dnode            disabled                      local
testpool  feature@sha512                 disabled                      local
testpool  feature@skein                  disabled                      local
testpool  feature@edonr                  disabled                      local
testpool  feature@userobj_accounting     active                        local

Then, like the fedora example, by enabling wanted features:

zpool create -d \
    -o feature@async_destroy=enabled \
    -o feature@empty_bpobj=enabled \
    -o feature@spacemap_histogram=enabled \
    -o feature@enabled_txg=enabled \
    -o feature@hole_birth=enabled \
    -o feature@bookmarks=enabled \
    -o feature@embedded_data=enabled \
    -o feature@large_blocks=enabled \
    testpool mirror \
    /dev/disk/by-id/ata-VBOX_HARDDISK_VB{5f2d4170-647f16b7,f38966d8-57bff7df}

which results in these features:

# zpool get all testpool | grep feature
testpool  feature@async_destroy          enabled                       local
testpool  feature@empty_bpobj            active                        local
testpool  feature@lz4_compress           disabled                      local
testpool  feature@multi_vdev_crash_dump  disabled                      local
testpool  feature@spacemap_histogram     active                        local
testpool  feature@enabled_txg            active                        local
testpool  feature@hole_birth             active                        local
testpool  feature@extensible_dataset     enabled                       local
testpool  feature@embedded_data          active                        local
testpool  feature@bookmarks              enabled                       local
testpool  feature@filesystem_limits      disabled                      local
testpool  feature@large_blocks           enabled                       local
testpool  feature@large_dnode            disabled                      local
testpool  feature@sha512                 disabled                      local
testpool  feature@skein                  disabled                      local
testpool  feature@edonr                  disabled                      local
testpool  feature@userobj_accounting     disabled                      local

In each case, I loaded some content:

# cp -a /boot /testpool

And then, rebooted into Grub:

grub> search --set --label testpool
grub> ls /
@/
grub> ls /@
error: compression algorithm 80 not supported
.
grub> ls /@/
error: compression algorithm inherit not supported
.

I tried enabling/disabling some features, most notably lz4_compress. I also tried creating a dataset on the pool. Nothing I tried worked inside Grub.

I expected to be able to list /boot or /@/boot.

Errors encountered include

  • compression algorithm inherit not supported
  • compression algorithm 66 not supported
  • compression algorithm 80 not supported
  • incorrect dnode type

How should a ZFS zpool be created in order for it to be readable by Grub?

2

Grub cannot reliably perform a directory listing of a zpool due to a bug as confirmed via the mailing list:

listing directory contents in Grub is broken, I have a patch lying around that fixes that specific issue. if you get strange error messages (e.g., something like invalid BP type or compression algorithm), it's likely this issue.

This probelm is present in Grub installed from ArchLinux and also Fedora 28. Ubuntu, however, appears to have patched their Grub to fix this (confirmed with Ubuntu 16.10).

Grub can, however, read the files required to boot. You can create a zpool that Grub will boot like this:

zpool create -m none "$ZPOOL" "$RAIDZ" "${DISKS[@]}"

where the variables define the name of the pool, e.g mypool, the raid level, e.g. mirror and the disks, e.g /dev/disk/by-id/... (two disks being required for a mirror).

You need to create a dataset

zfs create -p "$ZPOOL"/ROOT/archlinux

And you need to set the dataset's mount point:

zfs set mountpoint=/ "$ZPOOL"/ROOT/archlinux

You can then boot it with Grub using commands like:

insmod part_gpt
search --set --label mypool
linux /ROOT/archlinux@/boot/vmlinuz-linux zfs=mypool rw
initrd /ROOT/archlinux@/boot/initramfs-linux.img
boot

I scripted this to test it in a VirtualBox machine. I installed Arch from the ISO onto a normal ext4 root and then used that to instal a new ZFS root onto two mirrored virtual disks. The above is a summary - see the script for full details.

  • Dunnow how you got this woirking, but on recent centos grub2-probe / fails to detect zfs when all of the features are enabled for a pool. It even states the exact feature it stumbles on (and zfs detection fails after that) and then, sure, it cannot boot when asked. This is an interesting post, but all the grub directives you're providing are pretty much regular ones required for a boot from zfs. – drookie Jan 31 at 19:22
  • And yeah - from my opinion - this has nothing to do with ability to actually list zpool/zfs contents from grub [rescue shell]. – drookie Jan 31 at 19:25
  • @drookie my systems work with all listed features showing as either enabled or active (the listed features being those identified in the question). Nothing is disabled. I don't use grub-probe though and I hand roll my own grub.cfg. – starfry Feb 1 at 10:26
  • Could you please show your grub version ? – drookie Feb 1 at 16:52
  • 1
    Sure, @drookie, my Grub version shows as 2.03 on the Grub screen. I did build it myself from source because I also use these crypto extensions, I specifically built from this tag (not that that would make any difference to the ZFS support). – starfry Feb 3 at 19:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.