rEFInd loads with blank screen (logo only, no options to boot from)

Question

Upon booting and choosing rEFInd, I get a blank screen as shown in this post.

My filesystem is like so:

$ mount
/dev/mapper/luks-71740d10-cf57-4b82-9311-aa584acaa81e on / type btrfs (rw,relatime,compress=lzo,ssd,space_cache,subvolid=256,subvol=/arch)
/dev/nvme1n1p2 on /boot type ext4 (rw,relatime)
/dev/nvme1n1p1 on /boot/efi type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro)

The contents of my boot partition is (omitting refind icons directory for brevity):

$ ls -lR /boot/
/boot/:
total 159116
drwxr-xr-x 3 root root     4096 Dec 31  1969 efi
-rw-r--r-- 1 root root 85725131 Nov  7 13:32 initramfs-linux-fallback.img
-rw-r--r-- 1 root root 62696900 Nov  7 13:32 initramfs-linux.img
-rw-r--r-- 1 root root  4769792 Jun  8 13:31 intel-ucode.img
-rw-r--r-- 1 root root  9735104 Nov  5 13:39 vmlinuz-linux

/boot/efi:
total 4
drwxr-xr-x 4 root root 4096 Nov  7 14:38 EFI

/boot/efi/EFI:
total 8
drwxr-xr-x 6 root root 4096 Nov  7 14:38 refind
drwxr-xr-x 2 root root 4096 Apr 19  2021 tools

/boot/efi/EFI/refind:
total 412
drwxr-xr-x 2 root root   4096 Apr 19  2021 drivers_x64
drwxr-xr-x 3 root root   8192 Nov  7 13:25 icons
drwxr-xr-x 2 root root   4096 Apr 19  2021 keys
drwxr-xr-x 2 root root   4096 Nov  7 14:38 vars
-rwxr-xr-x 1 root root    140 Nov  7 13:25 BOOT.CSV
-rwxr-xr-x 1 root root  37839 Nov  7 14:39 refind.conf
-rwxr-xr-x 1 root root  34513 Nov  7 13:25 refind.conf-sample
-rwxr-xr-x 1 root root 317712 Nov  7 13:25 refind_x64.efi

/boot/efi/EFI/refind/drivers_x64:
total 72
-rwxr-xr-x 1 root root 69776 Nov  7 13:27 ext4_x64.efi

My configuration is as follows:

menuentry "arch" {
  icon      /EFI/refind/icons/os_arch.png
  volume    3c3b698a-8212-324b-9b63-4dc3a7a98aee
  loader    /vmlinuz-linux
  initrd    /initramfs-linux.img
  options   "luks.uuid=71740d10-cf57-4b82-9311-aa584acaa81e root=UUID=15773c68-a4e5-49b5-8071-f390750b7162 rootflags=compress=lzo,ssd,subvol=arch ipv6.disable_ipv6=1 add_efi_memmap rw"
}

menuentry "windows" {
  icon      /EFI/refind/icons/os_win.png
  volume    13aa9955-4538-40ac-b674-006c334b5088
  loader    \EFI\Microsoft\Boot\bootmgfw.efi
}

My devices are as follow:

$ sudo blkid
/dev/nvme0n1p5: LABEL="Windows RE Tools" BLOCK_SIZE="512" UUID="08C4E6C5C4E6B45A" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="4eced110-9723-4dde-955c-b0ab8576869b"
/dev/nvme0n1p3: TYPE="BitLocker" PARTLABEL="Basic data partition" PARTUUID="a358833d-f07c-45f8-9119-3592577bfd23"
/dev/nvme0n1p1: LABEL="SYSTEM" UUID="4CE7-C215" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="EFI system partition" PARTUUID="13aa9955-4538-40ac-b674-006c334b5088"
/dev/nvme0n1p4: BLOCK_SIZE="512" UUID="D2063641063626BF" TYPE="ntfs" PARTUUID="32a62eef-7764-4de9-b230-e582a2fba9ef"
/dev/mapper/luks-71740d10-cf57-4b82-9311-aa584acaa81e: UUID="15773c68-a4e5-49b5-8071-f390750b7162" UUID_SUB="b79765d9-5a84-4327-b8e0-b4a7c27107a8" BLOCK_SIZE="4096" TYPE="btrfs"
/dev/nvme1n1p2: UUID="db6871f3-f5d5-4d46-965e-7bbd9d335d34" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="3c3b698a-8212-324b-9b63-4dc3a7a98aee"
/dev/nvme1n1p3: UUID="71740d10-cf57-4b82-9311-aa584acaa81e" TYPE="crypto_LUKS" PARTUUID="004162c0-d9e6-5043-8427-630a0efce107"
/dev/nvme1n1p1: UUID="0298-8CEA" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="82f72318-1bba-2d4b-936d-41142e979d9f"
/dev/nvme0n1p2: PARTLABEL="Microsoft reserved partition" PARTUUID="2d6499be-a167-4920-93d6-7a15a8a82ee6"

There are no actual errors reported, rEFInd just boots to this blanks screen and stays there indefinitely. Pressing enter (in case this is a graphical problem and behind the scenes everything is as it should be) does nothing. Using refind-install from a live Arch USB disc also does not fix this, creating an improper refind_linux.conf. I know my refind.conf to be correct as it has previously worked.

Answer 1

I'm asking this question to get the solution on a more "mainstream" and google-able result vs. the plethora of forum/reddit/mailing list incantations of this question. The arch linux wiki rEFInd article contained a troubleshooting section, but it only advised ensuring that esp/EFI/refind/drivers contained only the filesystem for the /boot partition and no other drivers.

In an email dialog with Rod Smith (author of rEFInd), he provided a custom version of rEFInd with some logging output. Here was the key finding of that exchange, which showed that rEFInd hung on scanning an NTFS volume on a separate Windows drive.

Rod hypothesized the potential cause as any of the following:

• You have rEFInd's NTFS driver installed. (You indicated in an earlier e-mail that you didn't, so you may want to double-check this.) If so, one of the following is likely true:
• The NTFS volume is damaged
• There's a bug in the NTFS driver
• There's something about the directory/file structure on this partition that's triggering a bug in rEFInd
• Your firmware includes its own NTFS driver. I've heard rumors of such implementations, but I've never encountered one myself. If this is the case, one of the three preceding sub-cases would also be true.
• rEFInd has misidentified a non-NTFS partition as being NTFS. This might happen because of leftover NTFS data after a partition was converted from another form. This shouldn't cause a problem by itself, but it's conceivable that the same hypothetical leftover NTFS data is causing the driver for the filesystem the partition actually holds to malfunction. In this case, backing up the partition, zeroing it out, creating a fresh filesystem, and restoring it should fix the problem.
• The built-in FAT or rEFInd EFI ext4fs driver is mistaking the NTFS partition as being FAT or ext4fs, respectively, and is hanging when it tries to access the volume.

The solution/workaround was to add the offending PARTUUID values to the dont_scan_volumes option in esp/EFI/refind/refind.conf.

One may find these via the output of blkid, and then add them as comma-separated values. For example:

$ sudo blkid
/dev/nvme0n1p1: LABEL="SYSTEM" UUID="4CE7-C215" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="EFI system partition" PARTUUID="13aa9955-1234-5678-9098-006c334b5088"
/dev/nvme0n1p5: LABEL="Windows RE Tools" BLOCK_SIZE="512" UUID="08C4E6C5C4E6B45A" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="4eced110-0987-6543-2123-b0ab8576869b"

$ grep dont_scan_volumes /boot/efi/EFI/refind/refind.conf
dont_scan_volumes 13aa9955-1234-5678-9098-006c334b5088,4eced110-0987-6543-2123-b0ab8576869b

The email exchange with Rod was in 2018 and required for me to migrate from rEFInd on Win7 to Win10, either due to a change in rEFInd's scanning behavior with Win10 partitions, or perhaps related to shipped options on a corporate computer (e.g. bitlocker). I'm adding this answer now, as a Windows 10 upgrade just added a new partition to the existing disc and caused this issue anew.

After hours of reading through other possible causes of this, it re-dawned on me to make sure that all Windows PARTUUID values were listed in dont_scan_volumes, finding that when checking refind.conf vs. blkid after this upgrade, there was a new partition to add. Once added, rEFInd again showed the expected boot options from my refind.conf (arch and windows).