2

After getting a new system, I moved the NVME drive from my old system to the new one, where both machines are set to boot using UEFI. As the EFI setup is machine specific, I expected to encounter trouble, and came prepared - I booted from a live USB, did the mounting and chrooting thing, and reinstalled GRUB according to instructions (found everywhere on the web and repeated below), after which a reboot brings me to the GRUB command prompt, and I can't figure out how to fix this - no matter what I do, if I select the Linux UEFI boot entry (as shown below), I get a grub prompt. If I select the Windows UEFI boot entry (this is a dual boot system), it loads fine.

I can always boot the Linux install from the GRUB prompt by typing configfile (hd0,gpt6)/@/boot/grub/grub.cfg which starts the GRUB menu I expected and from that I can boot any entry without problems (I'm using BTRFS as the root partition, and / is in subvolume @ - this is pretty standard for an Ubuntu install).

Here's the current setup:

# efibootmgr --verbose
BootCurrent: 0001
Timeout: 0 seconds
BootOrder: 0001,0005,0000,0002
Boot0000* UEFI Samsung SSD 970 EVO Plus 1TB S6P7NF0T423021F 1   HD(1,GPT,37b6d616-6865-44b1-a382-9987345e2cfa,0x800,0x32000)/File(\EFI\Boot\BootX64.efi)N.....YM....R,Y.
Boot0001* Neon  HD(1,GPT,37b6d616-6865-44b1-a382-9987345e2cfa,0x800,0x32000)/File(\EFI\Neon\shimx64.efi)
Boot0002* UEFI HTTPs Boot       PciRoot(0x0)/Pci(0x1f,0x6)/MAC(000000000000,0)/IPv4(0.0.0.00.0.0.0,0,0)/Uri()N.....YM....R,Y.
Boot0005* Windows Boot Manager  HD(1,GPT,37b6d616-6865-44b1-a382-9987345e2cfa,0x800,0x32000)/File(\EFI\Microsoft\Boot\bootmgfw.efi)WINDOWS.........x...B.C.D.O.B.J.E.C.T.=.{.9.d.e.a.8.6.2.c.-.5.c.d.d.-.4.e.7.0.-.a.c.c.1.-.f.3.2.b.3.4.4.d.4.7.9.5.}....................
# ll /boot/efi/EFI/
total 6
drwx------ 6 root root 1024 Jul 26 12:38 ./
drwx------ 4 root root 1024 Jan  1  1970 ../
drwx------ 2 root root 1024 Aug 28  2022 Boot/
drwx------ 5 root root 1024 Jul 26 12:38 Dell/
drwx------ 4 root root 1024 Aug 28  2022 Microsoft/
drwx------ 2 root root 1024 Jul 24 20:01 Neon/
# cat /boot/efi/EFI/Neon/grub.cfg 
search.fs_uuid 2886a665-f535-496e-a543-13c62983b0da root 
set prefix=($root)'/@/boot/grub'
configfile $prefix/grub.cfg
# ll /dev/disk/by-uuid/
total 0
drwxr-xr-x 2 root root 120 Jul 26 13:20 ./
drwxr-xr-x 9 root root 180 Jul 26 13:20 ../
lrwxrwxrwx 1 root root  15 Jul 26 13:20 2886a665-f535-496e-a543-13c62983b0da -> ../../nvme0n1p6
lrwxrwxrwx 1 root root  15 Jul 26 13:20 80D9-5688 -> ../../nvme0n1p1
lrwxrwxrwx 1 root root  15 Jul 26 13:20 82926928-1ed1-4a76-b0e8-d62a0171c1ee -> ../../nvme0n1p5
lrwxrwxrwx 1 root root  15 Jul 26 13:20 D8D6FA10D6F9EF1E -> ../../nvme0n1p4
# lsblk -e 7 -o name,fstype,size,fsused,label,partlabel,mountpoint,uuid,partuuid
NAME        FSTYPE      SIZE FSUSED LABEL PARTLABEL                    MOUNTPOINT            UUID                                 PARTUUID
nvme0n1               931.5G                                                                                                      
├─nvme0n1p1 vfat        100M  58.8M       EFI system partition         /boot/efi             80D9-5688                            37b6d616-6865-44b1-a382-9987345e2cfa
├─nvme0n1p2              16M              Microsoft reserved partition                                                            fdc1478a-852d-4f33-8e22-24a2ea209726
├─nvme0n1p3 BitLocker  50.2G              Basic data partition                                                                    e8584ca9-db86-4bb1-abf7-424afe77bc94
├─nvme0n1p4 ntfs        517M                                                                 D8D6FA10D6F9EF1E                     5373ade9-ed41-4164-85e2-c28b7026c25f
├─nvme0n1p5 swap       30.5G                                           [SWAP]                82926928-1ed1-4a76-b0e8-d62a0171c1ee d5d866a5-97d4-49bb-a182-6eec3abcba18
└─nvme0n1p6 btrfs     850.2G 440.5G linux                              /var/lib/docker/btrfs 2886a665-f535-496e-a543-13c62983b0da cb45a183-14d3-4a95-b89f-6b5e315609cd

I can try to reinstall by:

  1. Remove the offending EFI boot entry: efibootmgr -b 1 -B
  2. Reinstall the boot loaded to disk and EFI: grub-install /dev/nvme0n1 --target x86_64-efi --efi-directory /boot/efi/ --bootloader-id Neon
  3. Update the GRUB configuration (I'm not sure it is needed, but that's what the docs say): update-grub2

After which the configuration looks like I'm showing above, and on restart GRUB will not start the menu automatically. I'm not sure about the UUID in the EFI NVRAM dump, but this value isn't configured anywhere that I can find and it reproduces if I remove the entries and recreate them.

Any idea what I'm missing?

7
  • Reinstall of grub should have worked. You also need to compare GUID/partUUID in UEFI entry and of ESP. And then compare UUID in /EFI/Neon/grub.cfg with UUID of your install (or /boot if separate). lsblk -e 7 -o name,fstype,size,fsused,label,partlabel,mountpoint,uuid,partuuid Also check fstab: cat /etc/fstab UUID of ESP and / must be correct also, with no incorrect UUIDs trying to mount.
    – oldfred
    Commented Jul 26, 2023 at 12:52
  • @oldfred, thanks for looking at this issue. The ESP is mounted in fstab as UUID=80D9-5688 which matches the listing of disk/by-uuid (I like that output better than lsblk's). I showed that UUID in EFI/Neon/grub.cfg matches that of nvme0n1p6, and it is also what it says in fstab (I should have posted fstab too, but at this point it should be suffice to say that it has nothing unexpected). Once the GRUB menu appears, everything works fine: there are no mount failures nor any other blockers.
    – Guss
    Commented Jul 26, 2023 at 16:02
  • @oldfred apologies - I confused lsblk's PARTUUID and what fstab and /dev/disks/by-uuid calls UUID. Looking at the lsblk output (added to OP), I can see that the UUID in the efibootmgr output is the PARTUUID of the ESP. So - that seems OK as well.
    – Guss
    Commented Jul 26, 2023 at 16:31
  • What brand/model system? Is UEFI set to default boot in UEFI mode?
    – oldfred
    Commented Jul 26, 2023 at 18:42
  • It's a Dell Precision 5570 (the old one is a 5520). Both are set to UEFI with secure boot disabled. The default boot entry is "Neon".
    – Guss
    Commented Jul 26, 2023 at 18:54

1 Answer 1

0

TL;DR:

when using the Ubuntu-built GRUB boot loader, the UEFI boot entry MUST be named ubuntu (or at least, the GRUB EFI configuration file must be at EFI/ubuntu/grub.cfg.

Details:

The problem is with the GRUB setup in UEFI, Secure Boot (I assume) and how Ubuntu has this all set up: in legacy BIOS boot, the BIOS boots a tiny (512 bytes, I believe) executable that either immediately boots something else (in ye olde times) or finds a hardcoded section on the disk where the rest of the boot loader resides (a "stage 2 boot loader"). There are many problems with that setup, the least of it is that it can't work with advanced file systems that move things around, compress them, etc - which is why in the past if you wanted to run root on an advanced file system, you had to have a separate /boot partition in ext2, where the GRUB second stage lived.

With UEFI, there's a (comparatively) large FAT32 partition (the ESP) that operating system can deploy large boot loaders into and the UEFI firmware will load the entire boot loader at once. The way Ubuntu uses that feature is to deploy the entire GRUB boot loader as a single EFI executable file called grubx64.efi. This file gets executed by the UEFI firmware - either directly or through a shim (called shimx64.efi) for a Microsoft CA only Secure Boot system. This GRUB installation knows to load a small configuration file - grub.cfg from the EFI partition that contains instructions on where to find the full configuration file. Because you have the full GRUB boot loader available immediately, you can have BTRFS or ZFS drivers there that can read advanced file systems and you don't need a separate boot partition.

The problem is that all these file paths (up until after the grub.cfg file is loaded) are compiled into the GRUB EFI executable and are not configurable - and because this whole thing needs to be signed then you can't update this configuration during installation (unless you want to setup personal keys in the machine's TPM and start recompiling boot loaders). As a result the Ubuntu GRUB installation (what KDE Neon, which is based on Ubuntu LTS, is using) needs the EFI boot loader directory - where the grub.cfg is loaded from - to be EFI/ubuntu/grub.cfg, it can't be anything else because that path was compiled into the grubx64.efi file, and if the configuration file doesn't exist - you get the GRUB prompt, like I did.

We can see this by running set in the GRUB prompt: the output will have

prefix=(hd0,gpt1)/EFI/ubuntu

meaning that the prefix (where GRUB thinks it was loaded from and where it expects to see all the other files) is hardcoded to what Ubuntu uses.

When you install KDE Neon (which is what I'm using), Neon installer will create both an EFI/ubuntu and EFI/neon installations of the boot loader - I'm not sure why they even try, as the EFI/neon folder isn't even used (probably because they want the boot loader entry name to read "neon", and for some reason that means the folder must also be called "neon"? the UEFI spec doesn't require that), so that works.

What I did wrong was to assume that the EFI/ubuntu folder was some sort of legacy (I installed Neon on top of a previous Ubuntu installation) and removed it (and also re-installed the Neon bootloader into EFI/Neon, because I wanted the boot entry to be nicely capitalized) - so the GRUB executable was loaded (by the UEFI firmware) from the new folder, but once it was loaded - it tried to find the rest of the setup in EFI/ubuntu and that wasn't there. So we get dumped to the prompt to figure it out.

4
  • Since I install several Ubuntu installs, I have for years tried to create unique UEFI boot entries, but found everyone used /EFI/ubuntu/grub.cfg to boot. Originally, the unique folder in ESP, did not have a grub.cfg. Then when it did, I thought it would work, but did not. Something in Ubuntu's version is hard coded to use /EFI/ubuntu/grub and any official or unofficial flavor that uses Ubuntu's shimx64.efi or grubx64.efi will have that issue. Or I have to use one grub to boot all installs or create ESP on other drives to have a different default boot.
    – oldfred
    Commented Jul 27, 2023 at 19:28
  • So the sad situation - due to secure boot restrictions requiring all code to be precompiled and signed - is that all computers come with Microsoft CA, and though Microsoft offers a signing service - Canonical got their GRUB shim signed, everyone is just using their shim (and their GRUB whose signature is hardcoded in the shim.
    – Guss
    Commented Jul 28, 2023 at 13:11
  • To test my theory I downloaded & installed both Debian & Fedora, but did not use Secure Boot. I think they each have there own version of Secure Boot. But both let me choose which drive to install grub and added either Debian or Fedora as UEFI boot entries. So I could boot them directly from UEFI. Do not use Secure Boot, may in future.
    – oldfred
    Commented Jul 28, 2023 at 13:30
  • As a general rule, I think Secure Boot is useless and causes more issues than it helps - under most use cases (also see "kernel lockdown" if you want a horror story), but it is an extra step to take during installation and as I am supporting users in an enterprise environment - I want to run with as minimal changes to an off-the-shelf setup (less things that can go awry, and also, turning off Secure Boot does not disable Ubuntu's kernel lockdown 🙄).
    – Guss
    Commented Jul 29, 2023 at 7:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .