Welcome to the Unix & Linux StackExchange!
I can think of several possible reasons why your Debian 9 boot option might behave as you described.
Your picture of Debian 11's GRUB indicates it has a "System setup" boot option, which indicates the UEFI version of GRUB is successfully starting. Also, you said Windows boots just fine from a GPT-formatted disk, and that can only happen in UEFI mode.
I don't think GRUB can switch from UEFI to legacy mode on its own, so your symptom of "always booting Debian 9 on Legacy Boot Mode using sda1
" probably means the UEFI version of GRUB attempts to boot the Debian 9 kernel, but fails, and then the system firmware gets to decide what to do next. If there are no other valid UEFI boot options to try, it might start trying legacy boot options, and so end up finding a bootable MBR on sda
, which will fire up a BIOS version of GRUB and ultimately Debian 9 on sda1
.
When you cloned the Debian 9 installation to nvme0n1p5
, did you change the filesystem UUID? If there are two filesystems with the same UUID, and if the firmware can boot from NVMe in UEFI mode only, that might explain why the UEFI-mode boot of Debian 9 fails but legacy mode works: the UEFI-mode bootloader will see two partitions with the UUID expected for the Debian 9's root filesystem, and the legacy-mode bootloader will only see sda1
.
Once the Linux kernel starts up, it can support NVMe drives on its own regardless of the boot mode, so you might not notice any unusual symptoms other than being unexpectedly in legacy boot mode.
Another possibility would be if your Debian 9 installation is a 32-bit version. The legacy-mode boot starts in 16-bit mode(!), and one of the first tasks of the kernel in legacy-mode boot is to deactivate the compatibility functions to uncover the 32 and 64-bit capabilities of the processor. But UEFI boots in native 64-bit mode, and I'm not sure if it even allows booting a 32-bit kernel: I know a 32-bit UEFI can boot a 64-bit kernel, but I'm not sure if the opposite is supported. If the UEFI GRUB fails to boot because it's trying to load a 32-bit kernel, that might also cause the firmware to attempt a legacy-mode boot, which ends up using sda1
.
And if you have Secure Boot enabled, Debian implemented Secure Boot compatibility only in Debian 10 and newer. (Of course, claiming to have Secure Boot enabled and then seamlessly falling back to legacy BIOS boot if the UEFI boot fails is not very secure...)
So, you'll need more information on why the UEFI boot of Debian 9 is failing. One of the first things you could do is to temporarily disabling the boot splash and enabling verbose boot messages. In the GRUB boot menu, move the highlight to the "Debian 9 on nvme0n1p5" boot option, and press the E key to (non-persistently) edit the boot entry. You'll see a number of lines. Find a line that begins with the words:
linux /boot/vmlinuz-<kernel version number here> <kernel boot options...>
If the kernel boot options include the words quiet
and/or splash
, remove them. For good measure, you might also write in the words verbose
and nosplash
. This should cause the Linux kernel to output a lot of details at boot. To boot with the temporarily altered boot options, press F10 or Control+X.
If you still see a regular Debian 9 boot splash screen and the system ends up in legacy mode, that suggests the failure and restart in legacy mode happens faster than your screen can switch display modes - and that would suggest the problem happens right when GRUB is trying to start the kernel.
Once you are in Debian 9, log in, open a command prompt and type uname -m
. Does it respond with x86_64
, or something like i386
or i686
? The first means your OS has a 64-bit kernel, the others all indicate a 32-bit version.