3

I have 2 (soon to be more) independent OSes. Right now it is just CentOS 6.6 and 7, but will be adding several more.

I can boot CentOS 7 cleanly, but 6.6 will not boot. The error is:

error: no such device (long UUID here)
error: unknown filesystem

FYI, the disk is GPT, partitioned with gdisk/parted. There is a /boot for 7 on /dev/sda2 and another one for 6.6 on /dev/sda9. Both have the full /boot/grub2/ directory (not that it should matter), and identical grub2.cfg.

The working 7 one (sda2) is formatted xfs. The unsuccessful 6.6 one (sda9) fails with ext2, ext4 or xfs.

Going into grub command-line, I try to list the partitions. It sees all of the partitions, but cannot see a filesystem on anything but sda2:

grub> ls (hd0,2)
      Partition hd0,2: Filesystem type xfs, UUID 97df.... - Partition start at 2048 KiB - Total size 512000 KiB
grub> ls (hd0,9)
      Partition hd0,9: No known filesystem detected - Partition start at 279393720 KiB - Total size 495616 KiB

Booting into 7, I can mount sda9 directly, all looks good.

Partition table from parted:

(parted) p
Model: DELL PERC H730 Mini (scsi)
Disk /dev/sda: 8397GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name            Flags
 1      1049kB  2097kB  1049kB                               bios_grub
 2      2097kB  526MB   524MB   xfs
 3      526MB   537GB   537GB                                lvm
 4      537GB   1074GB  537GB                                lvm
 5      1074GB  1504GB  430GB                                lvm
 6      1504GB  1933GB  430GB                                lvm
 7      1933GB  2324GB  391GB                                lvm
 8      2324GB  2861GB  537GB                Linux LVM       lvm
 9      2861GB  2861GB  508MB   xfs          centos_66_boot
10      2861GB  2961GB  99.5GB               centos_66_root  lvm
11      2961GB  2971GB  9999MB               smart_data1
12      2971GB  3071GB  100GB                smart_data2

(nothing on the smart_* partitions, to be used for alternate app stuff).

If you prefer from gdisk:

Disk /dev/sda: 16401301504 sectors, 7.6 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 5B41D43E-6D17-4E34-9068-E66BD3753D70
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 16401301470
Partitions will be aligned on 2048-sector boundaries
Total free space is 10403256253 sectors (4.8 TiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048            4095   1024.0 KiB  EF02
   2            4096         1028095   500.0 MiB   0700
   3         1028096      1049612287   500.0 GiB   8E00
   4      1049612288      2098196479   500.0 GiB   8E00
   5      2098196480      2937065471   400.0 GiB   8E00
   6      2937065472      3775934463   400.0 GiB   8E00
   7      3775934464      4539299839   364.0 GiB   8E00
   8      4539299840      5587875839   500.0 GiB   8E00  Linux LVM
   9      5587875840      5588867071   484.0 MiB   0700  centos_66_boot
  10      5588867072      5783203839   92.7 GiB    8E00  centos_66_root
  11      5783203840      5802733567   9.3 GiB     0700  smart_boot
  12      5802733568      5998047231   93.1 GiB    BF00  smart_data

Relevant parts of /boot/grub2/grub.cfg:

menuentry 'CentOS Linux 7 (Core), with Linux 3.10.0-229.14.1.el7.x86_64' --class rhel fedora --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-3.10.0-229.14.1.el7.x86_64-advanced-d171cb6d-c39b-45d5-b09a-33f641c3f397' {
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_gpt
        insmod xfs
        set root='hd0,gpt2'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2 --hint='hd0,gpt2'  97dfd142-3423-459a-a39e-9a6099b3e5fc
        else
          search --no-floppy --fs-uuid --set=root 97dfd142-3423-459a-a39e-9a6099b3e5fc
        fi
        linux16 /vmlinuz-3.10.0-229.14.1.el7.x86_64 root=/dev/mapper/root-lv_root ro rd.lvm.lv=root/lv_root rd.lvm.lv=centos/swap crashkernel=auto rhgb quiet
        initrd16 /initramfs-3.10.0-229.14.1.el7.x86_64.img
}
    menuentry 'CentOS release 6.6 (Final) (on /dev/mapper/vg_centos6-lv_centos6_root)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux-simple-b5002a5f-a521-423c-bae5-82e0aefe5e17' {
        insmod part_gpt
        insmod xfs
        set root='hd0,gpt9'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt9 --hint-efi=hd0,gpt9 --hint-baremetal=ahci0,gpt9 --hint='hd0,gpt9'  5715cdc8-9d49-49c3-814f-eb887d69b7d4
        else
          search --no-floppy --fs-uuid --set=root 5715cdc8-9d49-49c3-814f-eb887d69b7d4
        fi
        linux /vmlinuz-2.6.32-504.el6.x86_64 root=/dev/dm-5
        initrd /initramfs-2.6.32-504.el6.x86_64.img
}

I also tried alternate partition codes for sda9, nothing works. What am I missing?

For what it's worth, grub2 version:

# rpm -qi grub2
Name        : grub2
Epoch       : 1
Version     : 2.02
Release     : 0.17.0.1.el7.centos.4
Architecture: x86_64
Install Date: Mon 26 Oct 2015 05:48:42 PM IST
Group       : System Environment/Base
Size        : 7394739
License     : GPLv3+
Signature   : RSA/SHA256, Wed 23 Sep 2015 07:13:19 PM IDT, Key ID 24c6a8a7f4a80eb5
Source RPM  : grub2-2.02-0.17.0.1.el7.centos.4.src.rpm
Build Date  : Wed 23 Sep 2015 05:51:07 PM IDT
Build Host  : kbuilder.dev.centos.org
Relocations : (not relocatable)
URL         : http://www.gnu.org/software/grub/
Summary     : Bootloader with support for Linux, Multiboot and more

At request of commenter, output:

# file -s /dev/sda9
/dev/sda9: SGI XFS filesystem data (blksz 4096, inosz 256, v2 dirs)
# file -s /dev/sda2
/dev/sda2: SGI XFS filesystem data (blksz 4096, inosz 256, v2 dirs)

And grub2-probe seems content with both:

# grub2-probe --device /dev/sda2
xfs
# grub2-probe --device /dev/sda9
xfs

grub2-fstest also seems to return cleanly for those partitions:

# grub2-fstest -vvv -r /dev/sda2 /dev/sda ls -- -l
Device proc: Filesystem type procfs - Sector size 512B - Total size 0KiB
Device loop0: No known filesystem detected - Sector size 512B - Total size 8200650752KiB
    Partition loop0,gpt12: No known filesystem detected - Partition start at 2901366784KiB - Total size 97656832KiB
    Partition loop0,gpt11: No known filesystem detected - Partition start at 2891601920KiB - Total size 9764864KiB
    Partition loop0,gpt10: No known filesystem detected - Partition start at 2794433536KiB - Total size 97168384KiB
    Partition loop0,gpt9: Filesystem type xfs, UUID b5c911d7-4f55-46da-80fd-d956bad72234 - Partition start at 2793937920KiB - Total size 495616KiB
    Partition loop0,gpt8: No known filesystem detected - Partition start at 2269649920KiB - Total size 524288000KiB
    Partition loop0,gpt7: No known filesystem detected - Partition start at 1887967232KiB - Total size 381682688KiB
    Partition loop0,gpt6: No known filesystem detected - Partition start at 1468532736KiB - Total size 419434496KiB
    Partition loop0,gpt5: No known filesystem detected - Partition start at 1049098240KiB - Total size 419434496KiB
    Partition loop0,gpt4: No known filesystem detected - Partition start at 524806144KiB - Total size 524292096KiB
    Partition loop0,gpt3: No known filesystem detected - Partition start at 514048KiB - Total size 524292096KiB
    Partition loop0,gpt2: Filesystem type xfs, UUID 97dfd142-3423-459a-a39e-9a6099b3e5fc - Partition start at 2048KiB - Total size 512000KiB
    Partition loop0,gpt1: No known filesystem detected - Partition start at 1024KiB - Total size 1024KiB

Help?

7
  • 1
    What is the reason for using xfs on boot parts?
    – user140866
    Commented Nov 29, 2015 at 6:37
  • 1
    No particular reason. The CentOS 7 install was first, happened to use xfs on its boot partition sda2. When I did sda9 and it refused to see it, I tried ext4, ext2, xfs. None worked.
    – deitch
    Commented Nov 29, 2015 at 6:59
  • 1
    @siblynx please tell me you have some idea what is going on here?
    – deitch
    Commented Nov 29, 2015 at 7:10
  • 1
    I have no any particular idea other than using filesystems native to Linux kernel. For /boot ext2 is more than sufficient, and ext2 is most widespread and well tested. There is a chance you'd hit a grub2 bug (for bootoader, grub2 is bloated thing). I use LILO, and it might work for you too if you're in fire. LILO does not try to parse filesystems, but just records at which offset which kernel is present and tries to directly boot them.
    – user140866
    Commented Nov 29, 2015 at 8:20
  • 1
    I thought of that, but grub2 seems perfectly content to see the filesystem on sda2. I am tempted to switch, but I know this bug will come up again at some point. Better to solve it now.
    – deitch
    Commented Nov 29, 2015 at 8:31

1 Answer 1

1

You're using a system disk that is way bigger than 2 TiB - and the presence of a bios_grub partition and absence of an EFI System Partition indicate that you are using legacy BIOS boot mode, even though you are using GPT partitioning.

Your sda9 partition is beyond the first 2 TiB of the disk.

grub2-probe seems to recognize the filesystems just fine, and that uses the same codebase as the filesystem drivers in the actual installed GRUB bootloader. That strongly suggests it isn't an issue of GRUB not understanding CentOS 6.x's XFS.

So, the XFS detection routine from GRUB's codebase works fine when it's relying on the Linux kernel for disk access, but fails when running on top of the system firmware at boot time. Your system firmware seems not to support access beyond the first 2 TiB using legacy BIOS functions. That's probably why GRUB is failing to find the boot partition of CentOS 6.

Getting rid of the 2 TiB limit is precisely one of the major reasons why GPT partitioning and UEFI were developed for x86 architecture. Booting from a >2TiB disk using legacy BIOS and GPT partitioning is a corner case that is not supported by Windows (still the "commercially dominant" OS on the x86 platform), so the hardware vendors won't test it. You might be doing something completely new and untested with your hardware here.

As far as I can see, your options are:

  • if you want to stay with BIOS-style booting, you must arrange for the /boot partitions (or equivalents) of all your OSes to be within the first 2 TiB of the disk.
  • or you might transition to native UEFI-style booting, create a larger-than-minimum EFI System Partition (ESP for short), and have all your OSs install their UEFI bootloaders in there. The UEFI specification explicitly allows that: each OS is supposed to install their bootloader to <ESP root>/EFI/<vendor_or_distribution_name>/ sub-directory within the ESP.

If all the OSs you're planning to install will support UEFI, I would recommend the second option. You'll need to learn some new things and unlearn some old "truths", but you'll get to use your system without needing to think about that pesky old 2 TiB limit at all.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .