0

I am having an issue where I can mount my disks manually using the mount command. I than add the disks to fstab. Once I restart: sda1 points at the correct mount point (/mnt/da), but the rest do not. Please help, I am out of ideas.

Server setup:

  • 2 x nvme drives in software raid1
  • 10x 16tb drives without raid, stand alone disks (had to remove raid from these after initial setup)
  • OS: Debian 12
  • xfs file system
  • Using UUID, I get from the blkid command, to add the devices to fstab

Tried:

  • Manually mounted the disks
  • Tried mounting one disk at a time
  • Tried adding one disk at a time to fstab and reloading
  • Tried adding all the disks to fstab and reloading

df -h

root@data7 ~ # df -h
Filesystem      Size  Used Avail Use% Mounted on
udev             63G     0   63G   0% /dev
tmpfs            13G  896K   13G   1% /run
/dev/md2        875G 1013M  829G   1% /
tmpfs            63G     0   63G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
/dev/md1        989M   66M  873M   7% /boot
/dev/sdb1        15T  104G   15T   1% /mnt/db
/dev/sdd1        15T  104G   15T   1% /mnt/dc
/dev/sda1        15T  104G   15T   1% /mnt/da
tmpfs            13G     0   13G   0% /run/user/0

lsblk

root@data7 ~ # lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
sda           8:0    0  14.6T  0 disk
└─sda1        8:1    0  14.6T  0 part  /mnt/da
sdb           8:16   0  14.6T  0 disk
└─sdb1        8:17   0  14.6T  0 part  /mnt/db
sdc           8:32   0  14.6T  0 disk
└─sdc1        8:33   0  14.6T  0 part
sdd           8:48   0  14.6T  0 disk
└─sdd1        8:49   0  14.6T  0 part  /mnt/dc
sde           8:64   0  14.6T  0 disk
└─sde1        8:65   0  14.6T  0 part
sdf           8:80   0  14.6T  0 disk
└─sdf1        8:81   0  14.6T  0 part
sdg           8:96   0  14.6T  0 disk
└─sdg1        8:97   0  14.6T  0 part
sdh           8:112  0  14.6T  0 disk
└─sdh1        8:113  0  14.6T  0 part
sdi           8:128  0  14.6T  0 disk
└─sdi1        8:129  0  14.6T  0 part
sdj           8:144  0  14.6T  0 disk
└─sdj1        8:145  0  14.6T  0 part
sdk           8:160  0  57.7G  0 disk
nvme0n1     259:0    0 894.3G  0 disk
├─nvme0n1p1 259:1    0     4G  0 part
│ └─md0       9:0    0     4G  0 raid1 [SWAP]
├─nvme0n1p2 259:2    0     1G  0 part
│ └─md1       9:1    0  1022M  0 raid1 /boot
└─nvme0n1p3 259:3    0 889.3G  0 part
  └─md2       9:2    0 889.1G  0 raid1 /
nvme1n1     259:4    0 894.3G  0 disk
├─nvme1n1p1 259:5    0     4G  0 part
│ └─md0       9:0    0     4G  0 raid1 [SWAP]
├─nvme1n1p2 259:6    0     1G  0 part
│ └─md1       9:1    0  1022M  0 raid1 /boot
└─nvme1n1p3 259:7    0 889.3G  0 part
  └─md2       9:2    0 889.1G  0 raid1 /

blkid

GETTING UUID
root@data7 ~ # blkid | g sda
/dev/sda1: UUID="cea5e8d9-1ddf-4502-a609-3a17af37082c" BLOCK_SIZE="4096" TYPE="xfs" PARTUUID="d0a89050-f533-7245-877e-5006d974516c"
root@data7 ~ # blkid | g sdb
/dev/sdb1: UUID="e3ae1145-d37b-41d7-ac1f-5c6a646bd5ed" BLOCK_SIZE="4096" TYPE="xfs" PARTUUID="da1f85c1-9054-d147-a9b8-0965020b4d67"
root@data7 ~ # blkid | g sdc
/dev/sdc1: UUID="47bf4e70-ec50-4369-88c2-9dfd8dd5d422" BLOCK_SIZE="4096" TYPE="xfs" PARTUUID="f458a5c3-6b6c-054d-a069-27930dcb02f2"

fstab

/etc/fstab - WAS CORRECT BEFORE RESTART .. CHANGED IT 5+ TIMES AND BREAKS AFTER EVERY RESTART
proc /proc proc defaults 0 0
# /dev/md/0
UUID=e2f568f6-846b-4657-b88d-3c8108d5600c none swap sw 0 0
# /dev/md/1
UUID=a5868f6d-b7e5-43b1-ab81-4770a543d83a /boot ext3 defaults 0 0
# /dev/md/2
UUID=612c81e1-94e4-415e-863f-6dfcbe127dee / ext4 defaults 0 0
# /dev/sda1
UUID=cea5e8d9-1ddf-4502-a609-3a17af37082c /mnt/da xfs defaults 0 2
# /dev/sdb1
UUID=e3ae1145-d37b-41d7-ac1f-5c6a646bd5ed /mnt/db xfs defaults 0 2
# /dev/sdc1
UUID=2b28f001-d9a0-4759-8f29-4bf45a18aeb6 /mnt/dc xfs defaults 0 2

My Other Servers (in response to the comment that device names dont persist across reboots)

root@data2:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            126G     0  126G   0% /dev
tmpfs            26G  1.3G   24G   6% /run
/dev/sda3       5.5T  4.9T  246G  96% /
tmpfs           126G     0  126G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           126G     0  126G   0% /sys/fs/cgroup
/dev/sda2       923M   79M  781M  10% /boot
/dev/sde1       5.5T  5.0T  236G  96% /mnt/de
/dev/sdf1       5.5T  4.1T  1.2T  79% /mnt/df
/dev/sdd1       5.5T  4.7T  468G  92% /mnt/dd
/dev/sdc1       5.5T  5.1T   49G 100% /mnt/dc
/dev/sdb1       5.5T  5.1T   74G  99% /mnt/db
tmpfs            26G     0   26G   0% /run/user/1000
tmpfs            26G     0   26G   0% /run/user/0

root@data3:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            126G     0  126G   0% /dev
tmpfs            26G  2.5G   23G  10% /run
/dev/sda3       5.5T  4.2T  1.1T  81% /
tmpfs           126G     0  126G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           126G     0  126G   0% /sys/fs/cgroup
/dev/sda2       923M   79M  781M  10% /boot
/dev/sdc1        11T   11T  660G  95% /mnt/df
/dev/sdf1        11T  9.2T  1.8T  84% /mnt/de
/dev/sdd1        11T  9.8T  1.2T  90% /mnt/dc
/dev/sde1        11T   11T  191G  99% /mnt/dd
/dev/sdb1        11T   11T  855G  93% /mnt/db
tmpfs            26G     0   26G   0% /run/user/1001
tmpfs            26G     0   26G   0% /run/user/0


root@data4:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            126G     0  126G   0% /dev
tmpfs            26G  2.5G   23G  10% /run
/dev/sda3        11T   11T  249G  98% /
tmpfs           126G     0  126G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           126G     0  126G   0% /sys/fs/cgroup
/dev/sda2       923M   80M  781M  10% /boot
/dev/sdc1        11T  9.3T  1.7T  85% /mnt/dc
/dev/sdd1        11T  9.0T  2.0T  82% /mnt/dd
/dev/sdb1        11T  9.8T  1.2T  90% /mnt/db
tmpfs            26G     0   26G   0% /run/user/1002
tmpfs            26G     0   26G   0% /run/user/1000
tmpfs            26G     0   26G   0% /run/user/1005
tmpfs            26G     0   26G   0% /run/user/0
11
  • You have shown us the UUID for sdc1 but you don’t use it in fstab anywhere from what we can see. A different UUID is specified for sdc1.
    – PonJar
    Dec 30, 2023 at 16:55
  • Device names are random so if you expect /dev/sdc1 to always mount as /mnt/dc then that's exactly what won't work. The drive order might change with every reboot, so next time it could be /dev/sde1 (for the same drive and filesystem). Perhaps consider using LABEL for both filesystems and mountpoint names (and fstab comments) to get a less confusing result? Or you could put drive serial numbers or something. Anyway a comment like # /dev/sdc1 in fstab is rather meaningless if you have many drives... Dec 30, 2023 at 18:11
  • @frostschutz Device names are not random (at least in Debian), they are assigned in order of detection. Dec 30, 2023 at 18:22
  • I get the UUID from lsblk command. I than add that to fstab using the UIUID. The example fstab is wrong after the reboot which is where my confusion comes from. I am adding a link to a guide I used for previous servers. The device names still match to this day following this guide and I never had to setup labels or do anything else. Do the device names matter once the server is running and has data? If not I will just remove the comments from the fstab and continue as normal. contabo.com/blog/mounting-additional-hard-disks-linux
    – lessharm
    Dec 30, 2023 at 18:34
  • 1
    Then you're "lucky" on your other servers. It's possible for devices to always be detected in the same order, but you can't rely on it at all. Device names changing is normal. That's why you use UUIDs. Dec 30, 2023 at 18:48

3 Answers 3

2

The device ID's are not guaranteed to be consistent across reboots...

you are using wrong vocabulary.

There are 6 ways to mount disks in linux, as shown under /dev/disk; this is using RHEL-7.9...

by-id/ by-label/ by-partlabel/ by-partuuid/ by-path/ by-uuid/

by-id is a scsi identifier or wwn (world-wide-number) identifier.

The unreliable (or inconsistent) way is mounting by-device-name which for example is doing just /dev/sdb1 /data in /etc/fstab, which is what you had been doing and are incorrectly referring to as device id's. The id's are consistent, the device-names (which are sda,sdb,sdc and so on) are not.

You will see that everything under those six /dev/disk/ folders are all links that point up and above to the device name {/dev/sda2 for example}

as mentioned in the comments, by device-name gets mapped out after boot by order the devices are recognized. Adding a new disk connected by SATA cable, does not put it at the end of the sda,sdb,sdc... list. It often times gets put at the front as sda and then everything shifts down, which is how the inconsistency comes about. Simply swap two disks and the SATA ports they are connected to on the motherboard - same issue.

  • by-UUID is very consistent - hence the name universally unique id
  • by-id as in the scsi id or wwn id, should also be very reliable; you will often find the wwn on the label on the disk
  • by-label should be reliable, up until you do something like label multiple disks (partitions actually) with the same label name.
  • I think by-path is inconsistent, for the reason if disks get connected onto different SAT/SAS ports then that is now a different path.
2
  • by-id seems to show a drive's serial number, it's excellent if you've cloned some partitions or an entire drive with dd since the by-UUID seems to only show one result, even if there's multiple
    – Xen2050
    Jan 2 at 14:56
  • i am fuzzy with by-uuid given superuser.com/questions/1160966/… stating stating can reasonably be considered unique... on the local system. While I have never had a problem using by-uuid it seems using by-id with that being based off disk serial number or wwn (which is from disk firmware) then i would think that might be the most reliable way. UUID's can be changed with uuidgen? Far less likely to change a disk serial # or wwn in disk firmware.
    – ron
    Jan 2 at 15:07
0

AFAIK in past, the order of disk was strictly defined by detecting order: primary IDE master = hda, slave = hdb; secondary IDE master = hdc, slave = hdd. New hardware brought sATA channels, what changed the device identifier to sda, sdb, etc. Beside the changes of hardware the boot resp. power-on software was changed a little, too. Both BIOS (resp. uEFI) and the bootloader and systemboot start to use more CPU cores. You may note Debian (and others) migrated from SysV init to systemd, so you can start more independent components in parallel and not in strict order defined with "alphabetical" order in the /etc/rc3.d directory. In schrodingerscatcuriosity comment link is written:

... first hard disk detected is named /dev/sda ...

Well, that means: disks are polled in standard order with SysV init; but on systemd the disk detection is done in parallel, what means that the first detected disk need not to be the disk attached to first sATA channel. The first touch is done by BIOS (uEFI) and the BIOS ROM of additional PCI(e) card - SCSI (SAS) e.g. Hence you cannot rely on disk detection order, because rotational disks need more time to start after power-on, but after warm restart can be detected immediately. That's why the UUID and LABEL was added to identify disk partition to be mounted. So you can see in fstab (at Debian based Linux) someting like:

# / was on /dev/sda1 during installation
UUID=98765432-9876-abcd-9090-1234567890ab / ext4 ...

what you can change to

LABEL=mysda1 / ext4 ...

if you set the LABEL of the partition to mysda1. The IDE neither sATA disk do not has any standard identifier like the MAC address of NIC is.

2
  • drive detection happens in kernel as soon as drivers/modules are loaded or, for builtin drivers, as soon as those are initialized. In a way, kernel configuration has a larger influence than the init system... systemd's nature of processing things in parallel is unrelated, systemd does not decide order of drives, it just processes whatever the kernel detects... Jan 2 at 15:21
  • I do not want argue, the difference between systemd and init is an example to demonstrate the parallel boot. The parallel disk detection may be done by uEFI or kernel. My experience, I wrote, says that disk detection after power-on may differ from the warm restart. It is a great pity, the educated people do not have enough time to write articles for average users. In fact before the kernel start to deal with block devices, the boot-loader (grub) has to detect and chose the right disk and look for kernel on it. Any way: all tutorial information is welcome.
    – schweik
    Jan 2 at 17:41
0

The device names (changed from ID's) are not guaranteed to be consistent across reboots. Using UUID's in your fstab still guarantee's that your disks are consistently mounted pointing to the same path. Data you place at /mnt/da, /mnt/db ... etc will continue to reside in those places even if the device id's change.

Thanks to @frostschutz and other's in the comments.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .