Why does `mount /dev/dm-8` mount `/dev/dm-39` instead?

Question

On a debian 10 GNU with Linux kernel system the following line does not - as expected - mount /dev/dm-8 but (imho incorreclty) /dev/dm-37:

mount /dev/dm-8 /mnt

I am puzzled really now the reason and/or how to I further investigate this issue, I assume to be in lay inside the UUID associated with filesystem on block devices within the linux kernel.

Background info and own research:

1. general info:

   root@ada:/# which mount
   /usr/bin/mount
   root@ada:/# file /usr/bin/mount
   /usr/bin/mount: setuid ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNd
   root@ada:/# uname -a
   Linux ada 4.19.0-12-amd64 #1 SMP Debian 4.19.152-1 (2020-10-18) x86_64 GNU/Linux

2. Logical Volume Manager is used. The one and only volume group is named vg.

3. The two volumes linked to the block devices contain a btrfs and since they resulted from snapshots they share the same fsid (UUID). Both logical volumes are set active.

4. the block devices /dev/dm-8 and /dev/dm-37 are connected to those logical volumes:

   • /dev/dm-8 -> /dev/vg/vm_docu
   • /dev/dm-37 -> /dev/vg/vm_mail

root@ada:/# realpath /dev/vg/vm_docu ; file -s /dev/dm-8 ;
/dev/dm-8
/dev/dm-8: BTRFS Filesystem sectorsize 4096, nodesize 16384, leafsize 16384, UUID=d8709bb6-8278-431a-95dd-211ceaf35c3d, 951652352/214748364800 bys
root@ada:/# realpath /dev/vg/vm_mail ; file -s /dev/dm-37 ;
/dev/dm-37
/dev/dm-37: BTRFS Filesystem sectorsize 4096, nodesize 16384, leafsize 16384, UUID=d8709bb6-8278-431a-95dd-211ceaf35c3d, 1599131648/21474836480 bys

5. via looking at /usr/bin/mount with strace I can see this mount(8) system call be issued:

mount("/dev/mapper/vg-vm_mail", "/mnt", "btrfs", 0, NULL) = 0

which would suggest that /dev/dm-8 while not literally forwared to the system call by /usr/bin/mount yet was not misconcepted eiter since

 
root@ada:/# realpath /dev/mapper/vg-vm_mail 
/dev/dm-8

In the list of suspects of tools/software involved that would stumple over the duplicate UUID here I include:

• udev
• linux kernel - btrfs code
• linux kernel - device mapper
• linux kernel - lvm (if that is not already device mapper internally)
• systemd

Answer 1

BTRFS filesystems must never have duplicate UUIDs.

Really, it's that simple. Btrfs uses the filesystem UUID to identify which devices belong to the same file system. This is so that you can have a FS composed of /dev/sda, /dev/sdb and /dev/sdc, yet you don't need any special command to "assemble the array" before mounting. You just run mount /dev/sda /somewhere and BTFRS will automatically detect which devices share the same UUID with /dev/sda and assemble the filesystem correctly.

This is for example why you must never use block-level copies of btrfs devices or LVM snapshots. Anything that leads to multiple "clones" of the same block device (having the same UUID) is bound to confuse the kernel and lead to a world of hurt.

So however you got into this situation with duplicate filesystem UUIDs, it's going to be fairly hard to recover from it. It would likely be easiest to recreate the affected filesystems from scratch and restore from a backup.