Why does udev concatenate two symlinks?

Question

My distribution is SLES 12 SP 2.

I work with DRBD (distributed replicated block device) right now. Here is an example of a configuration for a drbd resource.

resource HA1dat {
  device /dev/drbd1;
  disk /dev/mqdat/HA1;
  meta-disk internal;
  on srv0 {
    address 192.168.174.10:7000;
    node-id 0;
  }
  on srv1 {
    address 192.168.174.11:7000;
    node-id 1;
  }
  on srv9 {
    address 192.168.174.19:7000;
    node-id 2;
  }
  connection-mesh {
    hosts srv0 srv1 srv9;
  }
}

After activating this resource as a drbd device, normally the following objects are created.

brw-rw---- 1 root disk 147, 1 Feb 13 19:41 /dev/drbd1
lrwxrwxrwx 1 root root 14 Feb 13 19:41 /dev/drbd/by-res/HA1dat/0 -> ../../../drbd1
lrwxrwxrwx 1 root root 14 Feb 13 19:41 /dev/drbd/by-disk/mqdat/HA1 -> ../../../drbd1

These are the block device itself and two symbolic links to it.

This is the correct behaviour!

On some other servers (same distribution) the behaviour is different. The configuration of the drbd resource is very similiar.

resource LN0N001Edat {
  device /dev/drbd1;
  disk /dev/data1vg/LN0N001E_lv;
  meta-disk internal;
  on sedcmmwd0030 {
    address xxx.yyy.zzz.83:7000;
    node-id 0;
  }
  on sedcmmwd0040 {
    address xxx.yyy.zzz.99:7000;
    node-id 1;
  }
  on sedcmmwd0050 {
    address xxx.yyy.zzz.100:7000;
    node-id 2;
  }
  connection-mesh {
    hosts sedcmmwd0030 sedcmmwd0040 sedcmmwd0050;
  }
}

After activating this resource, only the block device and one link are created.

brw-rw---- 1 root disk 147, 1 Mar 2 09:49 /dev/drbd1
lrwxrwxrwx 1 root root 23 Mar 2 09:49 /dev/drbd/by-res/LN0N001Edat/0_drbd/by-disk/data1vg/LN0N001E_lv -> ../../../../../../drbd1

This is the problem! This link is a concatenation of the two expected links. I have no idea, why they are concatenated.

The link is created by udev. The following output show differences, what udev does on these nodes.

Server with correct behaviour

juser@srv0:~> udevadm info /dev/drbd1
P: /devices/virtual/block/drbd1
N: drbd1
S: drbd/by-disk/mqdat/HA1
S: drbd/by-res/HA1dat/0
E: DEVICE=drbd1
E: DEVLINKS=/dev/drbd/by-res/HA1dat/0 /dev/drbd/by-disk/mqdat/HA1
E: DEVNAME=/dev/drbd1
E: DEVPATH=/devices/virtual/block/drbd1
E: DEVTYPE=disk
E: MAJOR=147
E: MINOR=1
E: SUBSYSTEM=block
E: SYMLINK=drbd/by-res/HA1dat/0 drbd/by-disk/mqdat/HA1
E: TAGS=:systemd:
E: USEC_INITIALIZED=12263844870

Server with the problem

root@sedcmmwd0030:/root : udevadm info /dev/drbd1
P: /devices/virtual/block/drbd1
N: drbd1
S: drbd/by-res/LN0N001Edat/0_drbd/by-disk/data1vg/LN0N001E_lv
E: DEVICE=drbd1
E: DEVLINKS=/dev/drbd/by-res/LN0N001Edat/0_drbd/by-disk/data1vg/LN0N001E_lv
E: DEVNAME=/dev/drbd1
E: DEVPATH=/devices/virtual/block/drbd1
E: DEVTYPE=disk
E: MAJOR=147
E: MINOR=1
E: SUBSYSTEM=block
E: SYMLINK=drbd/by-res/LN0N001Edat/0 drbd/by-disk/data1vg/LN0N001E_lv
E: TAGS=:systemd:
E: USEC_INITIALIZED=1212108486973

The SYMLINK lines are still both correct. The differences start with the DEVLINKS lines.

The rules for drbd are the same in both cases cat /usr/lib/udev/rules.d/65-drbd.rules # This file contains the rules to create named DRBD devices.

SUBSYSTEM!="block", GOTO="drbd_end"
KERNEL!="drbd*", GOTO="drbd_end"

IMPORT{program}="/sbin/drbdadm sh-udev minor-%m"

# Use symlink from the environment if available
ENV{SYMLINK}!="", SYMLINK="$env{SYMLINK}", GOTO="have_symlink"

# Legacy rules for older DRBD 8.3 & 8.4 when drbdadm sh-udev did not yet export SYMLINK
ENV{DISK}!="", SYMLINK+="drbd/by-disk/$env{DISK}"
ENV{RESOURCE}!="", SYMLINK+="drbd/by-res/$env{RESOURCE}"

LABEL="have_symlink"

ENV{DEVICE}=="drbd_?*", SYMLINK+="$env{DEVICE}"

LABEL="drbd_end"

Does anybody have an explanation for the creation of the errorneous link?

Answer 1

The udev rules for DRBD come from the drbd-utils. What version of DRBD utils do you have installed?

The newest version, 9.2.2, includes additional rules with a comment from the devs that looks like it might address your issue:

# This file contains the rules to create named DRBD devices.

SUBSYSTEM!="block", GOTO="drbd_end"
KERNEL!="drbd*", GOTO="drbd_end"

IMPORT{program}="@sbindir@/drbdadm sh-udev minor-%m"

# Use symlink from the environment if available
# some udev version thought it was a good idea to change a long established
# default of string_escape=none to string_escape=replace :-/
# therefore, recent enough drbdadm will no longer export space separated lists.
ENV{SYMLINK_BY_DISK}!="", SYMLINK+="$env{SYMLINK_BY_DISK}"
ENV{SYMLINK_BY_RES}!="", SYMLINK+="$env{SYMLINK_BY_RES}", GOTO="have_symlink"
ENV{SYMLINK}!="", OPTIONS+="string_escape=none", SYMLINK="$env{SYMLINK}", GOTO="have_symlink"

# Legacy rules for older DRBD 8.3 & 8.4 when drbdadm sh-udev did not yet export SYMLINK
ENV{DISK}!="", SYMLINK+="drbd/by-disk/$env{DISK}"
ENV{RESOURCE}!="", SYMLINK+="drbd/by-res/$env{RESOURCE}"

LABEL="have_symlink"

ENV{DEVICE}=="drbd_?*", SYMLINK+="$env{DEVICE}"

LABEL="drbd_end"