The given constraints are:

  1. System: Oracle 12.2. ASM cluster
  2. Must use AD/LDAP based users and groups for software and ASM disk device ownership

The oracle ASM cluster uses a number of raw block devices, which must be owned by the Oracle software owner (grid) and admin group (asmadmin). This is usually accomplished by creating udev rules like this one:

ENV{DM_UUID}=="<dm_uuid>", SYMLINK+="oracleasm/asmsysdga", OWNER="grid", GROUP="asmadmin", MODE="0660"

This works well when the user and group are local. When they are AD users, testing the rule with udevadm test also works well, but on reboot udev is triggered well before the AD/LDAP integration is available and resolving the username understandably fails. As a result, the ASM devices end up owned by root:root instead.

UDEV does not attempt to resolve the username after the rules are loaded, so we need a reliable way to make udev reload the rules and reapply the rule set to the ASM devices after sssd.service has started and AD/LDAP integration is available, and before the oracle-ohasd.service starts.

It has been suggested to simply edit the oracle supplied script to include:

udevadm control --reload
udevadm trigger ...

before the cluster proper starts. However, that seems to be less appropriate as the script is vendor provided and could be replaced by them anytime (via a patch). And it is a kludge that is easily forgotten.

Is there a better and more supported way in systemd to reload the udev rules during boot-up at the point where AD integration works, and before oracle-ohasd starts?

  • 1
    You should at least consider adding the users locally. If you google for this issue, you'll see that advice often, for instance here in the Arch Linux wiki. There's really no problem with the user being defined in both places. As long as their definition is stable enough (but then, if those users are changing, you have other problems.) And as long as it's just a handful of users and groups. Particularly if it's only user grid and group asmadmin like in your case... Anyways, at least take that into consideration. – filbranden Nov 29 '18 at 2:54
  • 1
    I could not agree more. In fact Oracle explicitly advises not to use AD based users for this purpose. Sadly though, I have no say in the matter - we have to live with it and work around. – Tony Nov 29 '18 at 5:45
  • If you know it's a work around, having an additional kludge with an extra script doesn't sound like a "dirty" solution. I don't know your exact setup, but I suppose it should be possible to find a systemd node for the AD/LDAP service, and insert your own service with a condition to run after AD/LDAP and before oracle-ohasd.service. That service would just run the udevadm reload. As it's an additional systemd configuration file, it won't get overwritten by upgrades. – dirkt Nov 29 '18 at 8:02

Implementing the Kludge

If you really want to go the route of the workaround asking udev to re-evaluate its rules after AD/LDAP is reachable, then I guess my suggestion would be to use a dedicated service unit to run those udevadm commands.

Order that unit appropriately so that it runs after sssd.service is up (since that's what's querying your AD/LDAP backend) and also after network-online.target (to ensure it will be reachable, assuming sssd doesn't have a cache of those users/groups yet) and before the service that needs those devices. (For illustration here, let's assume it's called oracle-asm.service.)

To do so, create a new service unit, such as /etc/systemd/system/asm-device-ownership.service with contents such as:

Description=Update ownership of Oracle ASM devices
After=sssd.service network-online.target

ExecStart=/bin/udevadm control --reload
ExecStart=/bin/udevadm trigger --settle ... (your devices here) ...


Once that unit is in place, make systemd aware of it with command sudo systemctl daemon-reload and enable it with sudo systemctl enable asm-device-ownership.service.

Also notice I'm recommending using udevadm trigger --settle, in order to make that command synchronous. You want to make sure the operation is waited for, since the systemd ordering will ensure this service unit (setting up ownership of those devices) is finished before the one that depends on it (the one using the ASM devices) will start.

This should work, but there are a lot of moving parts at early reboot and reloading udev and retriggering rules in the middle of it might prove to have issues... So if you go this route, make sure you test this setup enough to make sure it's actually reliable.

Having said that, I really recommend against using this kludge.

Instead, consider one of the alternative solutions below:

Replicating user/group locally

In the comments, I had previously recommended creating the same users locally, as described in this article in the Arch Linux wiki.

There's not really much of a problem in having the user in both local /etc/passwd and /etc/group and also in AD/LDAP, considering the definitions in AD/LDAP will not change (but then, if they do change, you will probably have trouble anyways, particularly with the systems that are already running.)

I can see one problem with the local copy, which is if you want to manage the members of the asmadmin group on AD/LDAP and that information is likely to actually change (so duplicating it in local /etc/group is a problem.)

It turns out glibc 2.24 has a solution for that, by specifying that entries should be merged in /etc/nsswitch.conf:

passwd: files sssd
group:  files [SUCCESS=merge] sssd

(See the man page for nsswitch.conf for details on [SUCCESS=merge].)

But this option has some drawbacks:

  • The nsswitch.conf option to merge entries is only available starting on glibc 2.24, which is fairly recent. For instance, RHEL 7 ships version 2.17, so that's not available there.
  • The "merge" feature is not perfect, the man page for instance describes how it doesn't work with getgrent which is used to list all groups...
  • Using "merge" means sssd will be attempted for resolution even for groups which are present in local files! This potentially means longer delays in early boot, before sssd is actually up... (In fact, most of the solutions involving replicating users/groups locally are trying to solve the problems of the timeouts before the external services are available.)

So perhaps a better solution is:

Encode numeric UID/GID in udev configuration instead!

As mentioned before, I expect your user grid will have a fixed UID which will not change in time (since changing it during system operation would likely create more problems) and same for GID of asmadmin.

If that's indeed the case, if you can rely on that UID and GID being fixed and dedicated, then consider encoding those in the udev rule, which can take numeric UIDs and GIDs in the OWNER and GROUP fields.

For example, if grid has UID 501 and asmadmin has GID 601, then use the following rule instead:

ENV{DM_UUID}=="<dm_uuid>", SYMLINK+="oracleasm/asmsysdga", OWNER="501", GROUP="601", MODE="0660"

This way, no user/group resolution is needed early in boot when udev is setting up these devices, but then when user/group resolution from AD/LDAP is available, the ownership will be correctly set for ASM to work with those devices.

That's probably the best solution for this problem, really no significant drawbacks there.

  • Thanks, I did not know that that rule also takes UID/GID. I don't necessarily want to implement a kludge, which is why I posted the question, failing to find any comprehensive documentation on either udevd (and it's rules) or systemd. Do you by any chance have a link to such documentation? – Tony Dec 3 '18 at 1:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.