Trigger an init script when a specific device is up and a kernel module has been loaded

Question

On my Ubuntu 12.04 servers, I need to write an init.d script that waits until the infiniband interface (device: mlx4_0, interface: ib0) is completely up, and until the knem kernel module is loaded.

I have an init.d script that works if I wait until the system is up and manually service myscript start, but it does not work if I load it normally at boot time. I'm using boot order 99, but it does not start correctly because I need it to wait until those features are running.

What is the correct syntax in an init.d to achieve that (one kernel module and )? I guess there must be some keyword like # Required-Start: $remote_fs $syslog $network but I can't find the correct ones for specific interface and kernel modules.

For background: the init script is related to SLURM, openMPI and infiniband interactions. I have compiled SLURM with Mellanox infiniband driver support, and openMPI is linked to this version of SLURM. The result is that openMPI is using directly the mellanox infiniband driver, which is much more powerful than ipob (ip over inifiniband). To do that, it needs to use the registered memory of the system, which has to be set as unlimited.

SO:

I have added a few logger outputs in my init.d script. I notice that in fact the modules are up and running. So I don't understand fully the problem. It is strange, maybe related to some environment variables needed and that are set up only in a full user space and not at init time.

The problem concerns the memlock limits set in /etc/security/limits.conf. To make it work I had to set

*             -   memlock       unlimited
root          -   memlock       unlimited

and in this way, when I launch the slurm deamon with an ssh connection everything works, and when it is the init process that starts the daemon, it is like it does not consider the rules in /etc/security/limits.conf.

Answer 1

I don't know specifically about an init.d file, but a udev rule for running a script upon device-add might look like:

ACTION=="add", ATTRS{idVendor}=="VID", ATTRS{idProduct}=="PID", RUN+="/path/to/executable"

You should do some digging in udevadm to learn more about the way the device is typically added and its modules loaded. You'll also find the correct values for VID and PID in there.

Does it accept a combination of both device and kernel module?

Yes. Well... no. Maybe? The answer to this question entirely depends on the level at which you intercept the device-add. udev has to do many things from the point at which it initially detects a device to the point at which it has fully loaded and initialized the device and it can be considered up.

Some of these things are likely:

• initially detecting the hardware on its parent bus/device/subsystem

• finding and loading the appropriate kernel module

• populating the /dev devfs file system with the appropriate device special file

• detecting any child-devices the current device adds and rinse, repeat

You can specify a rule for all or any of these levels of action. You can also udevadm trigger or monitor or several other things directly in real-time to determine exactly what these levels of action might be.

I recommend you look closely at this information if you wish to become more familiar with it.

Answer 2

Use an Upstart job for this:

# Ensures that the device is up and filesystem is too
start on filesystem and net-device-up IFACE=ib0
stop on runlevel [016]

# Ensures that module is loaded
pre-start exec modprobe -q knem

exec /path/to/exec