Process management and pkill

Question

I've been studying process management using shell scripts and I'm starting to realise how difficult it is to make sure that it's done right.

For example, you can record the PID of a program to a file, wait on it, and clean up the PID file after the program exits.

If you were to try and kill this daemon from an init script, for example, you might think of doing something like this:

do_stop() {
  kill $(</var/run/program.pid)
}

This obviously doesn't work. Between obtaining the PID and sending the kill signal, another process could have died and taken its place.

The correct way seems to require using IPC in the parent of the program to send a kill signal to its child. This will ensure that the PID of the process hasn't been reused by another.

I've been trying to write my own init scripts that are as correct as possible. In this case, I've been writing one for NRPE. NRPE unfortunately daemonizes and disowns itself to init, which means I can't wait on it. Instead, I came up with the following solution:

do_stop() {
  echo "Stopping (sending SIGTERM to) nrpe"
  pkill -u nrpe || { echo >&2 "nrpe isn't running"; exit 1; }
}

The only process that the nrpe user runs is NRPE itself, and considering the system is under my control I consider this a relatively sane solution.

What I'm curious about is the atomicity of pkill (if that's the right word to use). I assume pkill follows these steps:

1. Looks up the PID in the process table after parsing the arguments for the process criteria.
2. Sends SIGTERM (by default) to the obtained PID

Let's say pkill -u nrpe gives a PID of 42 in step 1. Is it possible that nrpe's process could die and another one could spawn in its place before step 2 occurs?

Answer 1

You are correct to suspect that there is a (small!) atomicity problem.

No matter what method you use, whether it's a system-standard utility like start-stop-daemon, a roll-your-own PID file, using pkill to query and kill by user ID, by executable binary, or whatever, there is always an interval between finding what process you want to kill and giving that process ID to the kill system call to send it a signal.

Basically, you should just not worry about it. In order to run into trouble, both of the following would have to happen:

• The target process dies between the time you identity its process ID and the time you actually kill it.
• The process IDs for newly created process would have to, during the same time interval, happen to cycle around to reuse the process ID just vacated.

It's just really unlikely.

Note that in the particular case you're studying, you actually do have a way of protecting yourself against this. The only process that the nrpe user runs is NRPE itself, so if you switch to the nrpe user (from root, probably) before issuing the kill command, you might under very unlikely circumstances try to kill a poor innocent process belonging to something else, but you won't have permission to do it and it won't have any effect.