3

I've been studying process management using shell scripts and I'm starting to realise how difficult it is to make sure that it's done right.

For example, you can record the PID of a program to a file, wait on it, and clean up the PID file after the program exits.

If you were to try and kill this daemon from an init script, for example, you might think of doing something like this:

do_stop() {
  kill $(</var/run/program.pid)
}

This obviously doesn't work. Between obtaining the PID and sending the kill signal, another process could have died and taken its place.

The correct way seems to require using IPC in the parent of the program to send a kill signal to its child. This will ensure that the PID of the process hasn't been reused by another.

I've been trying to write my own init scripts that are as correct as possible. In this case, I've been writing one for NRPE. NRPE unfortunately daemonizes and disowns itself to init, which means I can't wait on it. Instead, I came up with the following solution:

do_stop() {
  echo "Stopping (sending SIGTERM to) nrpe"
  pkill -u nrpe || { echo >&2 "nrpe isn't running"; exit 1; }
}

The only process that the nrpe user runs is NRPE itself, and considering the system is under my control I consider this a relatively sane solution.

What I'm curious about is the atomicity of pkill (if that's the right word to use). I assume pkill follows these steps:

  1. Looks up the PID in the process table after parsing the arguments for the process criteria.
  2. Sends SIGTERM (by default) to the obtained PID

Let's say pkill -u nrpe gives a PID of 42 in step 1. Is it possible that nrpe's process could die and another one could spawn in its place before step 2 occurs?

  • 1
    What distro are you using? You're reinventing the wheel with distros that provide systemd or upstart for example. Even within SysV style init scripts there are often "facilities" available to you to do most of the heavy lifting so there's typically no need to do these things yourself, unless you're just learning about it. – slm Jul 3 '14 at 13:34
  • @slm Just learning - I'm aware of utilities like daemon and start-stop-daemon. systemd provides ExecStart and ExecStop, which I assume are required. If your application doesn't provide a "--shutdown" flag or similar, as far as I know you would have to provide your own scripts for starting/stopping the process? – AlephBeth Jul 3 '14 at 13:51
  • Yes I believe that's correct. – slm Jul 3 '14 at 15:23
1

You are correct to suspect that there is a (small!) atomicity problem.

No matter what method you use, whether it's a system-standard utility like start-stop-daemon, a roll-your-own PID file, using pkill to query and kill by user ID, by executable binary, or whatever, there is always an interval between finding what process you want to kill and giving that process ID to the kill system call to send it a signal.

Basically, you should just not worry about it. In order to run into trouble, both of the following would have to happen:

  • The target process dies between the time you identity its process ID and the time you actually kill it.
  • The process IDs for newly created process would have to, during the same time interval, happen to cycle around to reuse the process ID just vacated.

It's just really unlikely.

Note that in the particular case you're studying, you actually do have a way of protecting yourself against this. The only process that the nrpe user runs is NRPE itself, so if you switch to the nrpe user (from root, probably) before issuing the kill command, you might under very unlikely circumstances try to kill a poor innocent process belonging to something else, but you won't have permission to do it and it won't have any effect.

  • Thanks! The unprivileged user trick is a great suggestion. I'm glad to see my theory was correct. I'd like to write my scripts to be as logically sound as possible, taking account for (admittedly extreme) edge cases. At least now I'm happy to say that there's no way to be sure without being the parent, or being the process itself. The fact that pkill does this and that it's not at least briefly mentioned in the man pages slightly concerns me. People should be aware that it's not always going to kill the same process. Would I be crazy to send a bug report and ask to get it added to the docs? – AlephBeth Jul 3 '14 at 21:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.