We are seeing an issue using monit during a reboot or shutdown sequence. First some background of the some of the configuration of monit.

  1. Monit version is 5.5
  2. Monit is started from inittab as null::respawn:/usr/bin/monit –Ic /etc/monitrc
  3. Monit is configured to check services every 30 secs.
  4. Monit is monitoring certain custom applications and restarts them if they are not running by calling the init script of that particular custom application.
  5. The applications have their own init script in /etc/init.d/ folder that are started on bootup. Monit starts after these init scripts are run from inittab.

Now to the problem:

During a shutdown/reboot of the system first a command is sent to monit to stop monitoring all services. After this all the kill scripts in /etc/init.d/ are called. For the most part it works but there is a corner case where monit starts an application when the application has already shutdown gracefully, which is wrong. I have tried to capture the problem in the Sequence diagram below.

Sequence Diagram

I looked at monit source code and it looks like when the check for a service fails monit tries to run the command mentioned by forking and executing it. So this forked process can still be in progress when monit receives a unmonitor command. It also looks like monit does not stop processes that it started when it receives this command and immediately returns.

Is there a way to wait till all the child processes that are started by monit are complete?

Any other recommendation to avoid this problem?

  • SO is for programming questions. Unix tech support belongs in unix.SE or SuperUser. – Barmar Nov 17 '16 at 18:01
  • I' would start by checking if is possible that when "a command is sent to monit to stop monitoring all services" a flag gets created to avoid the " forking and executing" of the failed process... also I' would try to create a log of the previously forked so I can check if they are running and kill them. Just ideas from the top of my head... – DIEGO CARRASCAL Nov 17 '16 at 18:09
  • @Barmar, Im new to this however I have flagged this post hope the moderator can move this to unix stack exchange – Shankar SM Nov 17 '16 at 21:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.