0

I need to monitor a daemon every few minutes with a command and analyse both the exit code and the (multi-line) output.

In case there is a problem I want to be notified one time, but I don't want to keep receiving spam until the problem is fixed, so I also want to suppress the notifications for a few hours. In the meantime, all results are recorded in a database on another server.

I've considered SEC - Simple Event Correlator or a cron job running a batch script, but I see a few problems with either approach.

  • I have a (mostly) working Bash script that I'm testing with cron, using env variables to keep the time of the last sent notification. This approach got complicated when I couldn't find a way to make the env variables persist between successive runs of the cron job. Maybe I could write the time variables to disk; but I need to set a few env variables and handling files and line substitutions just seems like an ugly and complicated solution.

  • SEC on the other hand, handles the whole notification suppression bit beautifully, but I can't find a way to have it analyse the exit codes from spawn actions. Less importantly, it seems a bit cumbersome handling multi-line outputs, needing some convoluted code for something that in bash is fixed with double quotes.

What would be a good solution for this sort of situation? Maybe Ruby/YAML files? I'm not experienced with that. I'm open to alternatives.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.