5

I would like to monitor a log file for errors and then send an email to administrators.

The log file contains data like below

11 Aug 02:30 Service1 restarted
11 Aug 05:35 Service1 restarted
11 Aug 08:43 Service2 restarted
11 Aug 11:20 Service1 restarted
11 Aug 14:53 Service2 restarted

I would like to create a script which runs for every 5 minutes and checks the last occurrence of service restart and send an email.

For example : if the script runs at 02:35 it sees that Service1 restarted so it will send an email like Service1 restarted at 02:30 . Now when the script runs at 5:45 then it should send email that Service1 restarted at 05:35 only (should not include 02:35 restart)

Is there a way to achieve this requirement ? I am basically new to Linux and Shell scripting

  • What shell you're using? Is that the real log output or each line starts with the date? – tachomi Aug 11 '15 at 13:21
  • Do you know logcheck? – sebix Aug 21 '15 at 18:20
4

Start by making a five minute crontab:

*/5 * * * * myscript.sh

Which runs myscript.sh (in $HOME dir)

 #!/bin/bash
 tail -1 /path/to/file.log > /some/dir/after
 if cmp -s /some/dir/after /some/dire/before
 then
     cat /some/dir/after | mail -s "restart" admin@exemple.com
     cp /some/dir/after /some/dir/before
 fi

With the correct values (of course).

Note that this implies there will not be two restarts within five minutes.

3

I assume that at 02:35, the last line of the file will be the 02:30 restart, so all you need is a cron entry that looks like

*/5 * * * * tail -1 /path/to/file.log | mailx -s "Last restart" admin@example.com

The downside of this approach is that the admin will receive the same email at 02:35, 02:40, 02:45, ...

  • @Archemar , @ tachomi Thank you very much for your valuable suggestions , I have created a small script and it works fine. – YRVK Aug 14 '15 at 13:14
  • @YRVK, please read unix.stackexchange.com/help/someone-answers – glenn jackman Aug 14 '15 at 13:19
  • Sorry missed it , thanx for letting me know – YRVK Aug 20 '15 at 7:15
2

Try this:

crontab

*/5 * * * * /path/to/checklog.bash

Script

#!/bin/bash

service=$(tail -1 /path/to/logfile | awk '{print $4}')
logdate=$(tail -1 /path/to/logfile | awk '{print $1,$2,$3}')

if  test 'find "/path/to/logfile" -mmin -5'
then
    echo "$service was restarted on $logdate" | mail -s "Restart notification" mail@domain.com
fi
0

One improvement on the answer provided by @glenn-jackman would be to use the -E option of mailx. From the man page for mailx:

-E

Do not send messages with an empty body. This is useful for piping errors from cron(8) scripts.

0

A more reliable way to deal with this would be to use a tool that was built for the exact scenario you have in mind (which is to monitor a log and send emails out on newly detected patterns).

Download this tool

After unzipping it, and running it, you'll see many real life examples you can mimic. The one example you want to focus on is MODEL13, which addresses the task of sending email notifications to specific email addresses whenever a pattern is detected in one or multiple different logs.

./logrobot localhost /var/tmp/logXray autonda /var/log 60m '.*error.*_P_.*fatal.*' '.'  1 1 errchk -ndshow email=John.Blah@blah.com

In the above command, below are the parameters you have to change, as you see fit:

  • '.*error.*_P_.*fatal.*' - here, two patterns are being searched for in all logs in the /var/log directory

  • /var/log - this is the directory whose logs you want to monitor. If you wish to monitor a specific log file, replace this with the absolute path to the log file.

  • 1 - Send a warning alert if 1 entry is found in the log(s)
  • 1 - Send a critical alert if 1 entry is found in the log(s)
    • If warning and critical are the same, the check will alert as critical when the monitored pattern is found in the logs.
  • errchk - This is the tag for this particular log check. Give this a deserving name...a name matching the type of log(s) being monitored...i.e. tomcatLogCheck
  • And of course the email=. Change this to the appropriate email addresses.

This is a better option because it includes the necessary logic that ensures only new lines that are added to the monitored log(s) get scanned. It's also far more efficient. And it should also be portable.

Of course, make sure postfix is installed on the host so emails can be sent out. I may be forgetting something here, but that should be the gist of how the tool works.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.