17

Background : I need to receive an alert when my server is down. When the server is down, maybe the Sysload collector will not be able to send any alert. To receive an alert when the server is down, I have an external source (server) to detect it.

Question : Is there any way (i prefer bash script) to detect when my server is down or offline and sends an alert message (Email + SMS)?

24

If you have a separate server to run your check script on, something like this would do a simple Ping test to see if the server is alive:

#!/bin/bash
SERVERIP=192.168.2.3
NOTIFYEMAIL=test@example.com

ping -c 3 $SERVERIP > /dev/null 2>&1
if [ $? -ne 0 ]
then
   # Use your favorite mailer here:
   mailx -s "Server $SERVERIP is down" -t "$NOTIFYEMAIL" < /dev/null 
fi

You can cron the script to run periodically.

If you don't have mailx, you'll have to replace that line with whatever command line email program you have and probably change the options. If your carrier provides an SMS email address, you can send the email to that address. For example, with AT&T, if you send an email to phonenumber@txt.att.net, it will send the email to your phone.

Here's a list of email to SMS gateways:

http://en.wikipedia.org/wiki/List_of_SMS_gateways

If your server is a publicly accessible webserver, there are some free services to monitor your website and alert you if it's down, search the web for free website monitoring to find some.

  • Can you make detail regarding SMS alert message? – Md Mahbubur Rahman Nov 22 '12 at 4:16
  • 4
    Instead of pinging the server, you should test whatever it is that you care about the server doing. For example, if it's a mail server, it's much more important that it successfully sends and receives mail. – Jim Paris Nov 22 '12 at 6:52
  • I stayed away from application detection in my answer since the question didn't specify what services are running on the server - he asked for server up/down detection. There are hundreds of common services that could be detected and an unlimited number of custom applications. It sounds like he has a local service monitor (sysload) that's monitoring the services. – Johnny Nov 22 '12 at 17:20
  • Ping is all but useless for me. I have a server down yet I get 3 out of 3 packets returned and a final return value of 0. How come? The local machine is returning "Destination host unreachable" packets. – AlastairG Nov 9 '18 at 8:47
  • @AlastairG - this would be better asked as a question, there's not enough room in a comment to answer it. But search the site first to see if someone asked a similar question, you may find your answer there. – Johnny Nov 9 '18 at 17:47
9

Pinging is an option, but on many occasions a machine will be able to send a ping reply, while the actual server that it is all about is down. It is better do an end-to-end test. In the below example a page is requested from the webserver.

If it is a webserver, it would look something like this:

#!/bin/bash
wget -qO /dev/null 'http://webserver/some_existing_short_document.html' || {
    echo "Webserver down"
    # another mailer example
    sendemail -s mailserverip -f 'from@localhost' -t 'user@localhost' -u 'Webserver down' -m 'The webserver is down'
}

If you change the html document into a php document, and make de php script test things like the database connection, filesystems etc., you can even test more aspects of the server. That way you can start proactive monitoring of the machine (see problems before they make the server crash).

Similar with checking a mailserver, but instead of requesting a web page, you simply send an email through the mailserver and see if you receive it in your mailbox

  • if you got root emails forwarded to your inbox, you dont have to sendmail if you execute this script in the crontab (because it will send the stdout to you) – P.Scheit Dec 13 '17 at 10:52
5

Here is how I solved the same problem

#!/bin/bash
NOTIFYEMAIL=<your email>
SMSEMAIL=<cell phone number @ sms-gateway>
SENDEREMAIL=alert@localhost
SERVER=http://127.0.0.1/
PAUSE=60
FAILED=0
DEBUG=0

while true 
do
/usr/bin/curl -sSf $SERVER > /dev/null 2>&1
CS=$?
# For debugging purposes
if [ $DEBUG -eq 1 ]
then
    echo "STATUS = $CS"
    echo "FAILED = $FAILED"
    if [ $CS -ne 0 ]
    then
        echo "$SERVER is down"

    elif [ $CS -eq 0 ]
    then
        echo "$SERVER is up"
    fi
fi

# If the server is down and no alert is sent - alert
if [ $CS -ne 0 ] && [ $FAILED -eq 0 ]
then
    FAILED=1
    if [ $DEBUG -eq 1 ]
    then
        echo "$SERVER failed"
    fi
    if [ $DEBUG = 0 ]
    then
        echo "$SERVER went down $(date)" | /usr/bin/mailx -s "$SERVER went down" -r "$SENDEREMAIL" "$SMSEMAIL" 
        echo "$SERVER went down $(date)" | /usr/bin/mailx -s "$SERVER went down" -r "$SENDEREMAIL" "$NOTIFYEMAIL" 
    fi

# If the server is back up and no alert is sent - alert
elif [ $CS -eq 0 ] && [ $FAILED -eq 1 ]
then
    FAILED=0
    if [ $DEBUG -eq 1 ]
    then
        echo "$SERVER is back up"
    fi
    if [ $DEBUG = 0 ]
    then
        echo "$SERVER is back up $(date)" | /usr/bin/mailx -s "$SERVER is back up again" -r "$SENDEREMAIL" "$SMSEMAIL"
        echo "$SERVER is back up $(date)" | /usr/bin/mailx -s "$SERVER is back up again" -r "$SENDEREMAIL" "$NOTIFYEMAIL"
    fi
fi
sleep $PAUSE
done
  • I had to adapt to run on my mac /usr/bin/mail -s "$SERVER went down" "$SENDEREMAIL" "$SMSEMAIL" . – MikeiLL Apr 13 '15 at 17:20
  • This solution surely works BUT . (1) if you are calling this on the local machine I guess it will tell you that apache died, though I wouldn't do it this way. (2) If you use this on the local machine and the network is down or something between this box and it's edge server or something, you won't get the email and or it won't know there is an issue. (3) If you use this on another machine, your curl command will take a LONG time to time out unless specified, so ifyou were looking at mulitple hosts it would grind to a halt.. – Mike Q May 12 '18 at 17:33
1

I would highly recommend using Nagios, it is infrastructure for monitoring and alerting on any service you want (many plugins available and you can write your own). It can of course do simple pings to check on servers availability, but as others have pointed, it is better to check for services availability (e.g. web, email, etc.) instead (which nagios can do easily).

  • it's a cool product, I wonder about security concerns – Mike Q May 12 '18 at 17:29

protected by Community Aug 7 '15 at 20:52

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.