Bash script to detect when my server is down or offline

Question

Background : I need to receive an alert when my server is down. When the server is down, maybe the Sysload collector will not be able to send any alert. To receive an alert when the server is down, I have an external source (server) to detect it.

Question : Is there any way (i prefer bash script) to detect when my server is down or offline and sends an alert message (Email + SMS)?

Answer 1

If you have a separate server to run your check script on, something like this would do a simple Ping test to see if the server is alive:

#!/bin/bash
SERVERIP=192.168.2.3
[email protected]

ping -c 3 $SERVERIP > /dev/null 2>&1
if [ $? -ne 0 ]
then
   # Use your favorite mailer here:
   mailx -s "Server $SERVERIP is down" -t "$NOTIFYEMAIL" < /dev/null 
fi

You can cron the script to run periodically.

If you don't have mailx, you'll have to replace that line with whatever command line email program you have and probably change the options. If your carrier provides an SMS email address, you can send the email to that address. For example, with AT&T, if you send an email to phonenumber@txt.att.net, it will send the email to your phone.

Here's a list of email to SMS gateways:

http://en.wikipedia.org/wiki/List_of_SMS_gateways

If your server is a publicly accessible webserver, there are some free services to monitor your website and alert you if it's down, search the web for free website monitoring to find some.

Answer 2

Pinging is an option, but on many occasions a machine will be able to send a ping reply, while the actual server that it is all about is down. It is better do an end-to-end test. In the below example a page is requested from the webserver.

If it is a webserver, it would look something like this:

#!/bin/bash
wget -qO /dev/null 'http://webserver/some_existing_short_document.html' || {
    echo "Webserver down"
    # another mailer example
    sendemail -s mailserverip -f 'from@localhost' -t 'user@localhost' -u 'Webserver down' -m 'The webserver is down'
}

If you change the html document into a php document, and make de php script test things like the database connection, filesystems etc., you can even test more aspects of the server. That way you can start proactive monitoring of the machine (see problems before they make the server crash).

Similar with checking a mailserver, but instead of requesting a web page, you simply send an email through the mailserver and see if you receive it in your mailbox

Answer 3

Here is how I solved the same problem

#!/bin/bash
NOTIFYEMAIL=<your email>
SMSEMAIL=<cell phone number @ sms-gateway>
SENDEREMAIL=alert@localhost
SERVER=http://127.0.0.1/
PAUSE=60
FAILED=0
DEBUG=0

while true 
do
/usr/bin/curl -sSf $SERVER > /dev/null 2>&1
CS=$?
# For debugging purposes
if [ $DEBUG -eq 1 ]
then
    echo "STATUS = $CS"
    echo "FAILED = $FAILED"
    if [ $CS -ne 0 ]
    then
        echo "$SERVER is down"

    elif [ $CS -eq 0 ]
    then
        echo "$SERVER is up"
    fi
fi

# If the server is down and no alert is sent - alert
if [ $CS -ne 0 ] && [ $FAILED -eq 0 ]
then
    FAILED=1
    if [ $DEBUG -eq 1 ]
    then
        echo "$SERVER failed"
    fi
    if [ $DEBUG = 0 ]
    then
        echo "$SERVER went down $(date)" | /usr/bin/mailx -s "$SERVER went down" -r "$SENDEREMAIL" "$SMSEMAIL" 
        echo "$SERVER went down $(date)" | /usr/bin/mailx -s "$SERVER went down" -r "$SENDEREMAIL" "$NOTIFYEMAIL" 
    fi

# If the server is back up and no alert is sent - alert
elif [ $CS -eq 0 ] && [ $FAILED -eq 1 ]
then
    FAILED=0
    if [ $DEBUG -eq 1 ]
    then
        echo "$SERVER is back up"
    fi
    if [ $DEBUG = 0 ]
    then
        echo "$SERVER is back up $(date)" | /usr/bin/mailx -s "$SERVER is back up again" -r "$SENDEREMAIL" "$SMSEMAIL"
        echo "$SERVER is back up $(date)" | /usr/bin/mailx -s "$SERVER is back up again" -r "$SENDEREMAIL" "$NOTIFYEMAIL"
    fi
fi
sleep $PAUSE
done

Answer 4

I would highly recommend using Nagios, it is infrastructure for monitoring and alerting on any service you want (many plugins available and you can write your own). It can of course do simple pings to check on servers availability, but as others have pointed, it is better to check for services availability (e.g. web, email, etc.) instead (which nagios can do easily).