29

I'd like to do a health check of a service by calling a specific url on it. Feels like the simplest solution would be to use cron to do the check every minute or so. In case of errors, cron sends me an email.

I tried using cUrl for this but I can't get it to output messages only on errors. If I try to direct output to /dev/null, it prints out progress report.

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  5559  100  5559    0     0   100k      0 --:--:-- --:--:-- --:--:--  106k

I tried looking through the curl options but I just can't find anything to suit the situation where you want it to be silent on success but make noise on errors.

Is there a way to make curl do what I want or is there some other tool I should be looking at?

  • 2
    How about using icinga or any other monitoring system? – Stéphane Chazelas Jul 29 '13 at 21:12
  • I'm using this on a hobby project with limited resources on a shared host. I would love to use a monitoring system otherwise. Thanks for the tip anyways, I had not heard of icinga before – palto Aug 4 '13 at 19:13
  • I would like to point out that share hosting providers will frown on this if you are hitting all the pages on your account too frequently. Providers typically have concurrent process limits and max exec time limits. Also this method will not be accurate if there is a network outage between you and the DC of the provider. You should investigate the domain access logs on your shared account. Typically there is plenty of traffic from bots and just random visitors that will give you a very good idea of how available your site is. – rcjohnson Feb 13 at 16:42
35

What about -sSf? From the man pages:

  -s/--silent
     Silent or quiet mode. Do not show progress meter or error messages.  
     Makes Curl mute.

  -S/--show-error
     When used with -s it makes curl show an error message if it fails.

  -f/--fail
     (HTTP)  Fail silently (no output at all) on server errors. This is mostly
     done to better enable scripts etc to better deal with failed attempts. In
     normal  cases  when a HTTP server fails to deliver a document, it returns
     an HTML document stating so (which often also describes  why  and  more).
     This flag will prevent curl from outputting that and return error 22.

     This method is not fail-safe and there are occasions where non-successful
     response codes will  slip  through,  especially  when  authentication  is
     involved (response codes 401 and 407).

For example:

curl -sSf http://example.org > /dev/null
  • 3
    -sS didn't output the error message for some reason. I had to add -f also. The correct working command seems to be curl -fsS http://example.org > /dev/null. That will not output anything when nothing is wrong but it will print the status code on errors which is fine by me. – palto Jul 29 '13 at 15:50
  • 1
    Ok, added -f for future references. – ahilsend Jul 29 '13 at 16:08
7

I think that for the simplest way to check if the site is alive, you could use the following method:

curl -Is http://www.google.com | head -n 1

This will return HTTP/1.1 200 OK. If the return doesn't match your output then call out for help.

  • 1
    Checking status code makes probably more sense like the other proposals – Dukeatcoding Jan 6 '16 at 17:45
  • I get "HTTP/1.1 302 Found". – Sponge Comrade May 8 '17 at 23:34
  • 1
    This is great because it goes beyond a ping allowing me to check to see if my site is loading correctly (connection to mySQL etc.) and get more meaningful results. – Nathan Apr 9 at 20:47
6

You need the -s flag (silent), -f flag (fail with exit code on error) and can use the -o flag to redirect output:

curl www.websiteToTest.com -s -f -o /dev/null || echo "Website down." | mail -s "Website is down" admin@thesite.com 

This is just an bad example for a simple cron script. Normally, you want to get only one mail if a website is down.

6

you can capture network timing stats from curl. latencies of each phase in a request/response cycle can be useful to determine health.

$ URL=https://example.com
$ curl "$URL" -s -o /dev/null -w \
> "response_code: %{http_code}\n
> dns_time: %{time_namelookup}
> connect_time: %{time_connect}
> pretransfer_time: %{time_pretransfer}
> starttransfer_time: %{time_starttransfer}
> total_time: %{time_total}
> "
response_code: 200

dns_time: 0.029
connect_time: 0.046
pretransfer_time: 0.203
starttransfer_time: 0.212
total_time: 0.212
1

I was recently asked to come up with something that would act more like a sophisticated heartbeat.

for i in `curl -s -L cnn.com |egrep --only-matching "http(s?):\/\/[^ \"\(\)\<\>]*" | uniq` ; do curl -s -I $i 2>/dev/null |head -n 1 | cut -d$' ' -f2; done

Or, expanded for a little more readability,

for i in $(curl -s -L cnn.com |egrep --only-matching 'http(s?):\/\/[^ \"\(\)\<\>]*' | uniq)
do
    curl -s -I "$i" 2>/dev/null | head -n 1 | cut -d' ' -f2
done

What I did was curl a website, parse out all the links from the html, and then curl those parsed links, outputting only the status code. Then I would search for http status codes >= 400 to find errors.

1

Answer:

#!/bin/bash -eu
timeout 3s curl -fIsS http://example.org > /dev/null
# and if you have TLS (https), check if it's about to expire:
true | openssl s_client -connect example.org:443 2>/dev/null | openssl x509 -noout -checkend "$((3600*24*20))"

Explanations:

  • timeout 3s will set a 3-second timeout for your request. Answering slower is consedered to be "not healthy"
  • curl -f flag will fail early, -S will show errors, -s will suppress normal output, -I will only fetch HTTP headers, not the content. (As always, more details are available on man curl command.)
  • openssl -checkend directive checks expiry dates for a certificate. In my example, it is 20 days (specified in seconds).
  • I think you have the explanations of -s and -S swapped – nafg Apr 20 '17 at 1:11
1

This way will help you when trying to test the site when https is present:

#!/bin/bash
# put your domain in this var
https=https://www.somewebsite.com

# save the status in some variable 
status=`curl $https -k -s -f -o /dev/null && echo "SUCCESS" || echo "ERROR"`    

# print results (or use it in your scripts)
echo "testing $https=$status"
0

Curl has very specific exit status codes
Why not just check for those?

#!/bin/bash

##name: site-status.sh

FAIL_CODE=6

check_status(){
    LRED="\033[1;31m" # Light Red
    LGREEN="\033[1;32m" # Light Green
    NC='\033[0m' # No Color


    curl -sf "${1}" > /dev/null

    if [ ! $? = ${FAIL_CODE} ];then
        echo -e "${LGREEN}${1} is online${NC}"
    else
        echo -e "${LRED}${1} is down${NC}"
    fi
}


check_status "${1}"

Usage:

$ site-status.sh example.com

Result:

$ example.com is online

Notes:

This script only checks if the site can be resolved.

This code should get you by if all you care about is the site being up or down.
However, if you make a few changes to the if/else block, you can easily test for other status codes if you wish

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.