67

I'd like to do a health check of a service by calling a specific url on it. Feels like the simplest solution would be to use cron to do the check every minute or so. In case of errors, cron sends me an email.

I tried using cUrl for this but I can't get it to output messages only on errors. If I try to direct output to /dev/null, it prints out progress report.

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  5559  100  5559    0     0   100k      0 --:--:-- --:--:-- --:--:--  106k

I tried looking through the curl options but I just can't find anything to suit the situation where you want it to be silent on success but make noise on errors.

Is there a way to make curl do what I want or is there some other tool I should be looking at?

3
  • 2
    How about using icinga or any other monitoring system? Jul 29, 2013 at 21:12
  • I'm using this on a hobby project with limited resources on a shared host. I would love to use a monitoring system otherwise. Thanks for the tip anyways, I had not heard of icinga before
    – palto
    Aug 4, 2013 at 19:13
  • I would like to point out that share hosting providers will frown on this if you are hitting all the pages on your account too frequently. Providers typically have concurrent process limits and max exec time limits. Also this method will not be accurate if there is a network outage between you and the DC of the provider. You should investigate the domain access logs on your shared account. Typically there is plenty of traffic from bots and just random visitors that will give you a very good idea of how available your site is.
    – rcjohnson
    Feb 13, 2019 at 16:42

8 Answers 8

69

What about -sSf? From the man pages:

  -s/--silent
     Silent or quiet mode. Do not show progress meter or error messages.  
     Makes Curl mute.

  -S/--show-error
     When used with -s it makes curl show an error message if it fails.

  -f/--fail
     (HTTP)  Fail silently (no output at all) on server errors. This is mostly
     done to better enable scripts etc to better deal with failed attempts. In
     normal  cases  when a HTTP server fails to deliver a document, it returns
     an HTML document stating so (which often also describes  why  and  more).
     This flag will prevent curl from outputting that and return error 22.

     This method is not fail-safe and there are occasions where non-successful
     response codes will  slip  through,  especially  when  authentication  is
     involved (response codes 401 and 407).

For example:

curl -sSf http://example.org > /dev/null
4
  • 4
    -sS didn't output the error message for some reason. I had to add -f also. The correct working command seems to be curl -fsS http://example.org > /dev/null. That will not output anything when nothing is wrong but it will print the status code on errors which is fine by me.
    – palto
    Jul 29, 2013 at 15:50
  • 2
    Ok, added -f for future references.
    – ahilsend
    Jul 29, 2013 at 16:08
  • I added echo to so that I know the connection was successful. curl -sSf http://example.org > /dev/null && echo "success" Mar 15, 2022 at 11:57
  • Thanks. Found this helpful. How about throwing in a -L to follow redirects? Dec 5, 2022 at 14:37
37

I think that for the simplest way to check if the site is alive, you could use the following method:

curl -Is http://www.google.com | head -n 1

This will return HTTP/1.1 200 OK. If the return doesn't match your output then call out for help.

7
  • 2
    Checking status code makes probably more sense like the other proposals Jan 6, 2016 at 17:45
  • I get "HTTP/1.1 302 Found". May 8, 2017 at 23:34
  • 1
    This is great because it goes beyond a ping allowing me to check to see if my site is loading correctly (connection to mySQL etc.) and get more meaningful results.
    – Nathan
    Apr 9, 2019 at 20:47
  • What if the URL you want to test is a big file? like a pdf file or something? or a video stream which never ends?
    – Freedo
    Nov 13, 2021 at 5:52
  • @Freedo You can try the -I or --head flag which execute a HEAD request that fetches the response code and headers only. It may not work in all cases as severs can be configured to treat GET and HEAD requests differently.
    – Kuba Holuj
    Mar 12, 2022 at 4:18
12

you can capture network timing stats from curl. latencies of each phase in a request/response cycle can be useful to determine health.

$ URL=https://example.com
$ curl "$URL" -s -o /dev/null -w \
> "response_code: %{http_code}\n
> dns_time: %{time_namelookup}
> connect_time: %{time_connect}
> pretransfer_time: %{time_pretransfer}
> starttransfer_time: %{time_starttransfer}
> total_time: %{time_total}
> "
response_code: 200

dns_time: 0.029
connect_time: 0.046
pretransfer_time: 0.203
starttransfer_time: 0.212
total_time: 0.212
9

You need the -s flag (silent), -f flag (fail with exit code on error) and can use the -o flag to redirect output:

curl www.websiteToTest.com -s -f -o /dev/null || echo "Website down." | mail -s "Website is down" admin@thesite.com 

This is just an bad example for a simple cron script. Normally, you want to get only one mail if a website is down.

7

Curl has very specific exit status codes
Why not just check for those?

#!/bin/bash

##name: site-status.sh

FAIL_CODE=6

check_status(){
    LRED="\033[1;31m" # Light Red
    LGREEN="\033[1;32m" # Light Green
    NC='\033[0m' # No Color


    curl -sf "${1}" > /dev/null

    if [ ! $? = ${FAIL_CODE} ];then
        echo -e "${LGREEN}${1} is online${NC}"
    else
        echo -e "${LRED}${1} is down${NC}"
    fi
}


check_status "${1}"

Usage:

$ site-status.sh example.com

Result:

$ example.com is online

Notes:

This script only checks if the site can be resolved.

This code should get you by if all you care about is the site being up or down.
However, if you make a few changes to the if/else block, you can easily test for other status codes if you wish

1
  • Plus one, I also check for 404 - not found using the header flag in curl.
    – Timo
    Dec 2, 2022 at 8:56
3

Answer:

#!/usr/bin/env bash
set -euETo pipefail

timeout 3s curl -fIsS http://example.org > /dev/null
# and if you have TLS (https), check if it expires within next 20 days:
true | openssl s_client -connect example.org:443 2>/dev/null | openssl x509 -noout -checkend "$((20*24*60*60))"

Explanation:

  • timeout 3s will set a 3-second timeout for your request. Answering slower is consedered to be "not healthy"
  • curl -f flag will fail early, -S will show errors, -s will suppress normal output, -I will only fetch HTTP headers, not the content. (As always, more details are available on man curl command.)
  • openssl -checkend directive checks expiry dates for a certificate. In my example, it is 20 days (specified in seconds).
1
  • I think you have the explanations of -s and -S swapped
    – nafg
    Apr 20, 2017 at 1:11
2

I was recently asked to come up with something that would act more like a sophisticated heartbeat.

for i in `curl -s -L cnn.com |egrep --only-matching "http(s?):\/\/[^ \"\(\)\<\>]*" | uniq` ; do curl -s -I $i 2>/dev/null |head -n 1 | cut -d$' ' -f2; done

Or, expanded for a little more readability,

for i in $(curl -s -L cnn.com |egrep --only-matching 'http(s?):\/\/[^ \"\(\)\<\>]*' | uniq)
do
    curl -s -I "$i" 2>/dev/null | head -n 1 | cut -d' ' -f2
done

What I did was curl a website, parse out all the links from the html, and then curl those parsed links, outputting only the status code. Then I would search for http status codes >= 400 to find errors.

2

This way will help you when trying to test the site when https is present:

#!/bin/bash
# put your domain in this var
https=https://www.somewebsite.com

# save the status in some variable 
status=`curl $https -k -s -f -o /dev/null && echo "SUCCESS" || echo "ERROR"`    

# print results (or use it in your scripts)
echo "testing $https=$status"

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .