How to log Internet connection drops

Question

I need to execute a script as soon as my raspberry pi gets connected to the Internet. However I was wondering if there was a better way than just pinging Google every minute or so.

My problem is that my Internet connection drops 1-2 times during the day so I need a way to log such events.
It's just the ADSL dropping during the day, I was looking for some way to log when it occurs even when i don't notice it. I think I'll setup a script as suggested.

Answer 1

you can make a check on:

cat /sys/class/net/wlan0/carrier

where wlan0 is my internet interface. you can use whatever interface you are using , such as eth0 , eth1 , wlan0 for internet connectivity. if the output of that command is 1 then you are connected. otherwise not.so you may write script like this:

#!/bin/bash
# Test for network conection
for interface in $(ls /sys/class/net/ | grep -v lo);
do
if [[ $(cat /sys/class/net/$interface/carrier) = 1 ]]; then ; echo "online"; fi
done

you can also use the command:

#hwdetect --show-net

this script also works well:

#!/bin/bash

WGET="/usr/bin/wget"

$WGET -q --tries=20 --timeout=10 http://www.google.com -O   /tmp/google.idx &> /dev/null
if [ ! -s /tmp/google.idx ]
then
  echo "Not Connected..!"
else
  echo "Connected..!"
fi

Answer 2

Asking your system if it believes it has a connection is a proxy measure of "is my ISP dropping stuff?". Proxy measures are, by definition, a simplified model of the system of interest and do not preserve information. The question can only be answered by actually getting the information you are interested in.

ping is actually a poor choice of tests because it is an ICMP protocol which often gets special treatment. If, for example, you are interested in HTTP connectivity something like

curl --head http://www.example.com

will show if you can actually get pages. If you poll it, be polite and use a minimum of 60 second sleep between. An ISP outage of under a minute could be considered "not an outage".

Answer 3

Given your relatively simple requirement, yes, a simple ping will do. You needn't use Google as a test host. If your ISP has a public web site, which is quite likely these days, use it then.

Here's a [bash] script I intended to use with an old, paused project of mine. It just pings a host and collects statistics, which it prints on-demand to the system log. The only dependencies are ping and bc. Note that you'll need a not-so-old version of ping, i.e. a Linux-Capabilities-aware ping.

The script uses predefined interval and signature values and is designed to run interactively or in the background — e.g. in case you need some logging; an external command needs to be given. You'll get flapping detection and link up/down detection for the same price ;-) . You have embedded help in case you need it. Don't hesitate to ask if you have any question.

I wrote it as an exercise while keeping the memory footprint as low as possible. I remember planning to make it run with dash but there is one bashism I couldn't get rid of so far. Hoping you'll find it useful, here it goes:

#!/bin/bash
#
#  ping.sh
#
#  Copyright 2014 VinzC <vinzc@users.sf.net>
#
#  This program is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 2 of the License, or
#  (at your option) any later version.
#
#  This program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#  GNU General Public License for more details.
#
#  You should have received a copy of the GNU General Public License
#  along with this program; if not, write to the Free Software
#  Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
#  MA 02110-1301, USA.
#
# Runs in the background ping'ing given addresses. May send
# notifications through an external routine that is given as
# an argument.
#
# Uses environment variables:
#   * LOG_FILE (log file for debug purposes, silent otherwise)
#
# Symlinked as /usr/bin/anemon-ping

VERSION=1.0.3
AUTHOR="VinzC <vinzc@users.sf.net>"
DATE="Jan 2014"

# Background ping, keep only relevant information from the
# response, add timestamp in seconds. For use with graphing.

# Default padding for ICMP packets
ICMP_PADDING_DEFAULT=ffeab16b00b1e2

# Default number of seconds between ICMP probes
ICMP_INTERVAL_DEFAULT=300

# Number of samples to declare a host state steady
ICMP_STEADY_THRESHOLD=3

# Number of samples in which a host is allowed to go down
# 3 times before being flagged as flapping. (Why 3? 2 is
# not enough to make it a habbit, 3 is...)
ICMP_FLAPPING_THRESHOLD=24

# Number of samples after which dupe and damaged packets
# are gone for good. Default matches 24 hours.
ICMP_NET_ERROR_THRESHOLD=288

# Host state change command. The command is called with two or
# three arguments: host name and state, plus optional arguments.
# See usage for events.
host_cmd=false

# True if results shall be written to the standard output
verbose=false

# State variables
host_state=1            # Bit0: 1=up, 0=down
                        # Bit1: 1=flapping, 0=normal
host_unavail_state=0    # State counters
host_flap_state=0
host_warning_state=0

rtt_min=                # Ping statistics
rtt_max=
rtt_avg=

icmp_seq=0              # Number of requests
icmp_lost=0             # Count of losses

icmp_dupes=             # Network errors (counters)
icmp_damaged=

usage()
{
    cat <<EOF

USAGE
    ${0##*/} [-i interval] [-p pattern] [-P string] [-I interface]
        [-s packetsize] [-W timeout] [-v] [--host-command=CMD]
        [--flapping-threshold=N] [--error-threshold=N] destination
    ${0##*/} -h

ARGUMENTS
    Short options are a subset of standard ping arguments with the
    exception of -P and -v. Both -p and -P define a custom padding.
    Unlike ping's -p, option -P accepts any alphanumeric string,
    which will be converted to a hexadecimal string. The resulting
    pattern is eventually truncated to 16 bytes.

    Option -v switches to verbose mode. In this mode echo replies
    are sent to the standard output. This is useful for testing.

    -h  prints this help page.

    --host-command=CMD

        Run CMD on events. The first argument is the destination
        parameter, the second is the event name. Some events may
        have additional parameters.

        All events except the echo reply are sent to the system
        log. Events are:

        "start"     The monitoring process has been started. The
                external script can initialize its working
                context, e.g. create a round-robin database
                to store echo reply TTL values. An additional
                argument is passed with the interval value if
                option -i was specified. The default interval
                matches RRD default step, i.e. 300 seconds.
        "stop"      The monitoring process is stopping.
        "icmp"      Echo reply. Additional arguments are timestamp
                and roudtrip time. This event occurs every 5
                minutes by default and can be changed with -i.
        "up"        Host is up, steadily.
        "down"      Host is down, steadily.
        "flapping"  Host state is unstable.
        "damaged"   Damaged packets were detected.
        "dupes"     Duplicate packets were detected.

    --flapping-threshold=N

        The script attempts to detect flapping interfaces. It uses
        a discrete formula to keep a reasonable maximum delay in
        cases flapping occurs frequently, which must be considered
        a critical situation. Flapping detection is done as soon
        as no response from the remote host is received. No echo
        reply for more than 3 samples is a steady "down" state.

        An interface that has been flapping will take longer to be
        considered steady again. The value of --flapping-threshold
        sets that delay to the same amount of samples.

        The default value for N is 24. A bigger value will make an
        flapping interface wait proportionally longer before it is
        marked "up" again.

    --error-threshold=N

        Linux only: the background ping process informs whenever
        duplicated or damaged packets are received from a remote
        host. Such network errors are notified no more than once
        in N samples.

INTERACTIVE CONTROL
    The script reacts to signal SIGHUP to print statistics on the
    PING process running in the background. If verbose mode is
    enabled (-v) statistics are sent to the standard output. They
    are sent to the system log otherwise.

    Example:
        ${0##*/} --host-command=... host &
        PID=$!
        ...
        kill -HUP $PID

    PING statistics include min, avg and max roundtrip times and
    percentage of lost echo replies. Individual figures are also
    shown for damaged and duplicate packets if there are any.

    Note that the average value is a running average that uses a
    discrete averaging algorithm, i.e.:

        avg(i) = [ avg(i-1) + x(i-1) ] / 2

SEE ALSO
    man ping, ping(8)
EOF
    exit 0
}

help()
{
    cat <<EOF
${0##*/} version $VERSION, © $DATE by $AUTHOR

Monitors a remote host periodically sending ICMP packets from a ping
process running in the background. The script can execute a custom
command each time an important network condition occurs. The main
purpose is to record echo reply times in a round-robin database for
further graphing.
EOF
    usage
}

bc()
{
    # Workaround to print the leading zero for values <1
    /usr/bin/bc | sed 's/^\./0./g'
}

# Parse command line arguments. Only parse new or overriden arguments.
# Used to determine the remote host, mainly.
parse_args()
{
    # Need a F@!#^}G temporary variable to check getopt return code!
    args="$(getopt \
        -o i:I:p:P:s:W:hv \
        -l host-command:,flapping-threshold:,error-threshold \
        -- "$@")" && eval set -- "$args" && unset args || return $?

    # Now check the remaining arguments
    while [ -n "$1" ]; do
        [ "$1" = "--" ] || case $1 in

            --host-command)
                host_cmd=$2; shift;;

            --flapping-threshold)
                ICMP_FLAPPING_THRESHOLD=$2; shift;;

            --error-threshold)
                ICMP_NET_ERROR_THRESHOLD=$2; shift;;

            -i)
                ICMP_INTERVAL=$2; shift;;
            -p)
                ICMP_PADDING=$2; shift;;
            -P)
                ICMP_PADDING=$(printf "$2" | od -A n -t x1 | \
                    sed -r -e 's:\s+::g' -e 's:.::33g'); shift;;
            -I)
                ICMP_IFACE=$2; shift;;
            -s)
                ICMP_PKTSIZE=$2; shift;;
            -W)
                ICMP_TIMEOUT=$2; shift;;
            -v)
                verbose=true;;
            -h)
                help;;
            *)
                ICMP_HOST="$1";;
        esac; shift
    done
    [ -n "$ICMP_HOST" ] || usage 1>&2
}

logger()
{
    # Write to standard output in verbose mode, to syslog otherwise
    $verbose && echo "$@" || \
        /usr/bin/logger -t "${0##*/}[$$]" "$@"
}

set_response_time()
{
    # Call external command or prompt to the console in verbose mode
    $host_cmd $ICMP_HOST "icmp" $1 $3 && ! $verbose || \
        printf "%d: seq=%d, time=%s\n" "$1" "$2" "$3"
    return 0
}

set_state_up()
{
    # Clear flapping state ans set (steady) up flag
    host_state=0x01

    # Call the external notification function and log host state
    $host_cmd $ICMP_HOST "up"
    logger "Host interface or host @ $ICMP_HOST is now up."
}

set_state_down()
{
    # Clear up flag only
    host_state=$(( host_state & 0xFE ))

    # Call the external notification function and log host state
    $host_cmd $ICMP_HOST "down"
    logger "Host interface or host @ $ICMP_HOST is down!"
}

set_state_flapping()
{
    # Set flapping and down flags
    host_state=2

    # Call the external notification function and log host state
    $host_cmd $ICMP_HOST "flapping"
    logger "Host interface or host @ $ICMP_HOST is unstable!"
}

set_host_message()
{
    # Reset error counter to the maximum
    host_warning_state=$ICMP_NET_ERROR_THRESHOLD

    # Call the external notification function and log host state
    $host_cmd $ICMP_HOST $1
    logger "Errors received from host interface or host @ $ICMP_HOST ($1)!"
}

print_stats()
{
    if [ $icmp_seq -eq 0 ]; then
        logger "PING $ICMP_HOST: no packet sent"
    else
        local icmp_received=$(( icmp_seq - icmp_lost ))
        local icmp_losses=$( echo "scale=2; 100 * $icmp_lost / $icmp_seq" | bc )
        logger "PING $ICMP_HOST: $icmp_seq packets sent, $icmp_lost lost, ${icmp_losses}% loss${rtt_min:+; rtt min/avg/max = $rtt_min/$rtt_avg/$rtt_max ms}${icmp_dupes:+, $icmp_dupes dupes}${icmp_damaged:+, $icmp_damaged bad CRC}"
    fi
}

echo_reply()
{
    # First argument is time in seconds (icmp_seq is global)
    local TM=$1 ttl time msg; shift

    # Evaluate the remaining arguments as expressions
    eval "$@"

    # No time variable means host is not responding
    [ -z "$time" ] && return 1

    # Update statistics: average, minimum and maximum RTT
    rtt_avg=$( echo "scale=3; (${rtt_avg:-$time} + $time)/2" | bc )
    rtt_min=$( echo "scale=3; rtt_min=${rtt_min:-$time}; if ($time < rtt_min) $time else rtt_min" | bc )
    rtt_max=$( echo "scale=3; rtt_max=${rtt_max:-$time}; if ($time > rtt_max) $time else rtt_max" | bc )

    # Decrement the state counter if greater than zero
    [ $host_unavail_state -ne 0 ] && \
        host_unavail_state=$(( host_unavail_state - 1 ))

    # The host is not up if:
    # - it is flapping (bit 1 of the state flag) and the flapping
    #   counter is greater than 0  OR
    # - the state counter is greater than 0 (non flapping case).
    # As long as one of these condition is true, the state flag
    # will not be set to UP (bit 0 set, bit 1 cleared).
    #
    # Once the state counter reaches zero (steady "up" state) and
    # the host state is no longer flapping, change the state flag.
    # Also don't change the state flag if bit 0 was already set.
    [ $(( host_state & 0x02 )) -ne 0  ] && [ $host_flap_state -ne 0 ] || \
    [ $host_unavail_state -ne 0 ] || \
    [ $(( host_state & 0x01 )) -ne 0 ] || \
        set_state_up

    # Warn if damaged or duplicate packets. Don't warn
    # again until the warning counter reaches zero. Treat
    # damaged and dupe packets alike for both are very
    # unlikely to occur at the same time.
    if [ -n "$msg" ]; then
        eval rtt_$msg=$(( rtt_$msg + 1 ))
        [ $host_warning_state -eq 0 ] && set_host_message $msg
    fi

    # Run external command to store response time
    set_response_time $TM $icmp_seq $time
    return 0
}

no_response()
{
    # Store the number of lost replies
    icmp_lost=$(( icmp_lost + 1 ))

    # FLAPPING DETECTION
    # ------------------
    # Increment flapping state using a discrete low-pass formula
    # to prevent excessive values. Handle flapping only if host
    # has just come down, don't wait for a steady "down" state.
    [ $host_unavail_state -eq 0 ] && \
    [ $(( host_flap_state=(3*host_flap_state + 7*ICMP_FLAPPING_THRESHOLD) / 8 )) -gt $ICMP_FLAPPING_THRESHOLD ] && \
        set_state_flapping

    # Increment host state until it reaches the threshold, which
    # marks the steady "down" state. Only then call the external
    # command to allow notifying the host is "down". Just don't
    # call the command more than once if the host is still down
    # by the next time.
    [ $host_unavail_state -lt $ICMP_STEADY_THRESHOLD ] && \
    [ $(( host_unavail_state=host_unavail_state + 1 )) -eq $ICMP_STEADY_THRESHOLD ] && \
    [ $(( host_state & 0x03 )) -eq 1 ] && \
        set_state_down
}

# Parse command-line arguments and set globals
parse_args "$@" || exit $?

# Redirect stderr to LOG_FILE if defined
[ -z "$LOG_FILE" ] || exec 2>$LOG_FILE

# Print PING statistics upon receiving SIGUSR1
trap print_stats HUP

# Send even "stop" upon terminating
trap "printf '\n'; $host_cmd $ICMP_HOST stop; print_stats" INT QUIT TERM ABRT


# Notify monitoring starts
$host_cmd $ICMP_HOST "start"

# 1. filter out lines keeping only those that include response
#    times and those about non responding hosts.
# 2. Stick units to response times and keep only the multiplier
#    if it's different from "m"
# 3. Keep only the integer part of the timestamp, erase garbage
#    before the relevant information (var=value)
# 4. Warn about damaged and duplicate packets
#
# Make sure sed does NOT buffer output (hence -u)
while read R; do
    echo_reply $R || no_response

    # Decrement other state variables until it reaches zero
    [ $host_flap_state -eq 0 ] || \
        host_flap_state=$(( host_flap_state - 1 ))

    [ $host_warning_state -eq 0 ] || \
        host_warning_state=$(( host_warning_state - 1 ))

# Downside: need bash for process redirection, which is needed
# to access state variables outside the while/read loop...
done < <(LC_ALL=POSIX /bin/ping -OD \
    ${ICMP_IFACE:+-I $ICMP_IFACE} \
    ${ICMP_TIMEOUT:+-W $ICMP_TIMEOUT} \
    ${ICMP_PKTSIZE:+-s $ICMP_PKTSIZE} \
    -i ${ICMP_INTERVAL:-$ICMP_INTERVAL_DEFAULT} \
    -p ${ICMP_PADDING:-$ICMP_PADDING_DEFAULT} $ICMP_HOST |
sed -rnu \
    -e '/no answer|[0-9]+ bytes from/!d' \
    -e 's@(time=[0-9.]+)\s+m?(\w*)s@\1\2@g' \
    -e 's@\(DUP\!\)@msg="dupes"@g' \
    -e 's@\(BAD CHECKSUM\!\)@msg="damaged"@g' \
    -e 's@\[(\w+)\.\w+\][a-zA-Z0-9():. \-]+\s+@\1 @gp')

Answer 4

A network test is likely to be the easiest to implement, and a little bit of regular traffic may also help avoid time-outs if that's part of the picture of why your connection drops. I take it that you're looking to test just the link to your ISP, so do something like pinging the ISPs name server rather than pinging google.

If you want to avoid sending network traffic for testing, then your best bet is likely to be to get information about network state from your router. How to do that depends on what router you use. Depending what you've got, you may be able to access it via telnet, ssh, snmp or http, and various options for scripting exist depending which you use.

With common home use oriented commodity routers, you probably will have http, but may not have much else. You might find that your router already does logging of network up/down events, and your problem is to script an http session to negotiate authentication to the router, and grab a copy of the existing log to update a copy on your raspbian. Most routers don't store the log, and will lose it on reboot.

Your most flexible options would likely involve running alternative linux-based firmware on your router.

Answer 5

Install the shell script described here: coupure shell script. <ad>This is a kind of ping made less verbose and smarter.</ad>

Once installed use it in parallel to detect drops on your ISP (for example, use one of its DNS servers, which should be of topmost connectivity), and on the Internet behind your ISP (for example use one of the Google servers).

With this parallel runs you will be able to know if you have a problem with your ISP access or with the Internet behind.

Answer 6

add to @Ijaz Khan post an email notification when not present (will be sent later on...)

#!/bin/bash

WGET="/usr/bin/wget"
while true;do
        $WGET -q --tries=20 --timeout=10 http://www.google.com -O   /tmp/google.idx &> /dev/null
        if [ ! -s /tmp/google.idx ]
        then
          echo "Not Connected..! $(hostname)"| mail -s "Not connected $(hostname) $(date)" root
        else
          echo "Connected..!"
        fi
        sleep 1
done