I want to write logic in shell script which will retry it to run again after 15 sec upto 5 times based on "status code=FAIL" if it fails due to some issue.

10 Answers 10

up vote 67 down vote accepted

This script uses a counter n to limit the attempts at the command to five. If the command is successful, $? will hold zero and execution will break from the loop.

   n=0
   until [ $n -ge 5 ]
   do
      command && break  # substitute your command here
      n=$[$n+1]
      sleep 15
   done
  • 1
    you should add break if command successes then it will break the loop – Rahul Patil Jul 11 '13 at 7:39
  • Actually the proper way to write that is if command; then break; fi or more succinctly just command && break – tripleee Jul 11 '13 at 8:05
  • 1
    "command" is just the name of the command that you want to check the status of. – suspectus Jul 11 '13 at 13:00
  • 3
    Worth noting that you can test if n equals five at the end to know if command succeeded or not. – mattdm Dec 23 '15 at 14:20
  • 3
    Nice solution--but in the case of n failures, it needlessly sleeps one additional time before exiting. – ron rothman ℝℝ Mar 11 '16 at 19:17
for i in 1 2 3 4 5; do command && break || sleep 15; done

Replace "command" with your command. This is assuming that "status code=FAIL" means any non-zero return code.


Variations:

Using the {..} syntax. Works in most shells, but not BusyBox sh:

for i in {1..5}; do command && break || sleep 15; done

Using seq and passing along the exit code of the failed command:

for i in $(seq 1 5); do command && s=0 && break || s=$? && sleep 15; done; (exit $s)

Same as above, but skipping sleep 15 after the final fail. Since it's better to only define the maximum number of loops once, this is achieved by sleeping at the start of the loop if i > 1:

for i in $(seq 1 5); do [ $i -gt 1 ] && sleep 15; command && s=0 && break || s=$?; done; (exit $s)
  • 24
    +1 — succinct and clear. One suggestion: I would replace for i in 1 2 3 4 5 with for i in {1..5} because it's easier to maintain. – Paddy Landau Jul 11 '13 at 13:03
  • 4
    Just a note, this works because the && is evaluated before the || because of operator precedence – gene_wood Sep 4 '15 at 19:50
  • 4
    Another note, this will return code 0 even if command fails. – Henrique Zambon May 26 '17 at 15:34
  • 3
    @HenriqueZambon Added a version that also handles that. – Alexander Jul 10 '17 at 13:14
  • 2
    Doesn't this sleep after the final failure? Seems like an unnecessary 15s wait. I think you can put a check for [[ i -eq 5]] as an OR condition before the sleep to avoid this. – Dave Lugg Jan 15 at 18:46
function fail {
  echo $1 >&2
  exit 1
}

function retry {
  local n=1
  local max=5
  local delay=15
  while true; do
    "$@" && break || {
      if [[ $n -lt $max ]]; then
        ((n++))
        echo "Command failed. Attempt $n/$max:"
        sleep $delay;
      else
        fail "The command has failed after $n attempts."
      fi
    }
  done
}

Example:

retry ping invalidserver

produces this output:

ping: unknown host invalidserver
Command failed. Attempt 2/5:
ping: unknown host invalidserver
Command failed. Attempt 3/5:
ping: unknown host invalidserver
Command failed. Attempt 4/5:
ping: unknown host invalidserver
Command failed. Attempt 5/5:
ping: unknown host invalidserver
The command 'ping invalidserver' failed after 5 attempts

For a real-world, working example with complex commands, see this script.

  • 2
    This is a great solution. I like that it exits with a non-zero exit status after something has failed multiple times as well. – Ben Liyanage Feb 23 '17 at 0:44

Here is function for retry

function retry()
{
        local n=0
        local try=$1
        local cmd="${@: 2}"
        [[ $# -le 1 ]] && {
        echo "Usage $0 <retry_number> <Command>"; }

        until [[ $n -ge $try ]]
        do
                $cmd && break || {
                        echo "Command Fail.."
                        ((n++))
                        echo "retry $n ::"
                        sleep 1;
                        }

        done
}

retry $*

Output :

[test@Nagios ~]$ ./retry.sh 3 ping -c1 localhost
PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.207 ms

--- localhost ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.207/0.207/0.207/0.000 ms

[test@Nagios ~]$ ./retry.sh 3 ping -c1 localhostlasjflasd
ping: unknown host localhostlasjflasd
Command Fail..
retry 1 ::
ping: unknown host localhostlasjflasd
Command Fail..
retry 2 ::
ping: unknown host localhostlasjflasd
Command Fail..
retry 3 ::
  • I copy pasted your code in a new file called retry.sh and added a line #!/bin/bash at the top. While running with your given commands in the explanation I don't see anything just prompt comes again. – java_enthu Aug 19 '13 at 10:54
  • have you tried bash retry.sh 3 ping -c1 localhost – Rahul Patil Aug 19 '13 at 11:26
  • Yes Rahul I did try. – java_enthu Aug 19 '13 at 11:29
  • Sorry, I was bizy.., I have tested again, it's working , check output paste.ubuntu.com/6002711 – Rahul Patil Aug 19 '13 at 12:35

GNU Parallel has --retries:

parallel --retries 5 --delay 15s ::: ./do_thing.sh

Here is my favorite one line alias / script

    alias retry='while [ $? -ne 0 ] ; do fc -s ; done'

Then you can do stuff like:

     $ ps -ef | grep "Next Process"
     $ retry

and it will keep running the prior command until it finds "Next Process"

  • 1
    In zsh, use fc -e "#" instead of fc -s. – Ricardo Stuven Aug 17 '17 at 14:55

I use this script that makes the retries of a given command, the benefit of this script is that if fails all retries it will preserve the exit code.

#!/usr/bin/env bash

if [ $# -ne 3 ]; then
    echo 'usage: retry <num retries> <wait retry secs> "<command>"'
    exit 1
fi

retries=$1
wait_retry=$2
command=$3

for i in `seq 1 $retries`; do
    echo "$command"
    $command
    ret_value=$?
    [ $ret_value -eq 0 ] && break
    echo "> failed with $ret_value, waiting to retry..."
    sleep $wait_retry
done

exit $ret_value

Probably it can get simplier

  • I like how flexible this version is and how verbose and readable the code is! – yo.ian.g Jan 6 '17 at 12:43
  • To match the failed echo you could even add a succeeded echo with the [ $ret_value -eq 0 ] or testing the $ret_value afterwards – yo.ian.g Jan 6 '17 at 12:49
  • This version has the advantage of not sleeping after the command fails for the last time. – Alexander Aug 19 '17 at 7:27

See below Example :

n=0
while :
do
        nc -vzw1 localhost 3859
        [[ $? = 0 ]] && break || ((n++))
        (( n >= 5 )) && break

done

I'm trying to connect port 3389 on localhost, it will retry until 5 times fail , if success then it will break the loop.

$? it's exist status of command if it zero means command successfully run , if other than zero means command fai

Seems little bit complicated, may be someone do it better than this.

  • Thanks rahul.. will it be keep retry to run the script ?? – Sandeep Singh Jul 11 '13 at 7:28
  • Please check now, I have updated – Rahul Patil Jul 11 '13 at 7:32
  • $? it's exist status of command if it zero means command successfully run , if other than zero means command fail – Rahul Patil Jul 11 '13 at 7:34
  • is it required to give host and port address. can we do it by giving script location dir only. – Sandeep Singh Jul 11 '13 at 7:43
  • replace with any command that give exit status code $? – Rahul Patil Jul 11 '13 at 7:44

You can use the loop command, available here, like so:

$ loop './do_thing.sh' --every 15s --until-success --num 5 

Which will do your thing every 15 seconds until it succeeds, for a maximum of five times.

Here's a recursive retry function for functional programming purists:

retry() {
  cmd=$1
  try=${2:-15}       # 15 by default
  sleep_time=${3:-3} # 3 seconds by default

  # Show help if a command to retry is not specified.
  [ -z "$1" ] && echo 'Usage: retry cmd [try=15 sleep_time=3]' && return 1

  # The unsuccessful recursion termination condition (if no retries left)
  [ $try -lt 1 ] && echo 'All retries failed.' && return 1

  # The successful recursion termination condition (if the function succeeded)
  $cmd && return 0

  echo "Execution of '$cmd' failed."

  # Inform that all is not lost if at least one more retry is available.
  # $attempts include current try, so tries left is $attempts-1.
  if [ $((try-1)) -gt 0 ]; then
    echo "There are still $((try-1)) retrie(s) left."
    echo "Waiting for $sleep_time seconds..." && sleep $sleep_time
  fi

  # Recurse
  retry $cmd $((try-1)) $sleep_time
}

Pass it a command (or a function name) and optionally a number of retries and a sleep duration between retries, like so:

retry some_command_or_fn 5 15 # 5 tries, sleep 15 seconds between each

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.