I have a bash script which uses set -o errexit so that on error the entire script exits at the point of failure.
The script runs a curl command which sometimes fails to retrieve the intended file - however when this occurs the script doesn't error exit.

I have added a for loop to

  1. pause for a few seconds then retry the curl command
  2. use false at the bottom of the for loop to define a default non-zero exit status - if the curl command succeeds - the loop breaks and the exit status of the last command should be zero.
#! /bin/bash

set -o errexit

# ...

for (( i=1; i<5; i++ ))
do
    echo "attempt number: "$i
    curl -LSso ~/.vim/autoload/pathogen.vim https://tpo.pe/pathogen.vim
    if [ -f ~/.vim/autoload/pathogen.vim ]
    then
        echo "file has been retrieved by curl, so breaking now..."
        break;
    fi

    echo "curl'ed file doesn't yet exist, so now will wait 5 seconds and retry"
    sleep 5
    # exit with non-zero status so main script will errexit
    false

done

# rest of script .....

The problem is when the curl command fails, the loop retries the command five times - if all attempts are unsuccessful the for loop finishes and the main script resumes - instead of triggering the errexit.
How can I get the entire script to exit if this curl statement fails?

up vote 10 down vote accepted

Replace:

done

with:

done || exit 1

This will cause the code to exit if the for loop exits with a non-zero exit code.

As a point of trivia, the 1 in exit 1 is not needed. A plain exit command would exit with the exit status of the last executed command which would be false (code=1) if the download fails. If the download succeeds, the exit code of the loop is the exit code of the echo command. echo normally exits with code=0, signally success. In that case, the || does not trigger and the exit command is not executed.

Lastly, note that set -o errexit can be full of surprises. For a discussion of its pros and cons, see Greg's FAQ #105.

Documentation

From man bash:

for (( expr1 ; expr2 ; expr3 )) ; do list ; done
First, the arithmetic expression expr1 is evaluated according the rules described below under ARITHMETIC EVALUATION. The arithmetic expression expr2 is then evaluated repeatedly until it evaluates to zero. Each time expr2 evaluates to a non-zero value, list is executed and the arithmetic expression expr3 is evaluated. If any expression is omitted, it behaves as if it evaluates to 1. The return value is the exit status of the last command in list that is executed, or false if any of the expressions is invalid. [Emphasis added]

  • Do you think it would be a good idea to put true before the break statement to be explicit and ensure the exit value of the loop? – RobertL Nov 6 '15 at 7:35
  • 1
    I think that explicit is better than implicit. That is why I wrote exit 1 when just plain exit would have worked. It is, though, a question of style and others can have their own opinions. – John1024 Nov 6 '15 at 7:37
  • 1
    works nicely! thanks :) personally I would read exit as a plain exit - that terminates the script in its own right. exit 1 would read to me as a "signal" to some other process (i.e. errexit) - that it should terminate the script based on the "result" of exit 1. - so I have gone with exit but thanks for explanation – the_velour_fog Nov 6 '15 at 7:58
  • 1
    If your script is exiting because of an error condition, you should call exit 1. That does not affect errexit at all. It merely tells the calling program that something went wrong. The false command contains one statement: exit(1). 99.9% of Unix commands return 0 on success and non-zero on error. Your's should too. – RobertL Nov 6 '15 at 8:27

If you have errexit set, then the false statement should cause the script to exit immediately. Same thing if the curl command failed.

You're example script, as written, should exit after the first curl command failure the first time it calls false if errexit is set.

To see how it works (I use the shorthand -e for to set errexit:

$ ( set -e;  false; echo still here )
$

$ ( set +e;  false; echo still here )
still here
$

So if the curl command executes more than once, this script doesn't have errexit set.

  • set -e is more subtle than that. It will not exit after the first failed command in a loop. You can prove that to yourself by running (set -e; for (( i=1; i<5; i++ )); do echo $i; false; done || echo "FAIL"; ) and noting that the code runs false four times. For more on set -e, see Greg's FAQ #105. – John1024 Nov 6 '15 at 7:55
  • @John1024 Thanks. This one's going down down down and out. – RobertL Nov 6 '15 at 7:57
  • @John1024 But I guess the evidence is still that errexit wasn't set. Please apply the logic to the script in the question. Run this: (set -e; for (( i=1; i<5; i++ )); do echo $i; false; done ; echo still here ) Yes testing return values with if while || && etc does not trigger errexit. The original script did not || the for loop. – RobertL Nov 6 '15 at 8:06
  • I just noticed I hadn't shown the set -o errexit command in my example code, have added it now - and for me it was not error exiting as expected. I needed to keep the false as the last command in the for loop, then close loop with done || exit [1] - then it worked nicely! – the_velour_fog Nov 6 '15 at 8:06
  • @RobertL I see your point. – John1024 Nov 6 '15 at 8:18

set -o errexit can be tricky in loops and subshells, because you have to pass the way back out of the process.

Breaking a loop (even in normal operation) is considered bad practice. You can call me old-school to prefer a while-loop rather than a for-loop for two conditions, but I find it better to read:

i=1
RET=-1
while [ $i -le 5 ] && [ $RET -ne 0 ]; do
    [ $i -eq 1 ] || sleep 5
    echo "attempt number: "$i
    curl -LSso ~/.vim/autoload/pathogen.vim https://tpo.pe/pathogen.vim
    RET=$?
    i=$((i+1))
done
exit $RET

If errexit is set and the curl command fails the script terminates right after the failed curl command. In the bash manual there is no hint that set -e ignores any failed return status of a single in a compound command. This would be only the case if the compound command is executed in a context where set -e is ignored.
https://www.gnu.org/software/bash/manual/bash.html#The-Set-Builtin

Try a slightly adapted example posted by RobertL. This stops at the first for-iteration right after the false command:

( set -e; for (( i=1; i<5; i++ )); do echo $i; false; echo "${i}. iteration done"; done ; echo "loop done" )

You can simply add the --fail option to the curl command, this will solve your problem, the script will fail and exit on error if the curl command fails, if very useful too when using curl in jenkins pipeline :

curl -LSso --fail ~/.vim/autoload/pathogen.vim https://tpo.pe/pathogen.vim

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.