0

How can i skip a folder in a bash script using wget to batch download files, if the last file checked does not exist?

Here is the sample code:

#!/bin/bash
# Script to download Reports @ brazil.crl.edu/bsd/bsd
# Url : start @ http://brazil.crl.edu/bsd/bsd/u2038/ ends: /u2176
# There are less than 1000 files per address.
base1='http://brazil.crl.edu/bsd/bsd/u'
for i in {2038..2176};do 
    for s in {000001..001000}; do
        wget -r $base1$i'/'$s'.tiff'
        if [ $? -eq 0 ]; then
            break
        fi
    done
done

It's downloading only the first file. Can someone help?

  • You are querying for an exit status of 0, which stands for "no errors occurred". Seemingly all 1st files exist. Change this to if [ $? -ne 0 ] ; then, i.e. "any error occurred" and you should be fine. See man wget at the EXIT STATUS section. You could also use bash's boolean logic as wget -r $base1$i'/'$s'.tiff' || break which will execute break on a bad wget termination. – Fiximan Feb 21 '16 at 20:44
0

In order to enable you to mark this question as "solved", here the answer from my comment:

Your query

if [ $? -eq 0 ]

is checking for an exit status of 0, which stands for "No problems occurred" (according to man wget, EXIT STATUS section). Thus reversing the if-query to

if [ $? -ne 0 ]

i.e. "any problem occurred" will resolve the problem.

An exit status of 0 is the standard for successful execution of a command.

The if query could be reduced using bash's operator || which will check for unsuccessful command termination:

wget -r $base1$i'/'$s'.tiff' || break

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.