Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

im using wget in a bash script to get a list of files from a website, the files are sequential and in the format http://site.com/00001.csv etc, is there a way to terminate/end the script when I hit the last file?

This is the script I have so far.

#!/bin/bash 
url="site.com"; 
for a in {00000001..18923230} do 
wget -nc -q ${url}$a.csv 
done 
share|improve this question
1  
Do you have any script? It would be good if you post the script and then ask the question. –  Ramesh Mar 7 at 16:24
    
#!/bin/bash url="site.com"; for a in {00000001..18923230} do wget -nc -q ${url}$a.csv done –  Kevin Wincott Mar 7 at 16:25
    
Add an echo statement after done. –  Ramesh Mar 7 at 16:27
    
its going to be run on a cronjob so ideally it needs to terminate when it gets the first http 404 rather than iterating from 1 to 18923230 each time it runs (every 2 hours) –  Kevin Wincott Mar 7 at 16:29
add comment

2 Answers 2

The simplest way would be to check if the file exists. Using your current script:

#!/bin/bash 
url="site.com"; 
for a in {00000001..18923230}
do 
    wget -nc -q ${url}$a.csv || exit;
done 

That will exit as soon as one of the files is not downloadable. Alternatively, assuming there is an index.html file that links to these, you could download that and grep for the file names.

share|improve this answer
    
@devnull fair point, I used exit 'cause the OP said it should terminate whet it gets the 1st 404. –  terdon Mar 7 at 16:45
add comment

wget appears to return 0 when successful and non-zero otherwise, so:

    wget blahblah
    if [[ $? -ne "0" ]]; then
        break;
    fi

...inside your for loop.

share|improve this answer
    
Or you could just do if ! wget blahblah; then... or even better, what terdon did in his comment. –  orion Mar 7 at 16:41
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.