Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I'm writing a script that involves downloading and manipulating a file, and I want to ensure that the file is not incomplete (due to e.g. a dropped connection) before I work on it.

share|improve this question

2 Answers 2

up vote 4 down vote accepted

The most common way to verify the integrity of downloaded files is to use MD5 checksums. This assumes that the site you are downloading from actually published MD5 checksums of their files. You can verify a MD5 checksum by creating your own checksum of the downloaded file and comparing it to the published checksum. If they are identical the file you have downloaded is complete and not tampered with.

If you don't expect the file you are downloading to change you can precompute a checksum and hard code it into the script, but if the file is ever updated the verification will fail.

To create a MD5 checksum of a file run md5sum myFile. In the case of wget you might find this command useful, especially if the file you are downloading is large:

wget -O - http://example.com/myFile | tee myFile | md5sum > MD5SUM.

This will create a checksum of "myFile" while downloading and save it to the file MD5SUM, possibly saving you some time.

In the case of a dropped connection I think the best way would be to check the exit codes of wget. If the download is successful without any errors wget will return 0. Anything else indicates something went wrong. Take a look at the "Exit status" section of man wget.

share|improve this answer
2  
exit codes: gnu.org/software/wget/manual/html_node/… –  mikeserv Jun 7 at 13:48

The return code of the command used to download the file will tell you if the command executed successfully or not. Typically, a return code of 0 denotes success and any non-zero number denotes an error. You can access the return code through the $? variable.

A basic example using wget would go:

#!/bin/bash

wget foo.tgz &> /dev/null

if [[ "$?" != 0 ]]; then
    echo "Error downloading file"
else
    echo "Success"
fi

&> /dev/null redirects all of wget's output to /dev/null so it's ideal for scripting BUT it's makes debugging wget errors more difficult.

share|improve this answer
2  
you might do instead: wget -q ... || { handle ; error ; } –  mikeserv Jun 7 at 13:55
    
@mikeserv Didn't even know that was there, nice touch –  Creek Jun 7 at 13:58
1  
only found it in man while reading through to answer and the two things I was going to say were already here in two answers - so I did two comments. Nice touch to you, too. –  mikeserv Jun 7 at 13:59
    
I expect this to not work when used with SOCKS proxies like tor. –  CodesInChaos Jun 7 at 17:47
    
@CodesInChaos as long as wget can resolve the url thats passed to it, it should work. Proxied or not, it's just making a connection and downloading –  Creek Jun 7 at 18:38

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.