How to tell if a file is being downloaded by wget?

Question

I use wget to download files (most are zip files) automatically for me during the night.

However, sometimes in the morning I find that a few files cannot be unzipped. I don't know why this is happening, perhaps it's something wrong with the remote server.

I want to write a script to test zip files in my download folder periodically using 'unzip -t', but I don't want to test on the files that are being downloaded. So how can I tell if a file is being used by wget?

Answer 1

You can use fuser, or lsof.

fuser foo.zip

The output looks like so:

$ fuser archlinux-2013.02.01-dual.iso 
/home/chris/archlinux-2013.02.01-dual.iso: 22506
$ awk -F'\0' '{ print $1 }' /proc/22506/cmdline 
wget

Answer 2

Change your downloading script to download files to a temporary name or directory, then move the result when the downloading is finished. This is more robust than trying to detect that wget is still working; in particular, if wget is interrupted (e.g. by a reboot), you won't risk mistaking a partial file for a complete download. Also, be sure to test the return status of wget. Chances are that your broken files are in fact truncated files resulting from an interrupted download.

cd incoming &&
wget http://example.com/foo.zip &&
mv foo.zip ..

Answer 3

You can use lsof to check if the file is opened by some program.

for FILE in /path/to/files/*.zip
    do
        if [ -z "$(lsof $FILE)" ]
            then
                unzip $FILE
        fi
done

or something similar