I was mirroring a site but something happened and wget died. So I had to run it again but I realized that Last-modified header missing -- time-stamps turned off. was repeated so often in the output so I suspect wget is downloading already downloaded files what should I do?

Here is my original command:

wget -m -nc -p -E -k -np -e robots=off www.site.com

Here is my resuming command:

wget -m -c -p -E -k -np -e robots=off www.site.com

-nc is --no-clobber while, -c is --continue. Is this what you intended?

However, it seems like the server doesn't support the If-Modified-Since header. That makes it harder for Wget to resume downloading. With the -c option, wget will still attempt to check if the file has been completely downloaded, but that again depends on the server's ability to handle Range: headers

We could help you more if atleast a part of Wget's output was shared. Preferably with the --debug switch enabled

  • But am I using the right options for continuing the interrupted mirror? I don't care if a few files will become corrupt – yukashima huksay Feb 9 '18 at 16:10
  • Seems like it, yes – darnir Feb 9 '18 at 18:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.