1

I am downloading a couple of thousand files listed in a file using:

wget -i filename

Sometimes it encounters the following error reported by the server for a particular file:

HTTP request sent, awaiting response... 500 Internal Server Error

The problem is that wget then hangs. What I want for it to skip that file and continue downloading the rest of the list. How can I do that?

3

Try using wget --tries=1 --waitretry=1 -i filename. This will try only once after a failure and only wait one second before doing so. It's also possible the server is not closing the socket after sending the 500 error. In this case, adding --read-timeout=30 will timeout the connection after 30 seconds of no data from the server. See the manual

  • 1
    --tries=0 will retry indefinitely. The server may be intentionally leaving the connection open - try setting --read-timeout=15 – Outurnate Oct 3 '17 at 20:52
  • @JBentley On it – Outurnate Oct 3 '17 at 23:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.