0

I'm using wget to mirror an old website with all of its --page-requisites. Unfortunately, one of the domains which used to serve some of these requisites - is now offline. Every time wget tries to get anything from that domain, it blocks for 10+ seconds until the connection finally times out.
This issue is substantially exacerbated by the 20 retries. I want to keep the default retry count at 20, but I know this one particular domain is never going to respond.
Is there a way to tell wget to ignore that domain entirely, i.e. not try to fetch anything from there?
If not, can I at least apply different configurations to request from that domain, for example specify 0 retries for it?
Otherwise, any other suggested solutions for this problem?

  • 1
    wget man pages talks about an option --exclude-domains . Did you try that? – Sree Nov 26 '14 at 5:26
  • re-posted it as the answer – Sree Nov 26 '14 at 7:15
  • @Sree: where? I don't see an answer for this question. If you do, I'll mark it as the solution. – Dun Peal Nov 26 '14 at 19:58
1

You can use the option --exclude-domains with wget to exclude specific domains.

From the man page of wget:

--exclude-domains domain-list
           Specify the domains that are not to be followed.

.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.