I want to download the entire wikipedia to have an offline copy, I try using that command:

wget --recursive --domains --convert-links --no-clobber https://it.wikipedia.org/wiki/

But it download only the first page. How can I use it?


1 Answer 1


Wikipedia’s robots.txt excludes wget:

# Sorry, wget in its recursive mode is a frequent problem.
# Please read the man page and use it properly; there is a
# --wait option you can use to set the delay between hits,
# for instance.
User-agent: wget
Disallow: /

Please follow their advice. In particular, you probably shouldn’t download Wikipedia using a tool such as wget, but use their offline database download instead.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.