I want to download the entire wikipedia to have an offline copy, I try using that command:
wget --recursive --domains --convert-links --no-clobber https://it.wikipedia.org/wiki/
But it download only the first page. How can I use it?
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.
Sign up to join this communityI want to download the entire wikipedia to have an offline copy, I try using that command:
wget --recursive --domains --convert-links --no-clobber https://it.wikipedia.org/wiki/
But it download only the first page. How can I use it?
Wikipedia’s robots.txt
excludes wget
:
#
# Sorry, wget in its recursive mode is a frequent problem.
# Please read the man page and use it properly; there is a
# --wait option you can use to set the delay between hits,
# for instance.
#
User-agent: wget
Disallow: /
Please follow their advice. In particular, you probably shouldn’t download Wikipedia using a tool such as wget
, but use their offline database download instead.