0

I would like to download all the books from the gutenberg project using wget. I want to get them in the epub format. The link to the list is: http://www.gutenberg.org/robot/harvest?filetypes[]=epub.images&langs[]=fr but when i run:

  wget -H -w 2 -m "http://www.gutenberg.org/robot/harvest?filetypes[]=epub.images&langs[]=fr"

It downloads the html page not all the files linked in the page. Hope this is clear enough. It is inspired from this question: https://webapps.stackexchange.com/questions/12311/how-to-download-all-english-books-from-gutenberg

1 Answer 1

0

You're doing everything right. It's Gutenberg's servers that are broken (temporarily, I hope).

What happens is: the pages you're fetching link to EPUB files under aleph.gutenberg.org/cache/, but aleph.gutenberg.org disallows robot access to the /cache. That's why Wget skips all the EPUB files.

I wrote an email to Project Gutenberg, and I hope they'll fix this soon — their contact page promises answers within two days. Just try again in a few days. (Or find a way to work around robots.txt. I don't want to publish it here because most of the time people misuse it to do bad stuff.)

Update November 29th: the problem have been fixed.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .