3

I'm trying mirror a website where it has page:

http://www.site.com/news

and contents:

http://www.site.com/news/36-news/news-one
http://www.site.com/news/37-news/news-two
http://www.site.com/news/38-news/another-news-here

This is what I'm using

wget -m -Dsite.com -e robots=off -r -q -R gif,png,jpg,jpeg,GIF,PNG,JPG,JPEG \
    -t 10 http://www.site.com &

When wget is run, it will create a HTML file with the name news.

I'm unable to download the contents because the file with the name news already exists (I'm running Ubuntu: directories cannot have same name as files)

Below is the message that appeared when running wget without -q

www.site.com/news/36-news: Not a directory
www.site.com/news/36-news/news-one: Not a directory

4

You might try --no-clobber. However, it seems to me like you'd probably be better suited with something a little more full-featured, such as httrack. Here's the manual for the command line options: http://www.httrack.com/html/fcguide.html

You can prefix all of your directories like this:

httrack "http://www.site.com/news" -O /tmp/site -N "my%p/%n.%t"

So the full command, similar to yours, could be something like:

httrack "http://www.site.com/news" -O /tmp/site -N "my%p/%n.%t" --retries=10 --ext-depth=0 --robots=0 +*.gif +*.png +*.jpg +*.jpeg +*.GIF +*.PNG +*.JPG +*.JPEG

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.