2
votes
4answers
99 views

Find + Grep Recursive Search

I am mirroring a website using wget. On the server, there is a dump of the find / command, which includes filenames in the form : /dir1/ /dir1/page1.txt /dir1/page2.txt ... The mirrorring is very ...
2
votes
1answer
391 views

Why is wget skipping files while trying to mirror a site

I ran wget on this site: http://demo.codestag.com/geeklove/ but it only returns a single index.html wget --recursive --no-clobber --page-requisites --html-extension --convert-links ...
0
votes
1answer
1k views

How to disable gzip compression with wget?

There are other questions asking how to enable gzip compression with wget, and lots of web pages out there telling how to do this, but I need the opposite. I'm trying to locally mirror a site, and ...
2
votes
1answer
2k views

How do I create a local copy of a complete website section from OSX using wget?

This question follows from: How do I create a local copy of a complete website section from OSX using curl? After discovering OSX's native curl wouldn't do this task I downloaded wget from here: ...
3
votes
3answers
3k views

Is there a way to disable wget from getting files from parent directories to given depth?

wget has such option as -np which disables getting files from any parent directory. I need something similar but a bit more flexible. Consider: www.foo.com/bar1/bar2/bar3/index.html I would like to ...
2
votes
1answer
562 views

How to wget website with same page and content names?

I'm trying mirror a website where it has page: http://www.site.com/news and contents: http://www.site.com/news/36-news/news-one http://www.site.com/news/37-news/news-two ...
2
votes
3answers
4k views

Why am I failing to mirror a web site (using wget)?

I have tried using wget --mirror http://tshepang.net/, but it only retrieves one page, "tshepang.net/index.html". Is this a bug in wget? Here's the output, from using the --debug option: DEBUG ...