I want to use wget to recursively download a website. The site contains articles served by a php-script, eg. www.example.com/article.php?id=1021&ch=5 - these are what I want, all separate article with all chapters.

Unfortunately, the site also contains comments - lots and lots of comments. Comments are also served by a php-script, eg. www.example.com/comment.php?id=1021&no=144 - these I absloutely don't want.

Basically, I want everything except the comments. The problem is that links to articles, chapters and comments are interspersed on article-and chapter-lists, so there is no way to "fine tune" the level of recursion to avoid them... and doing level for level and manually sort-out between each, would take too much time and effort.

Since both are php-scripts, using the -A or -R options of wget to accept or reject files with a certain suffix won't work - or does it? Are there any other more precise way to exclude one php-script but not others when downloading recursively?


The following will provide some insight as to what you are looking for: http://www.gnu.org/software/wget/manual/html_node/Directory_002dBased-Limits.html

You should be able to use --exclude /comment.php to exclude all variations of the page comment.php

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.