wget - How to download recursively and only specific mime-types/extensions (i.e. text only)

Question

How to download a full website, but ignoring all binary files.

wget has this functionality using the -r flag but it downloads everything and some websites are just too much for a low-resources machine and it's not of a use for the specific reason I'm downloading the site.

Here is the command line i use: wget -P 20 -r -l 0 http://www.omardo.com/blog (my own blog)

Answer 1

You could specify a list of allowed resp. disallowed filename patterns:

Allowed:

-A LIST
--accept LIST

Disallowed:

-R LIST
--reject LIST

LIST is comma-separated list of filename patterns/extensions.

You can use the following reserved characters to specify patterns:

• *
• ?
• [
• ]

Examples:

• only download PNG files: -A png
• don't download CSS files: -R css
• don't download PNG files that start with "avatar": -R avatar*.png

If the file has no extension resp. the file name has no pattern you could make use of, you'd need MIME type parsing, I guess (see Lars Kotthoffs answer).

Answer 2

You could try patching wget with this (also here) to filter by MIME type. This patch is quite old now though, so it might not work anymore.