Sign up ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

There are other questions asking how to enable gzip compression with wget, and lots of web pages out there telling how to do this, but I need the opposite. I'm trying to locally mirror a site, and I'm just getting the home page as a gzipped file, which in turn breaks the recursion, so I can't get the whole site.

I can gunzip that file, but that still doesn't give me a recursive download of the whole cotton-pickin' site.

How do I turn off or prevent gzipping?

EDIT: The exact command I issued is

wget --random-wait -r -p -e robots=off -U mozilla --reject png,jpg,jpeg,gif  --progress=dot --wait=7
share|improve this question
Also, the manpage has no mention of gzipping at all. – iconoclast Mar 16 '13 at 1:15
wget does recursive downloading on its own, and I've never seen it to gzip (or otherwise pack) anything. Exactly what commands are you issuing? You might need to edit out parts of the URL for privacy, edit that into the question. – vonbrand Mar 16 '13 at 1:21

2 Answers 2

up vote 3 down vote accepted

D'oh! I figured it out. I had put

header = Accept-Encoding: gzip,deflate

in my ~/.wgetrc some time ago, assuming that it would only affect the way data was passed across the network, never thinking that wget would be unable to read the gzipped data.

In retrospect it makes sense: this is just a header that wget allows you to use (since it allows you to use any header that a browser might pass, or any you want to make up for that matter) rather than a switch built into wget, so why would anyone expect wget to automatically handle the gzipping? It would certainly be nice if it did, but there's no reason to assume that it would.

share|improve this answer

If specifying --header='Accept-Encoding: gzip,deflate' won't work, maybe this one would:

wget -O- | gunzip -c

Or by using curl and its libz library which suppose to do automatic decompression of compressed files over HTTP (curl -V | grep libz). It has also --compress parameter.

share|improve this answer
This may be useful, but it isn't quite aimed at what I'm actually trying to solve with this question. I'm trying to disable gzip, because it's preventing recursion. This answer doesn't solve that. – iconoclast May 9 at 1:17

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.