1

I have experience with the Windows version:

GNU Wget 1.19.4 built on mingw32.

But now on Ubuntu, I have:

GNU Wget 1.19.4, a non-interactive network retriever.

There is a relatively new option called --compression that was added in 1.19.2:

  • New option --compression for gzip Content-Encoding

And in 1.19.4 release notes this is also talked about.

When running wget -h the --compression is missing on the Ubuntu version. How could I get a version that has it, or how do I enable the possibility of using it? When I try to run an actual command it just says:

wget: unrecognized option '--compression=auto'
  • 1
    @steeldriver yes, and those changes were merged upstream in wget 1.19.4; but they address the runtime behaviour (--compression defaults to none even when the build supports it). – Stephen Kitt Aug 23 '18 at 12:11
  • Without this, it downloads some files - like SVG - as .svg.gz and the contents of the file is actually compressed data. The .html files still reference an .svg file iirc. Even if it referenced the .svg.gz file it wouldn't work, at least not when browsing offline, locally. – Firsh - LetsWP.io Aug 23 '18 at 12:43
4

--compression is only available if wget is built with zlib (a library used for compression and decompression). The wget package in Debian doesn’t explicitly build-depend on that library, it gets it indirectly, via another library, the GNU TLS library; the Ubuntu build drops the latter, and ends up losing support for compression as a result.

You can see this in the build logs:

checking for ZLIB... no
checking for compress in -lz... no

Rebuilding the package to enable --compression can be done as follows:

cd /tmp
apt source wget
cd wget-1.19.4
apt build-dep wget
apt install zlib1g-dev
dch -n "Rebuild with compression support."
dpkg-buildpackage -us -uc

This will produce a package in /tmp, which you can then install with dpkg -i. You might need to adjust the cd step above, depending on the version of wget your repositories contain. You might also need to add deb-src lines in /etc/apt/sources.list, matching your deb lines, to be able to download the source code using apt source.

Note that enabling compression by default (--compression=auto) can have surprising consequences, which is why the release notes mention that

As it turns out, implementing gzip support is not trivial; especially in the face of many buggy servers that we have to support. Hence, for the time being, connection compression support has been marked as experimental and disabled by default.

See Debian bug 887910 for an example (thanks to steeldriver for mentioning it). I’ve filed Debian bug 907047 and Ubuntu bug 1788608 asking for a fix, if the maintainer agrees you’ll see the Ubuntu package acquire support for --compression once the fix is implemented (and migrates to Ubuntu, if the Debian package is fixed first).

  • Sorry, I'm a newbie but is it possible to (re)build it with zlib? – Firsh - LetsWP.io Aug 23 '18 at 12:00
  • No need to apologise, we were all newbies at some stage ;-). It should be possible, I’ll check and update my answer. – Stephen Kitt Aug 23 '18 at 12:01
  • It worked, yay! – Firsh - LetsWP.io Aug 23 '18 at 20:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.