I have created zlib-compressed data in Python, like this:

import zlib
s = '...'
z = zlib.compress(s)
with open('/tmp/data', 'w') as f:
    f.write(z)

(or one-liner in shell: echo -n '...' | python2 -c 'import sys,zlib; sys.stdout.write(zlib.compress(sys.stdin.read()))' > /tmp/data)

Now, I want to uncompress the data in shell. Neither zcat nor uncompress work:

$ cat /tmp/data | gzip -d -
gzip: stdin: not in gzip format

$ zcat /tmp/data 
gzip: /tmp/data.gz: not in gzip format

$ cat /tmp/data | uncompress -
gzip: stdin: not in gzip format

It seems that I have created gzip-like file, but without any headers. Unfortunately I don't see any option to uncompress such raw data in gzip man page, and the zlib package does not contain any executable utility.

Is there a utility to uncompress raw zlib data?

migrated from stackoverflow.com Oct 18 '11 at 8:48

This question came from our site for professional and enthusiast programmers.

10 Answers 10

up vote 121 down vote accepted

It is also possible to decompress it using standard + , if you don't have, or want to use or other tools.
The trick is to prepend the gzip magic number and compress method to the actual data from zlib.compress:

printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" |cat - /tmp/data |gzip -dc >/tmp/out

Edits:
@d0sboots commented: For RAW Deflate data, you need to add 2 more null bytes:
"\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x00"

This Q on SO gives more information about this approach. An answer there suggests that there is also an 8 byte footer.

Users @Vitali-Kushner and @mark-bessey reported success even with truncated files, so a gzip footer does not seem strictly required.

@tobias-kienzler suggested this function for the :
zlipd() (printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" |cat - $@ |gzip -dc)

  • gzip doesn't work, but zlib-flate does (pdf page content stream). – Daneel S. Yaitskov May 16 '17 at 12:23

User @tino commented below the OpenSSL answer but I think this should be separate:

zlib-flate -uncompress < FILE

I tried this and it worked for me.

zlib-flate can be found in package qpdf (in Debian Squeeze and Fedora 23, according to comments in other answers)

  • 3
    In contrast to the other answers, this one works on OS X. – polym Dec 29 '15 at 18:32
  • 1
    @polym, how did you get zlib-flate installed on macOS? I don't see it anywhere. – Wildcard Oct 2 '16 at 4:34
  • 2
    @Wildcard sorry for the late response. I think it came with the qpdf package that I've installed with brew as mentioned in the comment above - or look at the last sentence of this answer :). Also, qpdf is really cool, so have a look at it too if you have time! – polym Oct 16 '16 at 13:28

I have found a solution (one of the possible ones), it's using openssl:

$ openssl zlib -d < /tmp/data

or

$ openssl zlib -d -in /tmp/data

*NOTE: zlib functionality is apparently available in recent openssl versions >=1.0.0 (OpenSSL has to be configured/built with zlib or zlib-dynamic option, the latter is default)

  • 23
    On Debian Squeeze (which has OpenSSL 0.9.8) there is zlib-flate in the qpdf package. It can be used like zlib-flate -uncompress < FILE. – Tino Sep 16 '12 at 14:09
  • 4
    zlib got removed from the latest versions of OpenSSL so this tip is is very helpful @Tino – Alexandr Kurilin Dec 2 '14 at 10:59
  • 1
    Thanks. This solution provides a better experience in decompressing short input files than the answer using "gzip" ("openssl" decompressed as much as it could while "gzip" aborted printing "unexpected end of file"). – Daniel K. Sep 16 '15 at 10:01
  • 2
    @Tino this should be a separate answer – Catskul Nov 1 '15 at 3:16
  • 1
    @Tino, it is also available via the package qpdf on Fedora 23. Alexandr Kurilin, zlib is still available in 1.0.2d-fips. – maxschlepzig Nov 24 '15 at 8:37

I recommend pigz from Mark Adler, co-author of the zlib compression library. Execute pigz to see the available flags.

You will notice:

-z --zlib Compress to zlib (.zz) instead of gzip format.

You can uncompress using the -d flag:

-d --decompress --uncompress Decompress the compressed input.

Assuming a file named 'test':

  • pigz -z test - creates a zlib compressed file named test.zz
  • pigz -d -z test.zz - converts test.zz to the decompressed test file

On OSX you can execute brew install pigz

  • 5
    Good find! It looks like it can detect zlib files by itself, so unpigz test.zz will work as well. – Stéphane Chazelas Sep 26 '16 at 12:55

zlib implements the compression used by gzip, but not the file format. Instead, you should use the gzip module, which itself uses zlib.

import gzip
s = '...'
with gzip.open('/tmp/data', 'w') as f:
    f.write(s)
  • ok, but my situation is that i have tens/hundreds thousands of those files created, so.. :) – mykhal Sep 20 '11 at 22:14
  • 1
    so... your files are incomplete. Perhaps you'll have to uncompress them with zlib and recompress them with gzip, if you don't still have the original data. – Greg Hewgill Sep 20 '11 at 22:18
  • 6
    @mykhal, why did you create ten/hundred thousands of files before checking that you could actually uncompress them? – Harpyon Sep 20 '11 at 22:19
  • 1
    harpyon, i can uncompress them, i just wonder which less or more common urility or zgip settings can be used for that, if i don't want to do it in python again – mykhal Sep 20 '11 at 22:47

This might do it:

import glob
import zlib
import sys

for filename in sys.argv:
    with open(filename, 'rb') as compressed:
        with open(filename + '-decompressed', 'wb') as expanded:
            data = zlib.decompress(compressed.read())
            expanded.write(data)

Then run it like this:

$ python expander.py data/*
  • thanks, i know about zlib.decompress. probably i'd use some walk function. i'm not sure if shell would handle my huge amount of files with glob wildcard :) – mykhal Sep 20 '11 at 22:28
  • The file that is created by expanded still checks out as "zlib compressed data" for me, using the shell file command? How is that? – K.-Michael Aye Nov 30 at 0:18

The example program zpipe.c found here by Mark Adler himself (comes with the source distribution of the zlib library) is very useful for these scenarios with raw zlib data. Compile with cc -o zpipe zpipe.c -lz and to decompress: zpipe -d < raw.zlib > decompressed. It can also do the compression without the -d flag.

You can use this to compress with zlib:

openssl enc -z -none -e < /file/to/deflate

And this to deflate:

openssl enc -z -none -d < /file/to/deflate
  • 2
    Gives unknown option '-z' on Ubuntu 16.04 and OpenSSL 1.0.2g 1 Mar 2016 – Tino May 22 at 10:50
  • same error on Mac – K.-Michael Aye Nov 30 at 0:13

On macOS, which is a full POSIX compliant UNIX (formally certified!), OpenSSL has no zlib support, there is no zlib-flate either and while the first solution works as well as all the Python solutions, the first solution requires the ZIP data to be in a file and all the other solutions force you to create a Python script.

Here's a Perl based solution that can be used as a command line one-liner, gets its input via STDIN pipe and that works out of the box with a freshly installed macOS:

cat file.compressed | perl -e 'use Compress::Raw::Zlib;my $d=new Compress::Raw::Zlib::Inflate();my $o;undef $/;$d->inflate(<>,$o);print $o;'

Nicer formatted, the Perl script looks like this:

use Compress::Raw::Zlib;
my $decompressor = new Compress::Raw::Zlib::Inflate();
my $output;
undef $/;
$decompressor->inflate(<>, $output);
print $output;
zcat -f infile > outfile 

works for me on fedora25

  • zcat only works with files in the gzip format. – Anthony Geoghegan Oct 17 '17 at 8:50

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.