0

I wanted to compress a ~29GB file significantly, and used the tar command in the terminal like this :

 tar cvzf file.tar.gz /path/to/directory

and it compressed it to ~26GB, so I looked up maximum compression on the internet and did this :

 export GZIP=-9
 env GZIP=-9 tar cvzf file.tar.gz /path/to/directory

but still the tar.gz file size is showing to be around ~26GB in properties. I thought this time it would be compressed to ~10GB, say. Am I missing something here?

  • What evidence do you have that the file should be 10 Gb after compression? (Perhaps GZIP=-9 found no additional compressible bits) – Jeff Schaller Sep 7 '18 at 16:12
  • I am not saying I know it can be compressed to ~10GB, I only expected a heavy amount of compression. My question is can it be better? Say I want it to be compressed to ~10GB or less? I have very little space left in my hdd. – null Sep 7 '18 at 16:14
0

Yes, it does.

You may want to try some of the other compression formats that tar offers. On my Linux box, GNU tar offers this variety. That said, if gzip -9 is only achieving 29GB->26GB, it's quite unlikely that another compression format will achieve the desired 29GB->10GB you're seeking.

$ tar --help|grep -A16 Compression
 Compression options:

  -a, --auto-compress        use archive suffix to determine the compression
                             program
  -I, --use-compress-program=PROG
                             filter through PROG (must accept -d)
  -j, --bzip2                filter the archive through bzip2
  -J, --xz                   filter the archive through xz
      --lzip                 filter the archive through lzip
      --lzma                 filter the archive through lzma
      --lzop
      --no-auto-compress     do not use archive suffix to determine the
                             compression program
  -z, --gzip, --gunzip, --ungzip   filter the archive through gzip
  -Z, --compress, --uncompress   filter the archive through compress

 Local file selection:
$
-1

I would never use the default builtin compression ratio and if I know that the unpacking machine is able to give a few hundred MB of RAM, I would use xz instead of gzip.

So my advise is do pipe the uncompressed tar output through xz -9.

This needs significantly more CPU time while compressing but the CPU time for unpacking t needs only a bit more than you need with gzip. For text files, this results in 25-30% better compression.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.