Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Is it possible to compress a very large file (~30 GB) using gzip? If so, what commands, switches, and options should I use?

Or is there another program (preferably one commonly available on Ubuntu distributions) that I can use to compress/zip very large files? Do you have any experience with this?

share|improve this question

3 Answers 3

up vote 11 down vote accepted

AFAIK there is no limit of size for gzip - at least not 30GB. Of course, you need the space for the zipped file on your disc, both versions will be there simultanously while compressing.

bzip2 compresses files (not only big ones :-) better, but it is (sometimes a lot) slower.

share|improve this answer
3  
bzip2 is especially good at compressing source code (and the like). It is a good competitor to gzip for generic use too. Availability for gzip across different operating systems is better than bzip2 –  jippie Jun 3 '12 at 19:20
    
Is bzip2 lossless compression? Thanks. –  Andrew Jun 3 '12 at 20:03
3  
@Andrew Yes, gzip/bzip2/etc... are all lossless. –  Renan Jun 3 '12 at 20:06
1  
And there is pbzip, too - that will use more than one CPU-core. But still way slower than gzip. –  Nils Jun 10 '12 at 20:45
1  
@Nils: on my Ubuntu there is only pbzip2 –  rubo77 Aug 29 '13 at 16:31

if you need a good compresion rate, you can try lzma. it is faster and more efficient than bzip2 and may be faster even than gzip (i don't know that for sure)

http://www.thegeekstuff.com/2010/06/lzma-better-compression-than-bzip2-on-unix-linux/

share|improve this answer
The gzip format represents the the input size modulo 2^32, so the --list option reports incorrect uncompressed sizes and compression ratios for uncompressed files 4 GB and larger. 

So, take bzip2 (v1.0.2 and upwards).

share|improve this answer
    
If it should precisely be gzip only, then your file has to be divided into smaller parts up to 4 Gb each. –  cuneiform Sep 11 at 7:29

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.