Tell me more ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.
up vote 10 down vote favorite
3

Is there something out there for parallel archiving of files?

Tar is great, but I don't use tape archives, and it's more important to me that the archiving happens quickly (with compression like bzip2) since I have smp.

share|improve this question
2  
tar is for more than just tapes. The name originally came from tape, but these days I see it being used mostly for when you want to put things into a single file for redistribution while maintaining directory structure information with optional compression. – Kevin M Oct 11 '10 at 14:48

3 Answers

up vote 16 down vote accepted

I think you are looking for pbzip2:

PBZIP2 is a parallel implementation of the bzip2 block-sorting file compressor that uses pthreads and achieves near-linear speedup on SMP machines.

Have a look at the project homepage or check your favorite package repository.

share|improve this answer
up vote 9 down vote

7zip can run on multiple threads when given the -mmt flag, but only when compressing into 7z-archives which offer great compression but are generally slower to create then zip archives. Do something like this.

7z a -mmt foo.7z /opt/myhugefile.dat
share|improve this answer
up vote 1 down vote

pigz is a parallel implementation of gzip, but can only really use multiple processors for compression, not decompression.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.