0

Comments so far have not been specific with the type of files they needed to compress and didn't reach a definitive answer

I have a 1 TB hard drive full of pdf,djvu mostly and the rest are images and I need to compress it to the least size possible and I don't care about speed.

I tried kgb and it doesn't take full benefit of my cpu.

I tried tar.lzr, tar.lzma, and tar.7z. tar.lzr was the best but it didn't offer as much as I expected.

I also noticed nanozip but I saw many people reporting problems with it even though it had the highest percentage in the benchmarks I saw so does anyone have a previous experience with it?

4
  • 2
    You ask for "best for specific extensions", then don't specify exactly what you want to compress... Most image file formats & PDFs are compressed already, there's not much to be gained by re-compressing them in general.
    – Mat
    Commented Aug 29, 2015 at 8:12
  • @Mat Well i know that algorithms behave differently on different extensions as they could degrade or even be useless so i needed to get a hairpin answer .... actually some pdf files with kgb are compressed to half their size while others not so much as it depends on recognizable patterns by the algorithms Commented Aug 29, 2015 at 8:23
  • CPU performance won't affect the ratio of lossless compression especially when you don't care about speed.
    – yaegashi
    Commented Aug 29, 2015 at 8:41
  • @yaegashi I just thought that slower meant more calculations for less output so more CPU performance would mean more speed eventually Commented Aug 29, 2015 at 11:10

1 Answer 1

1

7z and lzma are the same compression algorithm, with a different container. 7z with solid archive mode enabled should do about as well as tar.7z, and provide not as bad random random access to a single file. (Still bad, though.)

pdf uses gzip internally, which makes it not very compressible. Same for most image formats (although the choice of entropy coder varies; IIRC JPEG's entropy coder is simpler).

In theory, and I don't know of any implementation of this idea, you could have an archiver that undoes the simple gzip or other entropy coding of pdf, png, jpeg, and various other already-compressed file formats. Then compress that stream with something good like LZMA. On extraction, you'd extract data from the LZMA stream, and redo the pdf-internal compression on the parts that needed it. Your output would be the same pdf/jpg/whatever, but the files would potentially have different checksums/hashes. (So it would be lossless with respect to the final rendered pixels, but not wrt the file bytes.)

PNG and jpeg optimizers sort of implement this idea for a single file: redo the internal compression with the equivalent of gzip -9 instead of the default.

Your best bet is to try compressing a sample of 1GB or so with various compressors, and see what does well. 1GB should be representative, because that's about as big as is reasonable for a dictionary size.

1
  • so There is no definitive answer, well zpaq is the best so far Commented Aug 29, 2015 at 11:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .