What's the best way to estimate the compression level that can be achieved for a directory without storing the compressed files?
I have a large amount of directories and sub-directories with files with different nature (binaries, text files, compressed files, and so on). I need to compress some of these directories, but I don't have enough space to compress all of them (or even some of them) and them sort them by compression rate.
For this reason I need to list their compression rate (how much they can be compressed) to only then decide the ones that will be compressed.
I have a list of directories size for uncompressed directories:
du -f --maxdepth=3 > /tmp/list_of_directories
Tar, bz2 or gzip could be used to compress each directory using the fast compression level to a null pipe, but no compression rate statics are generated if the file is not preserved.
Since the entire contents must be somehow compressed to a zip pipe, this solution would be very processing intensive, but very light in terms of storage since the compressed results would be discarded as they are generated.
in short the question can be resumed to "Is there any tool that generates compression rate stats for a given stream, file or directory without storing the final compressed file?"