Tag Info

New answers tagged

2

Compressing a tar file with gzip, bzip2 or xz after it is created without compression, is the same tarring + compressing the original data with tar + the z, j, resp. J option.


2

Here's a (hopefully equivalent) Python version of Stephane Chazelas's solution python -c " import zlib from itertools import islice from functools import partial import sys with open(sys.argv[1]) as f: compressor = zlib.compressobj() t, z = 0, 0.0 for chunk in islice(iter(partial(f.read, 4096), ''), 0, None, 10): t += len(chunk) z += ...


3

You could try compressing one every 10 blocks for instance to get an idea: perl -MIPC::Open2 -nE 'BEGIN{$/=\4096;open2(\*I,\*O,"gzip|wc -c")} if ($. % 10 == 1) {print O $_; $l+=length} END{close O; $c = <I>; say $c/$l}' (here with 4K blocks).


5

As mentioned in the comment, the SO question had this pretty much covered. Now, I wanted to experiment on how this deflation actually works. So, I did the below testing. What is Entropy? Entropy is a measure of the unpredictability of an information stream. A perfectly consistent stream of bits (all zeroes, or all ones) is totally predictable (has no ...


1

The gzip format represents the the input size modulo 2^32, so the --list option reports incorrect uncompressed sizes and compression ratios for uncompressed files 4 GB and larger. So, take bzip2 (v1.0.2 and upwards).


2

If you have a moderately large website, unless it has a lot of user-contributed content, it has probably has a couple of megabytes of HTML, CSS, JavaScript, PHP, etc. This stuff can typically be compressed pretty well. Which means that the images are something like 99% of the tarball. Images are already compressed, so compressing them again won't reduce the ...


3

This is a question of lossless data compression, namely the one of its limitations. Usually the more random the data is the harder it is to compress, because compression basically is finding patterns and representing them with less information (you may think of it as of being able to predict following words from the beginning of a sentence). Thus a noise ...


0

With logrotate, log files can be compressed only one file at a time (that's a limitation, and not really a problem in practice, IMHO), and gzip is used by default, most probably for historical reasons and it has not changed yet. You can still use another compressor with the compresscmd command. See the logrotate(8) man page for more information. In the ...


5

Archiving (grouping files in a single entity) and compressing are usually done in separate steps under Unix. That gives more flexibility when you want to manipulate single files (that wouldn't benefit from archiving) — think pipelines such as gzip -cd mylog.gz | grep '^Oct.*error'. Achieving the same with 7z would be awkward.¹ 7z (and zip, and rar, and ...


6

Since tar files are a streaming format — you can cat two of them together and get an almost-correct result — you don't need to extract them to disk at all to do this. You can decompress (only) the files, concatenate them together, and recompress that stream: xzcat *.tar.xz | xz -c > combined.tar.xz combined.tar.xz will be a compressed tarball of all ...


1

You could mount a compressing filesystem on top of the tmpfs. If it would be still maintained, fusecompress would fit the purpose well. Unfortunately it seems to be outdated, so we need another implementation (it may be better to not use a FUSE-based filesystem for performance reasons anyway): It seems that both Btrfs and ZFS should work - but I can not ...



Top 50 recent answers are included