Tag Info

Hot answers tagged

39

XZ_OPT=-9 tar cJf tarfile.tar.xz directory tar's lowercase j switch uses bzip, uppercase J switch uses xz. The XZ_OPT environment variable lets you set xz options that cannot be passed via calling applications such as tar. This is now maximal See man xz for other options you can set.


32

Tar is an archiving tool (Tape ARchive), it only collects files and their metadata together and produces one file. If you want to compress that file later you can use gzip/bzip2/xz. For convenience, tar provides arguments to compress the archive automatically for you. Checkout the tar man page for more details.


31

Compression ratio is very dependent of what you're compressing. The reason text compresses down so well is because it doesn't even begin to fully utilize the full range of numbers representable in the same binary space. So formats that do (e.g compressed files) can store the same information in less space just by virtue of using all those binary numbers that ...


31

It seems that the original bzip was pulled circa 1998 due to patent issues with the arithmetic compression used in. A bit of digging (really only reading Wikipedia) turns up an archived link to the bzip2 website from around this time. Here is the relevant section detail this and other differences: How does it relate to your previous offering (bzip-0.21) ...


18

It is also possible to decompress it using standard shell script + gzip. The trick is to prepend the gzip magic number and compress method (see http://www.onicos.com/staff/iz/formats/gzip.html) to the actual data: printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" |cat - zlib.raw |gzip -dc


18

Most movie formats/encodings (and image formats too) are already compressed. You can't compress them much further by adding a second compression layer - same as if you try to zip a bzip2-compressed archive (or the other way around). You won't find a lossless compression algorithm that will compress these files much further. In fact, you might even end up ...


17

Assuming xz honors the standard set of commandline flags - including compression level flags, you could try: tar -cf - foo/ | xz -9 -c - > foo.tar.xz


16

I have found a solution (one of the possible ones), it's using openssl: $ cat /tmp/data | openssl zlib -d ... *NOTE: zlib functionality is apparently available in recent openssl versions >=1.0.0 (OpenSSL has to be configured/built with zlib or zlib-dynamic option, the latter is default)


15

Such an utility is zerofree. From its description: Zerofree finds the unallocated, non-zeroed blocks in an ext2 or ext3 file-system and fills them with zeroes. This is useful if the device on which this file-system resides is a disk image. In this case, depending on the type of disk image, a secondary utility may be able to reduce the size of the disk ...


14

You can configure the key bindings and set many settings for less in a file called ~/.lesskey. Once you've created the file, run the lesskey command; it generates a file called ~/.less which less reads when it starts. The setting you want is LESSOPEN. It's an input formatter for less. The less package comes with a sample formatter in /bin/lesspipe; it ...


13

Simple example, creating a gzipped tarball out of a directory, excluding a subdirectory and listing the contents: $ ls fruit/ apple banana peach tomato $ tar czf onlyfruit.tar.gz --exclude=tomato fruit/ $ tar tf onlyfruit.tar.gz fruit/ fruit/peach/ fruit/apple/ fruit/banana/


13

It depends on whether the disk image is a full disk image, or just a partition. Washing the partition(s) If the disk is in good working condition, you will get better compression if you wash the empty space on the disk with zeros. If the disk is failing, skip this step. If you're imaging an entire disk then you will want to wash each of the partitions on ...


12

You compression attempts failed because your data is already highly compressed and there's not much more to gain, see the other answers for more detailed explanations. However, if you can agree on lossy compression, in contrast to lossless like you tried before, you can compress the images significantly. But since data is cut away, it can not be undone. ...


11

.Z files are compressed with the older compress utility while .gz are compressed with gzip. Some ancient systems might be missing gzip/gunzip so will use uncompress and .Z files.


11

AFAIK there is no limit of size for gzip - at least not 30GB. Of course, you need the space for the zipped file on your disc, both versions will be there simultanously while compressing. bzip2 compresses files (not only big ones :-) better, but it is (sometimes a lot) slower.


11

tar produces archives; compression is a separate functionality. However tar alone can reduce space usage when used on a large number of small files that are smaller than the filesystem's cluster size. If a filesystem uses 1kb clusters, even a file that contains a single byte will consume 1kb (plus an inode). A tar archive does not have this overhead. BTW, ...


10

Most common image formats are already compressed (like jpg, png, gif), so you don't get much savings. 1% sounds about right. Adding more compression can actually make the result (slightly) larger, because the compression algorithm has no benefit on compressed data, and then the format (eg. gzip) has to add header and/or structure information to the output. ...


9

use the --exclude=directory flag. Works like a charm.


9

See this answer. Quoted below for convenience: Calculate the bitrate you need by dividing 1 GB by the video length in seconds. So, for a video of length 16:40 (1000 seconds), use a bitrate of 1000000 bytes/sec: ffmpeg -i input.mp4 -b 1000000 output.mp4 Additional options that might be worth considering is setting the Constant Rate Factor, which ...


9

I use atool. It does the job. It works with many, though not all formats: tar, gzip, bzip2, bzip, lzip, lzop, lzma, zip, rar, lha, arj, arc, p7zip etc. These compression tools are still needed, though as atool is simply a front end for them. I particularly like the als command it provides which lists the contents of any supported archive format. The main ...


8

zlib implements the compression used by gzip, but not the file format. Instead, you should use the gzip module, which itself uses zlib. import gzip s = '...' with gzip.open('/tmp/data', 'w') as f: f.write(s)


8

Install zip and use zip -r foo.zip . You can use the flags -0 (none) to -9 (best) to change compressionrate Excluding files can be done via the -x flag. From the man-page: -x files --exclude files Explicitly exclude the specified files, as in: zip -r foo foo -x \*.o which will include the contents of foo in ...


8

Your h264 stream is likely to already be highly compressed. Likely by attempting to compress it further you will in fact inflate the file size slightly (through the addition of headers, etc). You can't merely continue compressing a file down to a size that you like without compromising elsewhere, gzip and other compressors are not magic. Likely the only way ...


7

You can use p7zip. It automatically identifies the archive type and decompress it. p7zip is the command line version of 7-Zip for Unix/Linux, made by an independent developer. 7z e <file_name>


7

You can use the unzip utility with the -v flag: unzip -v files.zip Archive: files.zip Length Method Size Cmpr Date Time CRC-32 Name -------- ------ ------- ---- ---------- ----- -------- ---- 0 Stored 0 0% 11-23-2011 15:02 00000000 file1 0 Stored 0 0% 11-23-2011 15:02 00000000 file2 -------- ...


7

tar is actually not the best idea for medium-sized files (~10-50 MB): tar preserves creation time, modification time, permission, the directory structure, etc. which takes space. If this preservation is what you want, then XZ_OPT=-9e tar cJf tarfile.tar.xz directory is precisely what you need. If you don't care about FS metadata preservation, then give ...


6

You may have success using /dev/stdout as the filename and piping the output of your application to gzip. /dev/stdout is a symlink to /proc/self/fd/1. Similarly, you may be able to use /dev/stdin as a filename and pipe the output of gzip to the application. I say may, because the application may be expecting a seekable file that it writes to (reads from), ...


6

I'm not 100% sure, but as the initial ramdisk needs to be unpacked by the kernel during boot, cpio is used because it is already implemented in kernel code.


6

Not really. You can't really seek to a specific block in the compressed file without decompressing the whole thing first, which makes it difficult to use the compressed image as a block device. You could use something like dump and restore (or tar, really), all of which use a streaming format...so you can access invidividual files by effectively scanning ...


6

If read-only access is acceptable, then SquashFS is a good choice. However, it sounds like you want to be able to do in place updating as well. Btrfs may be an option for you. It is still considered somewhat experimental, but it does support transparent file compression, and is available to try in most distros. The other approach is to do this in ...



Only top voted, non community-wiki answers of a minimum length are eligible