Tag Info

Hot answers tagged

45

XZ_OPT=-9 tar cJf tarfile.tar.xz directory tar's lowercase j switch uses bzip, uppercase J switch uses xz. The XZ_OPT environment variable lets you set xz options that cannot be passed via calling applications such as tar. This is now maximal See man xz for other options you can set.


32

Tar is an archiving tool (Tape ARchive), it only collects files and their metadata together and produces one file. If you want to compress that file later you can use gzip/bzip2/xz. For convenience, tar provides arguments to compress the archive automatically for you. Checkout the tar man page for more details.


31

Compression ratio is very dependent of what you're compressing. The reason text compresses down so well is because it doesn't even begin to fully utilize the full range of numbers representable in the same binary space. So formats that do (e.g compressed files) can store the same information in less space just by virtue of using all those binary numbers that ...


31

It seems that the original bzip was pulled circa 1998 due to patent issues with the arithmetic compression used in. A bit of digging (really only reading Wikipedia) turns up an archived link to the bzip2 website from around this time. Here is the relevant section detail this and other differences: How does it relate to your previous offering (bzip-0.21) ...


21

It is also possible to decompress it using standard shell script + gzip. The trick is to prepend the gzip magic number and compress method (see http://www.onicos.com/staff/iz/formats/gzip.html) to the actual data: printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" |cat - zlib.raw |gzip -dc


21

Such an utility is zerofree. From its description: Zerofree finds the unallocated, non-zeroed blocks in an ext2 or ext3 file-system and fills them with zeroes. This is useful if the device on which this file-system resides is a disk image. In this case, depending on the type of disk image, a secondary utility may be able to reduce the size of the disk ...


18

Assuming xz honors the standard set of commandline flags - including compression level flags, you could try: tar -cf - foo/ | xz -9 -c - > foo.tar.xz


18

Most movie formats/encodings (and image formats too) are already compressed. You can't compress them much further by adding a second compression layer - same as if you try to zip a bzip2-compressed archive (or the other way around). You won't find a lossless compression algorithm that will compress these files much further. In fact, you might even end up ...


17

I have found a solution (one of the possible ones), it's using openssl: $ cat /tmp/data | openssl zlib -d ... *NOTE: zlib functionality is apparently available in recent openssl versions >=1.0.0 (OpenSSL has to be configured/built with zlib or zlib-dynamic option, the latter is default)


16

It depends on whether the disk image is a full disk image, or just a partition. Washing the partition(s) If the disk is in good working condition, you will get better compression if you wash the empty space on the disk with zeros. If the disk is failing, skip this step. If you're imaging an entire disk then you will want to wash each of the partitions on ...


15

You can configure the key bindings and set many settings for less in a file called ~/.lesskey. Once you've created the file, run the lesskey command; it generates a file called ~/.less which less reads when it starts. The setting you want is LESSOPEN. It's an input formatter for less. The less package comes with a sample formatter in /bin/lesspipe; it ...


14

Simple example, creating a gzipped tarball out of a directory, excluding a subdirectory and listing the contents: $ ls fruit/ apple banana peach tomato $ tar czf onlyfruit.tar.gz --exclude=tomato fruit/ $ tar tf onlyfruit.tar.gz fruit/ fruit/peach/ fruit/apple/ fruit/banana/


12

You compression attempts failed because your data is already highly compressed and there's not much more to gain, see the other answers for more detailed explanations. However, if you can agree on lossy compression, in contrast to lossless like you tried before, you can compress the images significantly. But since data is cut away, it can not be undone. ...


11

.Z files are compressed with the older compress utility while .gz are compressed with gzip. Some ancient systems might be missing gzip/gunzip so will use uncompress and .Z files.


11

Install zip and use zip -r foo.zip . You can use the flags -0 (none) to -9 (best) to change compressionrate Excluding files can be done via the -x flag. From the man-page: -x files --exclude files Explicitly exclude the specified files, as in: zip -r foo foo -x \*.o which will include the contents of foo in ...


11

AFAIK there is no limit of size for gzip - at least not 30GB. Of course, you need the space for the zipped file on your disc, both versions will be there simultanously while compressing. bzip2 compresses files (not only big ones :-) better, but it is (sometimes a lot) slower.


11

tar produces archives; compression is a separate functionality. However tar alone can reduce space usage when used on a large number of small files that are smaller than the filesystem's cluster size. If a filesystem uses 1kb clusters, even a file that contains a single byte will consume 1kb (plus an inode). A tar archive does not have this overhead. BTW, ...


10

See this answer. Quoted below for convenience: Calculate the bitrate you need by dividing 1 GB by the video length in seconds. So, for a video of length 16:40 (1000 seconds), use a bitrate of 1000000 bytes/sec: ffmpeg -i input.mp4 -b 1000000 output.mp4 Additional options that might be worth considering is setting the Constant Rate Factor, which ...


10

Most common image formats are already compressed (like jpg, png, gif), so you don't get much savings. 1% sounds about right. Adding more compression can actually make the result (slightly) larger, because the compression algorithm has no benefit on compressed data, and then the format (eg. gzip) has to add header and/or structure information to the output. ...


9

use the --exclude=directory flag. Works like a charm.


9

I use atool. It does the job. It works with many, though not all formats: tar, gzip, bzip2, bzip, lzip, lzop, lzma, zip, rar, lha, arj, arc, p7zip etc. These compression tools are still needed, though as atool is simply a front end for them. I particularly like the als command it provides which lists the contents of any supported archive format. The main ...


9

7zip is more a compactor (like PKZIP) than a compressor. It's available for Linux, but it can only create compressed archives in regular files, it's not able to compress a stream for instance. It's not able to store most of Unix file attributes like ownership, ACLs, extended attributes, hard links... On Linux, as a compressor, you've got xz that uses the ...


8

zlib implements the compression used by gzip, but not the file format. Instead, you should use the gzip module, which itself uses zlib. import gzip s = '...' with gzip.open('/tmp/data', 'w') as f: f.write(s)


8

You can use the unzip utility with the -v flag: unzip -v files.zip Archive: files.zip Length Method Size Cmpr Date Time CRC-32 Name -------- ------ ------- ---- ---------- ----- -------- ---- 0 Stored 0 0% 11-23-2011 15:02 00000000 file1 0 Stored 0 0% 11-23-2011 15:02 00000000 file2 -------- ...


8

Your h264 stream is likely to already be highly compressed. Likely by attempting to compress it further you will in fact inflate the file size slightly (through the addition of headers, etc). You can't merely continue compressing a file down to a size that you like without compromising elsewhere, gzip and other compressors are not magic. Likely the only way ...


8

lrzip is what you're really looking for, especially if you're compressing source code! Quoting the README: This is a compression program optimised for large files. The larger the file and the more memory you have, the better the compression advantage this will provide, especially once the files are larger than 100MB. The advantage can be chosen to ...


7

You can use p7zip. It automatically identifies the archive type and decompress it. p7zip is the command line version of 7-Zip for Unix/Linux, made by an independent developer. 7z e <file_name>


6

I'm not 100% sure, but as the initial ramdisk needs to be unpacked by the kernel during boot, cpio is used because it is already implemented in kernel code.


6

You may have success using /dev/stdout as the filename and piping the output of your application to gzip. /dev/stdout is a symlink to /proc/self/fd/1. Similarly, you may be able to use /dev/stdin as a filename and pipe the output of gzip to the application. I say may, because the application may be expecting a seekable file that it writes to (reads from), ...


6

Not really. You can't really seek to a specific block in the compressed file without decompressing the whole thing first, which makes it difficult to use the compressed image as a block device. You could use something like dump and restore (or tar, really), all of which use a streaming format...so you can access invidividual files by effectively scanning ...



Only top voted, non community-wiki answers of a minimum length are eligible