The tag has no wiki summary.

learn more… | top users | synonyms

37
votes
1answer
2k views

What happened to bzip1?

bzip2 had been a de facto standard for quite strong compression throughout many years already. I myself had typed the bzip2 command thousands of times already, which makes me wonder - what happened to ...
30
votes
5answers
20k views

How to XZ a directory with TAR using maximum compression?

So I need to compress a directory with max compression. How can I do it with xz? I mean I will need tar too because I can't compress a directory with only xz. Is there a oneliner to produce e.g. ...
23
votes
5answers
18k views

Is it possible to mount a gzip compressed dd image on the fly?

I like create an image backup for the first time I'm backing up a system. After this first time I use rsync to do incremental backups. My usual image backup is as follows: Mount and zero out the ...
19
votes
4answers
14k views

Clear unused space with zeros (ext3,ext4)

How to clear unused space with zeros ? (ext3,ext4) I'm looking for something smarter than cat /dev/zero > /mnt/X/big_zero ; sync; rm /mnt/X/big_zero Like FSArchiver is looking for "used space" ...
17
votes
3answers
2k views

Does tar actually compress files, or just group them together?

I usually assumed that tar was a compression utility, but I am unsure, does it actually compress files, or is it just like an ISO file, a file to hold files?
15
votes
1answer
34k views

How can I reduce a video's size with ffmpeg?

How can I use ffmpeg to reduce the size of a video by lowering the quality (as minimally as possible, naturally, but I need it to run on a mobile device that doesn't have much available space)? I ...
10
votes
2answers
4k views

How to create a tar.gz file from a folder excluding a folder

How do I exclude a file or folder while creating a tar.gz file with tar command?
10
votes
5answers
8k views

Which file compression software for linux offers the highest size reduction? [closed]

I do a ton of file compression. Most of the stuff I am compressing is just code, so I need to use loss less compression. I was wondering if there was anything that offers a better size reduction ...
10
votes
2answers
3k views

How can I view gzipped files in less without having to type zless?

I am using Ubuntu, and I would like to be able to type less compressed_text_file.gz and page the contents of the text file in uncompressed form. Is there a way to do this?
9
votes
4answers
14k views

How to uncompress zlib data in UNIX?

I have created zlib-compressed data in Python, like this: import zlib s = '...' z = zlib.compress(s) with open('/tmp/data', 'w') as f: f.write(z) (or one-liner in shell: echo -n '...' | python2 ...
9
votes
3answers
1k views

Universal extractors

With an increasing number of archive/compression file formats, is there a single free/open-source command line tool to rule them all? Perhaps something with a consistent / unified set of flags? (note ...
8
votes
1answer
10k views

Zip everything in current directory

I'd like to compress and package everything, including files and folders in current directory, into a single ZIP file on Ubuntu. What would be the most convenient command for this (and name of the ...
7
votes
3answers
11k views

Is it possible to compress a very large file (~30 GB) using gzip?

Is it possible to compress a very large file (~30 GB) using gzip? If so, what commands, switches, and options should I use? Or is there another program (preferably one commonly available on Ubuntu ...
7
votes
6answers
205 views

Is there any program to provide a consistent interface across multiple archive types?

At the moment, if I download a compressed file, it could be any of a .tar.gz archive, a tar.bz2 arhive, a .zip archive or a .gz archive. And each time I do so, I have to remember what the command line ...
7
votes
3answers
688 views

How to recompress 2 million gzip files without storing them twice?

I have about 2 million (60GiB) of gzipped small files and I would like to create a compressed archive containing all of them in an uncompressed version. Unfortunately, I cannot just uncompress them ...
6
votes
2answers
1k views

Why use cpio for initramfs?

I am making my own initramfs following the Gentoo wiki. Instead of the familiar tar and gzip, the page is telling me to use cpio and gzip. Wikipedia says that cpio is used by the 2.6 kernel's ...
6
votes
1answer
76 views

How should I combine many compressed files into one archive?

I have a few hundred .tar.xz files which are almost identical (they are daily database dumps, and the database changes slowly). I believe that due to the similarities in the uncompressed files, they ...
6
votes
1answer
160 views

How to specify additional parameters for GNU sort's --compress-program option?

sort --compress-program=/bin/gzip works fine, but how can I pass options down to the compression programm (e.g. --best or --fast)? sort --compress-program=/bin/gzip\ --best failes with the error ...
6
votes
1answer
2k views

On the Fly Compression for a Directory

I have a directory that is always going to be storing text files that are rarely (think weekly) used. Naturally this is a great place to use compression. However, rather than having to use tar ever ...
5
votes
3answers
2k views

Is there a command to list the compressed file sizes for files within a .zip file?

I'd like to see the absolute size in bytes of each file that has been compressed into single zip file. Having read the zip man page, I'm not sure that that utility can do it. This is on Mac OS X. ...
5
votes
2answers
2k views

What's the difference between tar.gz and tar.Z?

The downloads on this page, e.g. of NetCDF C/C++/Fortran 4.1.3, show tar.Z and tar.gz formats. The tar.Z doesn't even have a configure file in it. What is it used for?
5
votes
3answers
4k views

speed up gzip compression

Is it possible to speed up the gzip process? I'm using mysqldump "$database_name" | gzip > $BACKUP_DIR/$database_name.sql.gz to backup a database into a directory, $BACKUP_DIR. the manpage ...
5
votes
1answer
691 views

Appending checksum information to file

I'd like to embed checksum information into file, that I transfer. It's tar.gz or tar.xz file and I can only transfer one file to remote side. How do you recommend I embed checksum information? I ...
5
votes
2answers
2k views

rsync-like tool that supports compression and maintains hardlinks?

I use an rsync-based backup scheme to maintain daily (and soon weekly and monthly as well) "snapshots" of my file server, using rsync's link-dest feature to store hardlinks between the snapshots for ...
5
votes
1answer
217 views

Command-line incremental backup tools

Which command line backup tool(s) has the best compression ratio? I want to backup my entire system, including media files, text files etc. I found this list on the Arch wiki site but I don't know ...
5
votes
1answer
175 views

Is there any way in BTRFS to set compression permanently?

I have a USB HDD. I want to mount it with compression enabled. I can do in the fstab of my system or even using udev rules. The problem is that I won't mount my USB HDD on my computer only. Up to ...
4
votes
1answer
1k views

Does ssh compress connections in a tunnel individually?

I have one compressed SSH tunnel with one tunnel and multiple connections over it. Data in every connection tunneled is rather homogeneous, but compressing an interleaved stream will obviously make ...
4
votes
2answers
507 views

Best compression of similar files?

I've got a few full old backups of things like binary database dumps. Obviously, they don't differ much so doing full backups is not the smartest idea here. For now, I'm looking for a compression ...
4
votes
1answer
4k views

What's the difference between gunzip and unzip?

What is the difference between gunzip and unzip? Tried to search but couldn't find anything relevant with this.
3
votes
8answers
2k views

Triple compression and I only save 1% in space?

I've been trying to save space on my linux server, and I had a folder containting, in subfolders, 22GB of images. So I decided to compress them. First I used tar: tar -zcf folder.tar folder Then ...
3
votes
4answers
327 views

Is there a way to redirect a file that is written

I have an application which I can not change, that writes to a big 250 GB plain text file. After compressing with GZ it is only 30 GB. This application has no option of compressing it's output, and ...
3
votes
2answers
681 views

Compressed filesystem inside a file in Linux

I have a flash drive which is FAT32 formatted. I want to put a linux filesystem on the drive inside a file. I know I can do this by creating a file and formatting is with ext3 (or any other file ...
3
votes
4answers
7k views

How to read a .war file?

I am trying to view the content of a .war file. I first did set its permissions with chmod 777 then when I try to accces it using: cd /usr/local/standalone/deployments/Sample.war/WEB-INF/classes/ ...
3
votes
2answers
3k views

What Linux utility can split a large file into a multi-file archive that can be opened on Windows?

I have a 6GB file which I need to transport on DVDs. I have a Linux system, while the recipient has a Windows system. Is there any way for me to make a multi-part archive to spread the file across two ...
3
votes
2answers
72 views

In what situation would a tarball (.tgz) be almost the same size as the original

I just tarballed a dump of some websites (configs, image directory, css directory, and site content (html files etc)) using tar czf sitedump.tgz backup_folder/. The original "backup_folder" was about ...
3
votes
3answers
579 views

Different count for compressed file listing vs directory listing

I was taking a backup of a folder, where I used the following command : sudo tar -zcvf www.tar.gz /var/www/ It successfully compressed, just for verifying the number of files in the compressed ...
3
votes
1answer
2k views

How to extract files from uImage?

Buildroot is generating Images for an embedded device where they should run on. This is working very well. In those Images the rootfs is included. Due to some research I'd like to look into that ...
3
votes
1answer
3k views

How to make nemo support compressing files by context menu?

I use Cinnamon on ArchLinux and Nemo is it's default file manager. I've tried 3 GUI file archivers (p7zip with WxGTK, peazip & file-roller), but none of them add a "compress option" in the context ...
3
votes
1answer
2k views

initramfs image compression other than gzip

can the initramfs image be compressed by a method other than gzip, such as lzma?
3
votes
1answer
346 views

Is there an in-memory compressed swap facility (like Compcache) for FreeBSD?

Is there anything like CompCache for FreeBSD?
3
votes
1answer
115 views

zfs-fuse: enabling compression has no effect

I installed on Debian Wheezy zfs-fuse file system and enabled compression gzip-9 on one dataset ("storage/backup"). When I check if the compression is enabled on this dataset, it shows YES: $: zfs ...
3
votes
1answer
101 views

What do you do to scan with 600 dpi while maintaing a small output file?

I just installed xsane which I already successfully tested with my Epson Perfection V30. When scanning with these settings the output file is remarkably large. Is there a way to decrease the size, ...
3
votes
1answer
2k views

Clear unused space with zeros (btrfs)

How to clear unused space with zeros ? (I am asking for tool for btrfs filesystem) I'm looking for something smarter than cat /dev/zero > /mnt/X/big_zero ; sync; rm /mnt/X/big_zero Like ...
2
votes
1answer
21 views

Is it possible and how to merge identical files between different snapshots in a btrfs file system?

I regularly use btrfs's snapshot to backup the whole system, but it's redundant to have identical files between different snapshots. Commonly, for example, I took a snapshot of @ as ...
2
votes
2answers
98 views

Estimate compressibility of file

Is there a quick and dirty way of estimating gzip-compressibility of a file without having to fully compress it with gzip? I could, in bash, do bc <<<"scale=2;$(gzip -c file | wc -c)/$(wc ...
2
votes
1answer
228 views

Zip does not compress files when zipping

I have zipped a file recursively in a directory. But what I notice with these last few zips is that the files do not get compressed. adding: ...
2
votes
1answer
405 views

Will using a compressed filesystem over an encrypted volume improve performance?

Encryption/decryption is often the main bottleneck when accessing an encrypted volume. Would using a filesystem with a fast transparent compression (such as BTRFS + LZO) help? The idea is that there ...
2
votes
1answer
187 views

Backing Up to a Compressed Archive on Linux

How would you go about backing up just some files and subdirectories of a given directory to a compressed archive so that the tree structure could then easily be recreated from that archive. ...
2
votes
1answer
39 views

cpio VS tar - what the best archive solution in order to compress hundred of directories to one file

I have hundreds of directories under /var/Recording, and each directory there will have subdirectories, including files, hard links and soft links. I want to compress all directories under ...
2
votes
2answers
496 views

Making tar split archives as it compresses?

I want to compress a file with tar but I'd like to know if its possible to do it while splitting the files. I know that in windows is possible to do this with winrar, I also know I can use split but ...