Shrinking (compressing) and restoring (decompressing) of data.

learn more… | top users | synonyms

3
votes
1answer
3k views

Trying to compress avi file using ffmpeg

I used the command below to compress an avi file(using ubuntu) ffmpeg -i input.avi -vcodec msmpeg4v2 output.avi but I am getting these errors Incompatible sample format 's16' for codec 'ac3', ...
1
vote
0answers
33 views

Proper way to create a compressed, rsyncable mirror of a directory?

I would like to create a compressed mirror of some directories in such a way that it can be rsynced again, both from the original destination and to a remote destination. I can do it inefficiently in ...
4
votes
2answers
3k views

What Linux utility can split a large file into a multi-file archive that can be opened on Windows?

I have a 6GB file which I need to transport on DVDs. I have a Linux system, while the recipient has a Windows system. Is there any way for me to make a multi-part archive to spread the file across two ...
0
votes
1answer
21 views

Reduce a size of a file(.jar) in unix

I want to reduce the big file (*.jar) in unix. i have tried with tar,bzip,gzip and zip commands. but all are same result not compressed much. Could any one help on this ???
1
vote
1answer
1k views

unpigz (and untar) to a specific directory

I know how to gunzip a file to a selected location. But when it comes to utilizing all CPU power, many consider pigz instead of gzip. So, the question is how do I unpigz (and untar) a *.tar.gz file ...
0
votes
0answers
34 views

How to deal with corrupted .tar.lrz file

I have a file file.tar.lrz and tried to unzip it via lrzuntar file.tar.lrz but it gives an error: "Not an lrzip file" with lrunzip. I also tried to open it with file roller. file file.tar.lrz says ...
2
votes
1answer
54 views

How can I decompress and print the last few lines of a compressed text file?

I have 6 gzipped text files, each of which is ~17G when compressed. I need to see the last few lines (decompressed) of each file to check whether a particular problem is there. The obvious approach is ...
39
votes
6answers
43k views

How to uncompress zlib data in UNIX?

I have created zlib-compressed data in Python, like this: import zlib s = '...' z = zlib.compress(s) with open('/tmp/data', 'w') as f: f.write(z) (or one-liner in shell: echo -n '...' | python2 ...
12
votes
2answers
12k views

How to extract files from uImage?

Buildroot is generating Images for an embedded device where they should run on. This is working very well. In those Images the rootfs is included. Due to some research I'd like to look into that ...
2
votes
3answers
79 views

Work with compressed dd images

How Can I compress dd images and work with them at the same time? (Like on Windows I can mark it as compressed, and work with it like a normal file ) I cant re-partition, but I can mount some drives ...
26
votes
5answers
33k views

Which file compression software for linux offers the highest size reduction? [closed]

I do a ton of file compression. Most of the stuff I am compressing is just code, so I need to use loss less compression. I was wondering if there was anything that offers a better size reduction ...
1
vote
1answer
49 views

How to compress all files in folder and erase the untared versions via command line?

I have a backup disk containing uncompressed versions of my files that I would like to compress to save space on this disk. The disk (containing files to compress) is full (about 3TB with 17GB free). ...
0
votes
1answer
49 views

Configure webserver for compression

Based on this page: http://code.google.com/speed/page-speed/docs/payload.html#GzipCompression I need to enable compression on my website, but I have no idea how to do this. If this is the right ...
1
vote
1answer
2k views

How do I turn on http compression?

I'm running a CentOS server and I want to save bandwidth, make pages faster, all that. How can I turn on http compression? Is it a module I enable for all sites? Can it be that simple?
48
votes
2answers
82k views

How can I reduce a video's size with ffmpeg?

How can I use ffmpeg to reduce the size of a video by lowering the quality (as minimally as possible, naturally, but I need it to run on a mobile device that doesn't have much available space)? I ...
1
vote
2answers
982 views

How to install Uglify on Debian Sid

I have a question about how to install Uglify or Uglify2 on Debian. I'm also wondering whether I have to install Node.js as well. My main goal of asking this question is to minify all my CSS and .js ...
4
votes
2answers
74 views

Print archive file list instantly (without decompressing entire archive)

A problem with .tar.gz archives is that, when I try to just list an archive's content, the computer actually decompresses it, which would take a very long time if the file is large. Other file ...
1
vote
1answer
34 views

On the fly compression of read- and writeable folder (like Windows EFS)?

Is there a possibility to compress the contents of a directory in Unix (especially AIX) on the fly, while still being able to read and write in the directory? Like the Windows EFS encryption does?
0
votes
0answers
23 views

The included VM could not be uncompressed (GZIP/UNCOMPRESS)

I have a .bin file to install Bladelogic NSH console and I want to install it in my OSX(Capitan). When I execute the .bin file in the Terminal I get the following error: Preparing to install... ...
6
votes
1answer
4k views

Clear unused space with zeros (btrfs)

How to clear unused space with zeros ? (I am asking for tool for btrfs filesystem) I'm looking for something smarter than cat /dev/zero > /mnt/X/big_zero ; sync; rm /mnt/X/big_zero Like ...
1
vote
1answer
45 views

Compressing a folder but do not compress specific file types but include them in the gz file

I am using tar with pigz to compress a folder and save a backup. Size of this folder ~250 GB or more. This folder has variety of content including numerous text and log files, ISOs and zip files in ...
3
votes
2answers
44 views

tar while avoiding compression of certain file types

I'm trying to create a very large backup of an entire hard disk, using tar. However, there are many files such as CR2 (Canon RAW 2), JPG and a few other files which are already compressed. I have ...
3
votes
1answer
42 views

Corrupted gz file

I have a MySQL database backup in a gz file. When trying to uncompress it I get the following: gzip: db_stepup.sql.gz: not in gzip format I read that sometimes is just a matter of removing the gz ...
0
votes
2answers
41 views

NTFS compression and RSync

I currently have a hard drive that I backup using RSync. I do weekly; monthly snapshots. I would like to compress some of my hard drive using NTFS compression. How would that affect my future ...
1
vote
0answers
26 views

Where is my 7za data being written to

I have a weird question about 7za on Centos 7. I am adding a file to a 7za archive using the following command (the total size of the directory is around 40GB) /usr/bin/7za a -ppassword -t7z -mmt=off ...
1
vote
2answers
774 views

Unzip error, is recovery possible?

I compressed 4 separate folders with some wedding videos (!!) a few months ago. Today I tried to unzip them, and had no problem with the last 2, but the first 2 of them show an error: ...
0
votes
3answers
76 views

Compress and remove original directory with limited storage

On my 3 TB external hard drive, I have a 2.7 TB directory containing relatively small files. I would like to compress this 2.7 TB directory and remove it to keep only the compressed version. The issue ...
93
votes
8answers
11k views

Why would I tar a single file?

At my company, we download a local development database snapshot as a db.dump.tar.gz file. The compression makes sense, but the tarball only contains a single file (db.dump). Is there any point to ...
43
votes
6answers
69k views

How do I grep recursively through .gz files?

I am using a script to regularly download my gmail messages that compresses the raw .eml into .gz files. The script creates a folder for each day, and then compresses every message into its own file. ...
0
votes
1answer
39 views

how to compress all folders in a directory

I have a directory full of folders and I was wondering if there is any command to compress all the folders in a directory X with date in the name of the file and not update old ones in case they ...
17
votes
3answers
4k views

Fastest way of working out uncompressed size of large GZIPPED file

Once a file is gzipped, is there a way of quickly querying it to say what the uncompressed file size is (without decompressing it), especially in cases where the uncompressed file is > 4GB in size. ...
0
votes
1answer
49 views

gzip - is there a risk to lose files during compression if process dead?

Is there a risk of losing files during compression if process dies? I'm gzipping big files using wildcard, but accidentally running it without screen or nohup. Is there a risk to lose source files if ...
3
votes
1answer
75 views

GNU sort --compress-program compressing only first temporary

I am sorting big files (>100Go), and to reduce time spent on disk writes, I am trying to use GNU sort's --compress-program parameter. (Related: How to sort big files?) However, it appears in certain ...
0
votes
1answer
71 views

Linux Gzip increasing size

I try to gzip a file abc.log which has size of 111 bytes, but after gzip, the size of the file increased to 125 bytes, why is that? Is it when i perform gzip, it will create header and trailer that ...
1
vote
1answer
77 views

How do I reduce the size of a pdf file that contains images?

I have a pdf file that contains images and I want to reduce its size in order to upload it to a site with a size limit. So, how can I reduce the size of a pdf file from the command-line?
2
votes
2answers
106 views

Linux + how to compress the old log files automatically

Under /var/log/httpd folder, files are created with huge capacity!!! What solution can I use in order to compress the old files automatically (can logrotate handle this)? If yes, how to configure ...
10
votes
4answers
23k views

Is it possible to compress a very large file (~30 GB) using gzip?

Is it possible to compress a very large file (~30 GB) using gzip? If so, what commands, switches, and options should I use? Or is there another program (preferably one commonly available on Ubuntu ...
11
votes
3answers
9k views

How can I view gzipped files in less without having to type zless?

I am using Ubuntu, and I would like to be able to type less compressed_text_file.gz and page the contents of the text file in uncompressed form. Is there a way to do this?
2
votes
3answers
80 views

Unable to use tar to run an archive through bzip2 compression

I am having difficulty compression an archive with bzip using tar. The archive I have is user-logs.tar. I am attempting to run this: tar -jf user-logs.tar I get this error: tar: You must ...
13
votes
9answers
4k views

Memory problems when compressing and transferring a large number of small files (1TB in total)

I have 5 million files which take up about 1TB of storage space. I need to transfer these files to a third party. What's the best way to do this? I have tried reducing the size using .tar.gz, but ...
1
vote
1answer
71 views

linux unbuffered compression options

I'm working on a embedded system and due to persistent memory restrictions, I need to compress a log file "on the fly". My objective is to have a one liner inside a script, which would look like this:...
1
vote
2answers
46 views

Squashfs minimal compression?

I'd like to create a squashfs image of a directory, but it's a huge one and even minimal compression would be a big win. Is there a way to do something like mksquashfs -1?
2
votes
1answer
53 views

Is there a compression method that supports solid compression and also adding data to the compressed file?

I have a large compressed .tar.xz file containing log files. The compression ratio is very good - but it takes a long time to compress, and if I want to add additional log files to it, I have to ...
1
vote
0answers
34 views

On-the-fly decompression

I have 44 TB worth of gzipped files that are 1.5 GB each when compressed or 2.0 GB when uncompressed. I have a tool that can read only uncompressed files. I would like to avoid the overhead of ...
35
votes
5answers
38k views

Clear unused space with zeros (ext3,ext4)

How to clear unused space with zeros ? (ext3,ext4) I'm looking for something smarter than cat /dev/zero > /mnt/X/big_zero ; sync; rm /mnt/X/big_zero Like FSArchiver is looking for "used space" ...
1
vote
1answer
226 views

log compression using logrotate

I was trying to set compression for jboss log files. The log files I want to compress are console.log and server.log. Compression for console.log is working fine, but server.log I am seeing issues. I ...
68
votes
7answers
48k views

How to XZ a directory with TAR using maximum compression?

So I need to compress a directory with max compression. How can I do it with xz? I mean I will need tar too because I can't compress a directory with only xz. Is there a oneliner to produce e.g. foo....
6
votes
2answers
191 views

How do I layer compression on top of encryption on top of BtrFS, per-file?

BtrFS should support this really, but it doesn't. So ECryptFS looks like it will fill the gap. The only thing is, how would I layer compression on top of this? Reasons for compression on top of ...
0
votes
2answers
101 views

Tar big files in different directories

I have ~20 files, each about 10 GB in size, in different folders that I need to compress into a single tar. It is not practical to copy them all to one location. More precisely, the files are ...
0
votes
1answer
170 views

Compressing files using p7zip from the command-line

I would like to compress four ISO files using 7z into a new archive called ISOs.7z from the command-line on my Sabayon machine. These are the commands I have tried so far (I know none of these specify ...