Shrinking (compressing) and restoring (decompressing) of data.

learn more… | top users | synonyms

1
vote
1answer
46 views

How to compress all files in folder and erase the untared versions via command line?

I have a backup disk containing uncompressed versions of my files that I would like to compress to save space on this disk. The disk (containing files to compress) is full (about 3TB with 17GB free). ...
2
votes
3answers
67 views

Work with compressed dd images

How Can I compress dd images and work with them at the same time? (Like on Windows I can mark it as compressed, and work with it like a normal file ) I cant re-partition, but I can mount some drives ...
1
vote
1answer
25 views

On the fly compression of read- and writeable folder (like Windows EFS)?

Is there a possibility to compress the contents of a directory in Unix (especially AIX) on the fly, while still being able to read and write in the directory? Like the Windows EFS encryption does?
0
votes
0answers
12 views

The included VM could not be uncompressed (GZIP/UNCOMPRESS)

I have a .bin file to install Bladelogic NSH console and I want to install it in my OSX(Capitan). When I execute the .bin file in the Terminal I get the following error: Preparing to install... ...
4
votes
2answers
59 views

Print archive file list instantly (without decompressing entire archive)

A problem with .tar.gz archives is that, when I try to just list an archive's content, the computer actually decompresses it, which would take a very long time if the file is large. Other file ...
1
vote
1answer
31 views

Compressing a folder but do not compress specific file types but include them in the gz file

I am using tar with pigz to compress a folder and save a backup. Size of this folder ~250 GB or more. This folder has variety of content including numerous text and log files, ISOs and zip files in ...
3
votes
2answers
41 views

tar while avoiding compression of certain file types

I'm trying to create a very large backup of an entire hard disk, using tar. However, there are many files such as CR2 (Canon RAW 2), JPG and a few other files which are already compressed. I have ...
3
votes
1answer
39 views

Corrupted gz file

I have a MySQL database backup in a gz file. When trying to uncompress it I get the following: gzip: db_stepup.sql.gz: not in gzip format I read that sometimes is just a matter of removing the gz ...
0
votes
2answers
32 views

NTFS compression and RSync

I currently have a hard drive that I backup using RSync. I do weekly; monthly snapshots. I would like to compress some of my hard drive using NTFS compression. How would that affect my future ...
1
vote
0answers
25 views

Where is my 7za data being written to

I have a weird question about 7za on Centos 7. I am adding a file to a 7za archive using the following command (the total size of the directory is around 40GB) /usr/bin/7za a -ppassword -t7z -mmt=off ...
0
votes
3answers
57 views

Compress and remove original directory with limited storage

On my 3 TB external hard drive, I have a 2.7 TB directory containing relatively small files. I would like to compress this 2.7 TB directory and remove it to keep only the compressed version. The issue ...
0
votes
1answer
39 views

how to compress all folders in a directory

I have a directory full of folders and I was wondering if there is any command to compress all the folders in a directory X with date in the name of the file and not update old ones in case they ...
93
votes
8answers
11k views

Why would I tar a single file?

At my company, we download a local development database snapshot as a db.dump.tar.gz file. The compression makes sense, but the tarball only contains a single file (db.dump). Is there any point to ...
0
votes
1answer
44 views

gzip - is there a risk to lose files during compression if process dead?

Is there a risk of losing files during compression if process dies? I'm gzipping big files using wildcard, but accidentally running it without screen or nohup. Is there a risk to lose source files if ...
3
votes
1answer
67 views

GNU sort --compress-program compressing only first temporary

I am sorting big files (>100Go), and to reduce time spent on disk writes, I am trying to use GNU sort's --compress-program parameter. (Related: How to sort big files?) However, it appears in certain ...
1
vote
1answer
64 views

How do I reduce the size of a pdf file that contains images?

I have a pdf file that contains images and I want to reduce its size in order to upload it to a site with a size limit. So, how can I reduce the size of a pdf file from the command-line?
0
votes
1answer
58 views

Linux Gzip increasing size

I try to gzip a file abc.log which has size of 111 bytes, but after gzip, the size of the file increased to 125 bytes, why is that? Is it when i perform gzip, it will create header and trailer that ...
2
votes
2answers
82 views

Linux + how to compress the old log files automatically

Under /var/log/httpd folder, files are created with huge capacity!!! What solution can I use in order to compress the old files automatically (can logrotate handle this)? If yes, how to configure ...
2
votes
3answers
72 views

Unable to use tar to run an archive through bzip2 compression

I am having difficulty compression an archive with bzip using tar. The archive I have is user-logs.tar. I am attempting to run this: tar -jf user-logs.tar I get this error: tar: You must ...
1
vote
1answer
68 views

linux unbuffered compression options

I'm working on a embedded system and due to persistent memory restrictions, I need to compress a log file "on the fly". My objective is to have a one liner inside a script, which would look like this:...
1
vote
2answers
42 views

Squashfs minimal compression?

I'd like to create a squashfs image of a directory, but it's a huge one and even minimal compression would be a big win. Is there a way to do something like mksquashfs -1?
2
votes
1answer
43 views

Is there a compression method that supports solid compression and also adding data to the compressed file?

I have a large compressed .tar.xz file containing log files. The compression ratio is very good - but it takes a long time to compress, and if I want to add additional log files to it, I have to ...
1
vote
0answers
31 views

On-the-fly decompression

I have 44 TB worth of gzipped files that are 1.5 GB each when compressed or 2.0 GB when uncompressed. I have a tool that can read only uncompressed files. I would like to avoid the overhead of ...
1
vote
1answer
188 views

log compression using logrotate

I was trying to set compression for jboss log files. The log files I want to compress are console.log and server.log. Compression for console.log is working fine, but server.log I am seeing issues. I ...
0
votes
2answers
91 views

Tar big files in different directories

I have ~20 files, each about 10 GB in size, in different folders that I need to compress into a single tar. It is not practical to copy them all to one location. More precisely, the files are ...
0
votes
1answer
142 views

Compressing files using p7zip from the command-line

I would like to compress four ISO files using 7z into a new archive called ISOs.7z from the command-line on my Sabayon machine. These are the commands I have tried so far (I know none of these specify ...
0
votes
1answer
35 views

Verify files AFTER extracting gzip

I gzip a folder, transfer it to a remote location and then extract it to a folder. The problem is that this machine is a little unreliable and sometimes writes don't succeed and there can be zero-byte ...
3
votes
2answers
137 views

Extract compressed file by it's header as it is piped from stdout (locally or from remote location)?

I am sending compressed file with piping either local or from a network location. And on the receiving end, I would like to detect the type of compression and use the appropriate decompression utility ...
1
vote
1answer
236 views

How to convert tar.bz2 to .bz2?

I have a file with compression tar.bz2 , I would like to convert it to .bz2 . Is there a command that does this in one shot? I tried the below but no joy ! tar -xvjf US.tar.bz2 US bzip2: ...
2
votes
3answers
141 views

How can I keep rsync from using encryption and compression for the file transfers after the connection is authorized?

I understand that a secure connection (i.e. ssh) is needed to authorize a connection to the remote server. But after that is authorized can the data be transmitted without encryption and compression ...
5
votes
1answer
272 views

Get compressed size of piped output with gzip?

To get uncompressed size of already compressed file, I can use -l option in gzip utility: gzip -l compressedfile.gz However is there a way to get size of compressed file if I am piping the output? ...
22
votes
4answers
2k views

How to know how much space an uncompressed zip will take

Given a (really long) list of zip files, how can you tell the size of them once uncompressed?
0
votes
1answer
81 views

Browser (HTTP client) with support of bzip2 or lzma2 compression?

What browsers (HTTP clients) support HTTP compression using bzip2 or lzma2? (HTTP compression for page requests) I have only heard of Firefox patch, but I am not sure if it actually supports ...
0
votes
1answer
26 views

Generating files of varying variability?

I have used /dev/zero and /dev/random to generate files of certain size. However, is there a way to control variability of generated files? I would like to create few files that would have same ...
4
votes
3answers
320 views

How can I change the first line of a big gzip file without decompressing all of it?

I currently have a compressed file, A.gz which contains lots of tabulated data, including a header in the first line. I want to create another file, B.gz, which has the same data as the previous file, ...
0
votes
1answer
184 views

What is the highest compression ratio compressor for pdf, djvu, and images

Comments so far have not been specific with the type of files they needed to compress and didn't reach a definitive answer I have a 1 TB hard drive full of pdf,djvu mostly and the rest are images and ...
1
vote
0answers
42 views

How do I create a deflate file suitable for HTTP?

Say I have index.html. How do I obtain index.html.deflate appropriate for serving through HTTP with the "deflate" encoding? I am coding a simple web server and I don't want to implement streaming ...
4
votes
3answers
603 views

Compression / archiving that keeps permissions and file owner?

I know that similar questions have been asked about permissions, but which form of compression or archiving keeps the permissions and file owners of each file and directory? I was considering the tar....
11
votes
3answers
737 views

What does the `-C` flag exactly do in `scp`?

I always use either rsync or scp in order to copy file from/to a remote machine. Recently, I discovered in the manual of scp (man scp) the flag -C -C Compression enable. Passes the -C flag to ...
0
votes
0answers
264 views

where to compress=lzo, btrfs with virtual disk images, with their vm's internally using btrfs

I've been using compress=lzo on my btrfs volumes, because I read this fast compression method actually increases drive performance. (Haven't tested myself.) I'm starting to use kvm/libvirt/...
3
votes
4answers
2k views

Is it possible to make a .tar.gz file directly from stdin? Or, I need to tar together already gzipped files

I'm going to tell you exactly what I need in order to clarify the cryptic question in the title. I'm currently making scheduled MySQL backups of all my databases with something like: mysqldump ... | ...
2
votes
1answer
288 views

How to reduce size of a FLAC file without damaging its quality?

I got collection of vinyls which I paid for them to be digitized in FLAC. However my Hard Drive is getting full (91% full as of today) and the smallest of these vinyls is 2.3 GB in size! I would ...
0
votes
1answer
112 views

Can I use lzma compression, without using “tar”, on OSX 10.10.3?

I kind of think of OSX as Unix by another name. But, if I need to move this to the Apple forum, please tell me! I can create tar files compressed using lzma as such: $ ls ./DbDumps alpha.dmp ...
6
votes
3answers
400 views

Does gz compression ratio improve over time?

I have some process that creates a stream of millions of highly similar lines. I'm piping this to gz. Does the compression ratio improve over time in such a setup? I.e. is the compression ratio better ...
0
votes
2answers
41 views

How to open compressed file in less mode [duplicate]

How do I open a compressed file in less mode. I am able to grep using zgrep and using zcatI can open, but the file is big I want to open in less mode.
8
votes
3answers
559 views

Can tarring files improve compression?

Can tarring a bunch of files together improve compression with the standard tools, e.g. gzip, bzip2, xz? I've long thought this to be the case but never tested it out. If we have 2 copies of the same ...
0
votes
1answer
767 views

Compress multiple files and keep their name?

I have hundreds of .csv files that I need to compress into one file each. It should look like this originalname.csv.gz
1
vote
0answers
29 views

Pipe gzip output to Bluetooth transfer?

How do I pipe gzip output to Bluetooth transfer? I am able to send send file over Bluetooth using this command: bluetooth-sendto --device=12:23:35:96:45:45 file.tar However, piping gzip didn't ...
13
votes
9answers
4k views

Memory problems when compressing and transferring a large number of small files (1TB in total)

I have 5 million files which take up about 1TB of storage space. I need to transfer these files to a third party. What's the best way to do this? I have tried reducing the size using .tar.gz, but ...
2
votes
1answer
57 views

lrzip compression does not shrink random data

I'm trying to shrink down some big files around 3.0 GB each and I decided to use lrzip after seeing a few benchmarks and such however is it something I'm doing wrong or something as whenever I run ...