Questions tagged [compression]

Shrinking (compressing) and restoring (decompressing) of data.

Filter by
Sorted by
Tagged with
11
votes
2answers
17k views

What's the difference between gunzip and unzip?

What is the difference between gunzip and unzip? Tried to search but couldn't find anything relevant with this.
-1
votes
1answer
36 views

Choosing best directories to compress and save disk space

What's the best way to estimate the compression level that can be achieved for a directory without storing the compressed files? I have a large amount of directories and sub-directories with files ...
11
votes
3answers
3k views

Why use cpio for initramfs?

I am making my own initramfs following the Gentoo wiki. Instead of the familiar tar and gzip, the page is telling me to use cpio and gzip. Wikipedia says that cpio is used by the 2.6 kernel's ...
0
votes
0answers
27 views

pxz (parallel xz) output meaning

I am running pxz with -v option, to "show progress" according to the man page. I am getting as output a sequence of seemingly random numbers, like: 0 1 7 3 2 5 8 4 9 6 6 9 5 0 1 2 3 8 7 4 ... It ...
0
votes
1answer
22 views

Is there a way to create a (compressed) squashfs that contains an uncompressed file

We have a large file that needs to be uncompressed inside a compressed squashfs. Is this possible without patching mksquashfs? Something like this would be nice: $ mksquashfs existing_squash.sqsh ...
109
votes
12answers
96k views

How to XZ a directory with TAR using maximum compression?

So I need to compress a directory with max compression. How can I do it with xz? I mean I will need tar too because I can't compress a directory with only xz. Is there a oneliner to produce e.g. foo....
4
votes
1answer
38 views

Time-adaptive compression tool

I'm asking about a scenario of copying a big file to a remote server. A simplest case is: tar c myfile | ssh myserver tar x If network connectivity is fast then all is fine. On a slower network I ...
0
votes
0answers
30 views

Unzip files in loop

I have the following code which extracts numbers from netcdf files and every loop puts them in another netcdf file. mapfile Latitude < <(tr -s ' ' '\n' < final_ADCP_Saved.matLatitude.txt) #...
1
vote
1answer
773 views

compressed swapfile (without zram or zswap)

Is it possible to have a compressed swap file without using zram or zswap? Actually, I've successfully mounted a swap file over sftp with sshfs with these commands and I'd like to use compression ...
2
votes
1answer
53 views

How do I reduze the size of a video to a target size?

I want to reduce the size of a video to be able to send it via email and such. I looked at this question where I got good advice on how to reduce it. Problem is that I need to manually calculate the ...
185
votes
8answers
266k views

How can I reduce a video's size with ffmpeg?

How can I use ffmpeg to reduce the size of a video by lowering the quality (as minimally as possible, naturally, but I need it to run on a mobile device that doesn't have much available space)? I ...
1
vote
1answer
2k views

How to convert tar.bz2 to .bz2?

I have a file with compression tar.bz2 , I would like to convert it to .bz2 . Is there a command that does this in one shot? I tried the below but no joy ! tar -xvjf US.tar.bz2 US bzip2: ...
0
votes
1answer
74 views

How can I unload a filesystem module in a RHEL 7 server in order to harden it?

I'm trying to harden my RHEL Linux server 7 according to the CIS guidelines, so I want to unload the cramfs. I typed the following command: modprobe -n -v cramfs and I got the following answer: ...
10
votes
3answers
8k views

Private VPN compression for faster Internet

Background: I'm a student, and currently the Wi-Fi connection at my school is very slow (dead slow). I have an 8Gb 4G data-plan, but even with that, I'm running short at the end of the month. I ...
1
vote
2answers
2k views

What invocation will use LZMA to compress tar output on my Mac (OSX 10.12.x)?

I've tried several invocations of the OEM tar to create LZMA-compressed tarballs. More specifically, I tried: tar -c -f --lzma Windows\ 7.vmwarevm.tar.lzma Windows\ 7.vmwarevm My efforts created an ...
19
votes
2answers
59k views

Having trouble uncompressing a few files [duplicate]

So I have to uncompress 4 different files. When I do file file1 file file2 file file3 file file4 for example, different messages are displayed. For example one starts like "gzip compressed data, …, ...
0
votes
0answers
35 views

Why I cannot tar gzip all the contents of a specific folder

I have a folder containing several folders. I want to tar and gzip the parent folder with all the contents of the subfolders. However, when I use tar -cvzf folder.tar.gz Folder/ only a few ...
0
votes
1answer
266 views

Can I use lzma compression, without using “tar”, on OSX 10.10.3?

I can create tar files compressed using lzma as such: $ ls ./DbDumps alpha.dmp beta.dmp delta.dmp $ XZ_OPT=-9 tar cJf dbDumps.tar.xz DbDumps But, what I really want to do ( but cannot ) ...
0
votes
1answer
88 views

Is there an efficient command for listing the header of a particular file in tar.bz2 without uncompressing/extracting?

I was working with a data.tar.bz2, which contains lots of other big files of the scale of a few hundred Gigabytes. I just wanted to see the header (first line) of a particular file file-of-interest, ...
51
votes
5answers
70k views

Which file compression software for linux offers the highest size reduction? [closed]

I do a ton of file compression. Most of the stuff I am compressing is just code, so I need to use loss less compression. I was wondering if there was anything that offers a better size reduction ...
4
votes
1answer
363 views

Zip/Tar file compressed to larger size? [duplicate]

I have 4 files that I want to include them in a .zip file. drwxr-xr-x 7 skemelio themelis 4,0K Οκτ 18 00:21 CSharp drwxr-xr-x 11 skemelio themelis 4,0K Οκτ 18 00:17 gnome-games drwxr-xr-x 8 ...
0
votes
1answer
133 views

tar + compress folder to tar.gz and removed the folder

by the following way we tar the folder presto-server-0.210 to file tar -czvf presto-server-archive.tar.gz presto-server-0.210 so after we tar the folder , folder still exists is it possible to use ...
2
votes
0answers
53 views

Compress VirtualBox Linux Virtual Machine

I am trying to create a VirtualBox linux virtual machine on which I want to mainly run Wine to install a windows application. I managed to install Lubuntu 18.04.2, since it is the lightest Ubuntu ...
0
votes
0answers
84 views

How can I copy sparse files from a ext4 file system to an exfat file system and keep the apparent size?

My issue is that I am trying to copy files from my Ubuntu linux system which uses a ext4 file system to a Sandisk 1TB USB card, which uses an exFat system. In the ext4 system, the entire directory ...
1
vote
1answer
25 views

Using terminal to zip unknown amount of packages with dynamic names

I am trying to figure out a solution for unix based systems (maxOS, Linux) to zip unknown amount of packages, ideally without requiring users to install any additional third party software. I have ...
2
votes
1answer
124 views

Best compression for operating system image

I have an Operating system image of size 2.5G. I have a device with a limited size. Thus I was looking for the best possible solution for providing the compression. Below are the commands and ...
0
votes
0answers
36 views

Compare file compression methods by sending to /dev/null

I'd like to compare the time to completion for compression algorithms and parameters. I'm confident that the inputs will be set up properly. In order to save space and avoid some variability my ...
0
votes
0answers
47 views

How to make use of tar --verify directly?

This question is about compressing rather large data directories (around 22-23 GiB), specifically the integrity checks of such archives. I do these big backups about once a month, so the time it takes ...
2
votes
0answers
27 views

Resumable compression

I'm in the process of compressing a very large file (several TBs) with the following command, and it's been running for several days. tar -I pigz -cf /path/to/compressed.tgz /dataset/to/compress I'm ...
15
votes
6answers
67k views

How to read a .war file?

I am trying to view the content of a .war file. I first did set its permissions with chmod 777 then when I try to accces it using: cd /usr/local/standalone/deployments/Sample.war/WEB-INF/classes/ It ...
3
votes
0answers
47 views

How to unpack a tag.gz file without having 2x free space?

I have about 1 million small files, that take up about 20 GB. I need to upload them to a host that only has 23 GB free space. I would prefer compressing the files locally to increase the transmission ...
5
votes
4answers
5k views

Compress and remove original directory with limited storage

On my 3 TB external hard drive, I have a 2.7 TB directory containing relatively small files. I would like to compress this 2.7 TB directory and remove it to keep only the compressed version. The issue ...
19
votes
4answers
8k views

How to decompress jsonlz4 files (Firefox bookmark backups) using the command line?

There seems to be various JavaScript+browser specific ways of decompressing this, but isn't there some way to transform jsonlz4 files to something unlz4 will read?
0
votes
0answers
7 views

Least resource heavy file compression method on hosting server

There are many compression formats like tar, zip, bzip etc. On a shared hosting (Linux), if I were to archive a huge folder for migration, which format would be best (least resource heavy)? Storage ...
0
votes
0answers
29 views

Is it possible to dynamically add to a tar.lrz file?

A bit of a basic problem, I'm slightly embarrased I'm running into it, however man lrztar didn't show me any commands for this. Basically, lets say I've got an archiving station which handles ...
1
vote
1answer
151 views

Can someone explain what hunzip is for?

I understand what hunspell is for, but I was browsing the source code and the man, and I'm not sure: Does hunzip compress text files or just dictionaries ?
2
votes
1answer
40 views

Improving performance of Script Using find and exec [closed]

I have a script that iterates through a given directory and automatically compresses those that do not contain at least one file used in less than 30 days. Now, I am wondering if I could improve ...
1
vote
1answer
50 views

Stream text into a file while compressing it

I have an Arduino Nano connected to a Raspberry Pi which streams text data (the output of a GY-91 sensor) to a listener on the Raspberry Pi. P: 974.044 | T: 22.65 | AG: -0.17 -0.10 1.01 | GD: 1....
2
votes
3answers
2k views

Estimate compressibility of file

Is there a quick and dirty way of estimating gzip-compressibility of a file without having to fully compress it with gzip? I could, in bash, do bc <<<"scale=2;$(gzip -c file | wc -c)/$(wc -...
2
votes
2answers
367 views

compressing and decompressing dd image - zstd instead of gzip

Earlier I was using fsarchiver to create compressed partition image. Due to some weird behavior I am choosing to replace it with dd. However, I like how fsarchiver compressed with zstd. So, I ...
0
votes
1answer
48 views

fsarchiver taking ages to backup and restore

Using Linux Mint 19.1 bootable USB Ext4 filesystem Processor 8600k Ram 16gb By default fsarchiver uses zstd compression Earlier when I used the following two commands, each command individually took ...
1
vote
1answer
5k views

Unzipping a .gz file to get a text file but getting a binary file? [closed]

I have a .gz file called document.txt.gz file. It is supposed to be a text file which got compressed to a gzip file. When I uncompress, I am getting a corrupt file. I ran the following commands: $ ...
13
votes
8answers
484 views

Is there any program to provide a consistent interface across multiple archive types?

At the moment, if I download a compressed file, it could be any of a .tar.gz archive, a tar.bz2 arhive, a .zip archive or a .gz archive. And each time I do so, I have to remember what the command line ...
1
vote
2answers
3k views

how to reduce the size of a pdf file from command line

I know this question has been answered in other threads, but none of the solutions worked for me. I have a pdf file which contains 4 plots with millions of colourful dots (Manhattan plots). The pdf ...
4
votes
1answer
1k views

Is there any way to verify the integrity of a xz archive without decompressing the entire archive?

Can I check the integrity of the archive without decompressing it as the archive is 64GB. It is a backup of my old laptop and before i format my old laptop I wanted to verify the backup but it will ...
6
votes
1answer
725 views

How to specify additional parameters for GNU sort's --compress-program option?

sort --compress-program=/bin/gzip works fine, but how can I pass options down to the compression programm (e.g. --best or --fast)? sort --compress-program=/bin/gzip\ --best failes with the error ...
2
votes
1answer
160 views

how to set NOCOMPRESS flag on a file in btrfs?

according to https://btrfs.wiki.kernel.org/index.php/Compression#How_do_I_enable_compression.3F there exist a "NOCOMPRESS" flag, how do i set the NOCOMPRESS flag on a file?
0
votes
3answers
519 views

Linux determine ZIP size without compression

I would like to make a ZIP file using zip command. Zip file would contain large number of directories containing large numbers of subdirectories, files etc. Is there any way to calculate final size ...
2
votes
1answer
83 views

Bash script for uncompressing multiple files at a once

I have a set of files: lium3@l128-37:/DATA/fr94/01 $ ls -l total 7784 -rw-r--r-- 1 lium3 mcm1 395575 May 2 1996 fr940104.0z -rw-r--r-- 1 lium3 mcm1 113815 May 2 1996 fr940104.1z -rw-r--r-- 1 ...
7
votes
5answers
8k views

Is it possible to make a .tar.gz file directly from stdin? Or, I need to tar together already gzipped files

I'm going to tell you exactly what I need in order to clarify the cryptic question in the title. I'm currently making scheduled MySQL backups of all my databases with something like: mysqldump ... | ...