Questions tagged [compression]

Shrinking (compressing) and restoring (decompressing) of data.

Filter by
Sorted by
Tagged with
6
votes
3answers
196 views

How do I demonstrate the type of kernel compression in practice?

I have to find out the type of compression of the linux kernel of my arch linux system, but I can't find a way to get it more than the theory: now bzip2 (bz), formerly gzip (z). In my computer I run ...
6
votes
1answer
57 views

Is the rsync block compression dictionary reset for each file?

Does rsync -z compress blocks of each file without regard to the previous file, or is the compression dictionary reset for each file so that it is handled independently? As an example, consider a ...
0
votes
1answer
29 views

Is TLS-level compression with Apache possible?

Apache2 can transfer compressed data by using the deflate filter. However, it does a HTTP-level compression: it sends back a compressed response, and it shows in the response headers to the clients to ...
0
votes
1answer
24 views

list directory sizes and compress them

I have a list of directories I want to compress on an ubuntu machine. Is there any tool to automatically calculate the size of the of my folders and copy the size to the name of the compressed file? ...
0
votes
1answer
46 views

lz4 compression is only using a single core?

I'm using cat largeFile.iso | lz4 -10 - and it's going quite slowly (30mb/s?). Using htop I see it using only a single core to 100%? Are there any additional parameters I need to specify? I can't ...
0
votes
1answer
67 views

Can gzip compress application/octet-stream

Particularly in web, is it possible to compress application/octet-steam, I am serving video files from AWS to a web app and data transfer is very expensive at high scale
2
votes
2answers
71 views

bzip2: Check file's decompressed size without actually decompressing it

I have a big bzip2 compressed file and I need to check it's decompressed size without actually decompressing it (similar to gzip -l file.gz or xz -l file.xz). How can this be done using bzip2?
1
vote
1answer
39 views

List contents of a tgz archive inside a gz archive

Bit of a weird one - I've got a honeypot running dionaea, which is a tool that consolidates any binaries that were uploaded to the device in single location (/data/dionaea/binaries). However, every ...
0
votes
1answer
33 views

7zip client: Compress dirs with method PPMd & compression level Ultra

So today, I am trying to compress multiple folders, of old photos which I want to backup. I've found this article, that says method PPMd, and compression level Ultra, are far better for image ...
1
vote
1answer
50 views

/boot running out of space when updating. What now?

I just ran into a serious problem I need to update offline CentOS boxes with fix partition sizes. When updating the process terminates with an error that /boot need 32mb more in order to update. It ...
-2
votes
1answer
233 views

Using ffmpeg to Compressed a Two Frame Video

I'm trying to use Ubuntu Linux 19.04 to convert an OpenShot video into a more efficient file-size. The video only contains two still images, and 20 minutes of audio, but for each file-type I save-as, ...
1
vote
1answer
44 views

How do I pass arguments to lzip through tar?

I want to run tar --lzip -cvf file1.txt file2.txt. But lzip's help page states that by default it does not run at its fullest compression power (the option for which is -9, while the default is -6). I ...
0
votes
2answers
148 views

Compress a large folder tree while keeping an easy way to navigate and extract particular files

I have a large and heavy folder tree with many files and folders. I want to compress it to reduce the storage size. This tree will not be frequently used any more, but quite probably I will need to ...
-1
votes
1answer
56 views

Difference between tar -t and tar -v options

I created a archive using command tar -cvf filename.tar files , in this the verbose option simply showed me a list of all the files that i have archived.When i used tar -tf filename.tar ,it again ...
1
vote
1answer
33 views

tar on remote machine and save the compress file under specific folder

we want to tar the folder - current on remote machine and save the compressed file under /tmp so we do that: ssh 18.23.2.12 "tar -zcvf current_bck.tar.gz /var/current -C /tmp" but on remote ...
-1
votes
1answer
46 views

Choosing best directories to compress and save disk space

What's the best way to estimate the compression level that can be achieved for a directory without storing the compressed files? I have a large amount of directories and sub-directories with files ...
0
votes
0answers
145 views

pxz (parallel xz) output meaning

I am running pxz with -v option, to "show progress" according to the man page. I am getting as output a sequence of seemingly random numbers, like: 0 1 7 3 2 5 8 4 9 6 6 9 5 0 1 2 3 8 7 4 ... It ...
0
votes
1answer
38 views

Is there a way to create a (compressed) squashfs that contains an uncompressed file

We have a large file that needs to be uncompressed inside a compressed squashfs. Is this possible without patching mksquashfs? Something like this would be nice: $ mksquashfs existing_squash.sqsh ...
4
votes
1answer
47 views

Time-adaptive compression tool

I'm asking about a scenario of copying a big file to a remote server. A simplest case is: tar c myfile | ssh myserver tar x If network connectivity is fast then all is fine. On a slower network I ...
0
votes
0answers
83 views

Unzip files in loop

I have the following code which extracts numbers from netcdf files and every loop puts them in another netcdf file. mapfile Latitude < <(tr -s ' ' '\n' < final_ADCP_Saved.matLatitude.txt) #...
2
votes
1answer
215 views

How do I reduze the size of a video to a target size?

I want to reduce the size of a video to be able to send it via email and such. I looked at this question where I got good advice on how to reduce it. Problem is that I need to manually calculate the ...
0
votes
1answer
138 views

How can I unload a filesystem module in a RHEL 7 server in order to harden it?

I'm trying to harden my RHEL Linux server 7 according to the CIS guidelines, so I want to unload the cramfs. I typed the following command: modprobe -n -v cramfs and I got the following answer: ...
0
votes
0answers
38 views

Why I cannot tar gzip all the contents of a specific folder

I have a folder containing several folders. I want to tar and gzip the parent folder with all the contents of the subfolders. However, when I use tar -cvzf folder.tar.gz Folder/ only a few ...
4
votes
1answer
379 views

Zip/Tar file compressed to larger size? [duplicate]

I have 4 files that I want to include them in a .zip file. drwxr-xr-x 7 skemelio themelis 4,0K Οκτ 18 00:21 CSharp drwxr-xr-x 11 skemelio themelis 4,0K Οκτ 18 00:17 gnome-games drwxr-xr-x 8 ...
0
votes
1answer
617 views

tar + compress folder to tar.gz and removed the folder

by the following way we tar the folder presto-server-0.210 to file tar -czvf presto-server-archive.tar.gz presto-server-0.210 so after we tar the folder , folder still exists is it possible to use ...
2
votes
0answers
79 views

Compress VirtualBox Linux Virtual Machine

I am trying to create a VirtualBox linux virtual machine on which I want to mainly run Wine to install a windows application. I managed to install Lubuntu 18.04.2, since it is the lightest Ubuntu ...
0
votes
0answers
196 views

How can I copy sparse files from a ext4 file system to an exfat file system and keep the apparent size?

My issue is that I am trying to copy files from my Ubuntu linux system which uses a ext4 file system to a Sandisk 1TB USB card, which uses an exFat system. In the ext4 system, the entire directory ...
1
vote
1answer
28 views

Using terminal to zip unknown amount of packages with dynamic names

I am trying to figure out a solution for unix based systems (maxOS, Linux) to zip unknown amount of packages, ideally without requiring users to install any additional third party software. I have ...
2
votes
1answer
226 views

Best compression for operating system image

I have an Operating system image of size 2.5G. I have a device with a limited size. Thus I was looking for the best possible solution for providing the compression. Below are the commands and ...
0
votes
0answers
46 views

Compare file compression methods by sending to /dev/null

I'd like to compare the time to completion for compression algorithms and parameters. I'm confident that the inputs will be set up properly. In order to save space and avoid some variability my ...
0
votes
0answers
73 views

How to make use of tar --verify directly?

This question is about compressing rather large data directories (around 22-23 GiB), specifically the integrity checks of such archives. I do these big backups about once a month, so the time it takes ...
2
votes
0answers
31 views

Resumable compression

I'm in the process of compressing a very large file (several TBs) with the following command, and it's been running for several days. tar -I pigz -cf /path/to/compressed.tgz /dataset/to/compress I'm ...
3
votes
0answers
64 views

How to unpack a tag.gz file without having 2x free space?

I have about 1 million small files, that take up about 20 GB. I need to upload them to a host that only has 23 GB free space. I would prefer compressing the files locally to increase the transmission ...
0
votes
0answers
7 views

Least resource heavy file compression method on hosting server

There are many compression formats like tar, zip, bzip etc. On a shared hosting (Linux), if I were to archive a huge folder for migration, which format would be best (least resource heavy)? Storage ...
0
votes
0answers
38 views

Is it possible to dynamically add to a tar.lrz file?

A bit of a basic problem, I'm slightly embarrased I'm running into it, however man lrztar didn't show me any commands for this. Basically, lets say I've got an archiving station which handles ...
1
vote
1answer
70 views

Stream text into a file while compressing it

I have an Arduino Nano connected to a Raspberry Pi which streams text data (the output of a GY-91 sensor) to a listener on the Raspberry Pi. P: 974.044 | T: 22.65 | AG: -0.17 -0.10 1.01 | GD: 1....
2
votes
1answer
46 views

Improving performance of Script Using find and exec [closed]

I have a script that iterates through a given directory and automatically compresses those that do not contain at least one file used in less than 30 days. Now, I am wondering if I could improve ...
2
votes
2answers
1k views

compressing and decompressing dd image - zstd instead of gzip

Earlier I was using fsarchiver to create compressed partition image. Due to some weird behavior I am choosing to replace it with dd. However, I like how fsarchiver compressed with zstd. So, I ...
0
votes
1answer
100 views

fsarchiver taking ages to backup and restore

Using Linux Mint 19.1 bootable USB Ext4 filesystem Processor 8600k Ram 16gb By default fsarchiver uses zstd compression Earlier when I used the following two commands, each command individually took ...
2
votes
1answer
230 views

how to set NOCOMPRESS flag on a file in btrfs?

according to https://btrfs.wiki.kernel.org/index.php/Compression#How_do_I_enable_compression.3F there exist a "NOCOMPRESS" flag, how do i set the NOCOMPRESS flag on a file?
3
votes
1answer
299 views

Easy way to change compression level of a zip file [duplicate]

On linux, is there an easy way to recompress a zip file (changing compression-level) with preserving as most meta-data as possible (timestamps etc.)?
1
vote
1answer
449 views

tar: compress text stdin and stdout [duplicate]

Using tar, is it possible to compress stdin text and have the compressed data appear in stdout? Example: $ echo "test" | tar c - <compressed blob here> I'm using macOS Mojave.
0
votes
3answers
810 views

Linux determine ZIP size without compression

I would like to make a ZIP file using zip command. Zip file would contain large number of directories containing large numbers of subdirectories, files etc. Is there any way to calculate final size ...
9
votes
1answer
1k views

Uncompressed file estimation wrong?

I had a large (~60G) compressed file (tar.gz). I used split to break it into 4 parts and then cat to join them back together. However, now, when I am trying to estimate the size of the uncompressed ...
0
votes
1answer
181 views

Is there an efficient command for listing the header of a particular file in tar.bz2 without uncompressing/extracting?

I was working with a data.tar.bz2, which contains lots of other big files of the scale of a few hundred Gigabytes. I just wanted to see the header (first line) of a particular file file-of-interest, ...
1
vote
2answers
2k views

Compression Utility Max Files Size Limit | Unix/Linux

In Unix/Linux is there any max files size limit that a compression utility ( gzip/compress) can compress. I remember years ago it was mentioned in the gzip page that it can compress files up to 4 gb. ...
0
votes
2answers
1k views

Does tar.gz do effective file compression of large files?

I wanted to compress a ~29GB file significantly, and used the tar command in the terminal like this : tar cvzf file.tar.gz /path/to/directory and it compressed it to ~26GB, so I looked up maximum ...
-1
votes
1answer
704 views

How to split a compressed file?

I have a 44GB 7z compressed file that I compressed with lzma2 and it took around 11 hours (original file is a text file of 285GB). 7z a -t7z Files.7z -m0=lzma2 -mx=9 -aoa So, what I want is to do is ...
-2
votes
1answer
221 views

compress specific folder(directory) from folder with tar [closed]

I am trying to use tar to compress the subdirectories /root/folder/folder{1..5} into a single compressed archive test1.tar.bz2. I'm using the following command: tar -cvjf test1.tar.bz2 root/folder/...
0
votes
1answer
29 views

Arguments for, and the applicability of, using multiple forms of compression on one or more files?

I was wondering whether there are any scenarios which require file(s) to be compressed multiple times with the same or a different algorithm. As far as i can tell the file size can be reduced further ...