Questions tagged [gzip]
A data compression program. Originally a GNU utility it has been cloned by most Unix variants.
287
questions
10
votes
2answers
1k views
Make grown extracted tar file small again
I packed and compressed a folder to a .tar.gz archive.
After unpacking it was nearly twice as big.
du -sh /path/to/old/folder = 263M
du -sh /path/to/extracted/folder = 420M
I searched a lot ...
5
votes
1answer
74 views
Is there a compression tool with an arbitrarily large dictionary?
I am looking for a compression tool with an arbitrarily large dictionary (and "block size"). Let me explain by way of examples.
First let us create 32MB random data and then concatenate it to itself ...
0
votes
1answer
28 views
Need AIX command to extract a particular text file from a compressed file
I am looking for an AIX command to extract a particular text file from a compressed file.
For example, my compressed file abc.gz contains millions of files with multiple sub directories. I need to ...
0
votes
1answer
104 views
How may I emulate the -R feature of “zip”, but in gzip and xz?
The program "zip" has a -R feature which allows one to zip all files with a certain name in a directory tree: zip -r v/s zip -R
For example:
zip -R bigfile "bigfile"
Will zip all of the following: ...
0
votes
0answers
36 views
Command “wc -l” displays wrong count of lines, possibly because each line starts with a comma?
I have a czv.gz file and when issuing "less" command it looks like this
COLUMN1,COLUMN2,COLUMN3
,value1,value2,value3
,value4,value5,value6
...
There are thousands of lines in a file but when I ...
2
votes
3answers
251 views
How to use zgrep to find out what line number or give some contextual info. surrounding a .gz file
In Debian you can use zgrep to grep through a gunzipped archive file. The reason for making a gunzip file is easy enough, files such as changelogs are vast which can be highly compressed. The issue is ...
0
votes
0answers
31 views
How does the tar flag --totals interact with the -z flag?
-z, --gzip
filter the archive through gzip
--totals[=SIGNAL]
print total bytes after processing the archive; with an argument - print total bytes when this SIGNAL is delivered;
Allowed ...
0
votes
1answer
107 views
Multithreaded xz, with gzip, pv, and pipes - is this the most efficient I can get?
I'm excited to learn that xz now supports multithreading:
xz --threads=0
But now I want to utilise this as much as possible. For example, to recompress gzips as xz:
gzip -d -k -c myfile.gz | pv | ...
0
votes
1answer
144 views
Can gzip compress application/octet-stream
Particularly in web, is it possible to compress application/octet-steam, I am serving video files from AWS to a web app and data transfer is very expensive at high scale
0
votes
1answer
78 views
“-bash: {}: No such file or directory” using find exec [duplicate]
I'm trying to run find . -name "binaries.tgz.*.gz" -exec gzip -d -k < {} \; (ultimately I'm trying to run find . -name "binaries.tgz.*.gz" -type f -exec gzip -d -k < {} \; | tar tzf - but I'm ...
1
vote
1answer
61 views
List contents of a tgz archive inside a gz archive
Bit of a weird one - I've got a honeypot running dionaea, which is a tool that consolidates any binaries that were uploaded to the device in single location (/data/dionaea/binaries).
However, every ...
0
votes
0answers
48 views
Bash Script for Backing up/Gzip files with correct name
Having trouble forming a backup shell script for nested vm's
There are multiple filenames (vm1, webserver, sshserver, etc.) with nothing in common other than .qcow2. I need to store just the ...
11
votes
5answers
3k views
Is gzip atomic?
Is gzip atomic?
What happens if I stop the gzip process while it's in the middle of gzipping a file?
If it's not atomic, and if I already pressed Ctrl+C on a gzip *.txt process, how do I safely ...
0
votes
0answers
25 views
Extracting a zipped archive (.gz) doesn't extract all of the files (GalliumOS)
Basically, I am installing PyCharm. And when i extract the tar.gz file, a JSON file and a txt file don't appear in the new extracted location. The files are also missing if I use ls to try to view ...
0
votes
0answers
45 views
In qemu using gzip return unexpected end of file gzip
When i download a file in /dev/sda1, and qemu -device file=/dev/sda1,format=raw to boot with it, gzip -d return unexpected end of file gzip.
However, in my host machine, tar -xf it works fine.
So ...
2
votes
1answer
65 views
Find exec - Why {} can't be used as the output file name? [duplicate]
I tried this command to compress all the css files in all subdirectories.
find . -iname "*.css*" -exec gzip -c '{}' > '{}'.gz \;
But it only creates a {}.gz file. I ended up using this:
find . -...
1
vote
2answers
225 views
Extract gzip from data file
I'm looking for some guidance on how to get this gzip out of a data file.
$ file cut6
cut6: data
$ binwalk cut6
DECIMAL HEXADECIMAL DESCRIPTION
.......................................
1 ...
0
votes
2answers
118 views
Alternative of too much slow gzip -d command
I'm using an production server for loading large data set to Hadoop to access from Hive table.
We are loading subscribers web browsing data of Telecom Sector. We've large number of .csv.gz file (File ...
0
votes
0answers
232 views
Error trying to create a tar of external device “cannot stat: No such file or directory”
I'm trying to make a tar.gz of arm board that can be connected as a UMS. To do that I open terminal with root privileges on a Ubuntu form directory of connected device and I write next command:
tar ...
3
votes
4answers
899 views
gzip compress a local folder and extract it to remote server
I am new to these commands. I am trying to gzip a local folder and unzip the same on the remote server. The thing is, gzipping and unzip must happen on the fly. I tried many and one of the closest I ...
0
votes
0answers
41 views
Tar files in groups by total size
I need to tar (and gzip) a long list of files but I need to do that in groups based on each group size.
For example assume I have a list of 100 files, 1Mb each and I want to create archives that are ...
0
votes
2answers
82 views
How to know the real file type of files inside a zip or tar.gz file?
I have multiple .tar.gz and .zip files and I wanted to know the file types of files inside these files without extracting them. How can I achieve this. I can list the files of .tar.gz using the ...
0
votes
1answer
154 views
Writing a Shell script to tail and gzip a log file
I'm trying to make a script that can tail log files from a remote server to my local directory. tail -F is what I'm using but after piping it with gzip, nothing happens although a local copy of the ...
1
vote
3answers
246 views
Create archive (tar.gz, …) from file given via stdin
I would like to automatically create a compressed archive from a file given via stdin (or create the archive, if it doesn't exist yet), while specifying the file name that it's going to have inside ...
3
votes
1answer
63 views
Find files then move them and rename them at the same time?
Currently I use three steps to gzip some static assets and then use s3cmd to an S3 bucket (technically it's a Digital Ocean Spaces bucket). Here's what I do:
$ find . -type f -name '*.css' | xargs -I{...
0
votes
3answers
993 views
How to find the string in the list of .gz files?
I have a list of folders in each folder I have around 600 .gz files which are automatically generated by the system every 15 min. I need to display the particular string which is repeated in that .gz ...
0
votes
0answers
38 views
Why I cannot tar gzip all the contents of a specific folder
I have a folder containing several folders. I want to tar and gzip the parent folder with all the contents of the subfolders. However, when I use
tar -cvzf folder.tar.gz Folder/
only a few ...
0
votes
2answers
33 views
Is it possible to have a zipfile in which only some files where encripted/password protected?
Would like to create a default project structure, but some business areas of the company do not need to have access to some assets (so I wanna block them).
Is there any way I could create a zipfile ...
-2
votes
1answer
301 views
Replace a binary file within another binary file
Is it possible replace in place a binary file located inside another binary file?
For example, compressed firmware.bin file contains:
DECIMAL HEXADECIMAL DESCRIPTION
------------------------...
0
votes
1answer
271 views
Logrotate is not compressing
I need to compress the files in var/log/remots/*/* with the following logrotate rule
/var/log/remots/*/* {
rotate 178
daily
maxage 178
compress
}
located in /etc/logrotate.d/remots.
...
2
votes
1answer
255 views
Best compression for operating system image
I have an Operating system image of size 2.5G.
I have a device with a limited size. Thus I was looking for the best possible solution for providing the compression.
Below are the commands and ...
2
votes
1answer
59 views
Trying to Create a TAR Archive, But Exclude .messages Files
I want to create a GZIP compressed TAR archive containing the contents of /var/log, but excluding /var/log/messages file.
I tried tar -cvf var/log.tar var/log/ -x *.messages*, but got the error ...
1
vote
1answer
144 views
Is it possible to automatically zip newly created files in a directory
Context
I am required to use a poorly designed java application that logs A LOT of information while it is running. Under standard usage, it will create 100s of MB of logs per hour.
I don't need ...
0
votes
1answer
876 views
How to extract information using awk on .gz files without storing the uncompressed data on disk
I need to use this command:
awk 'NR==FNR {a[$1]; next} $1 in a{print}' inputfile1.txt inputfile2.gen.gz >output.txt
Awk doesn't read the .gz file.
I have tried
zcat inputfile2.gz | awk 'NR==...
2
votes
2answers
500 views
How can I tar and gzip my web directory files and download (using rsync) it in one command from my local machine?
I'm rewriting my original question because I realized this is more efficient:
I'd like to tar and gzip my website files and then download it.
I know I can tar and gzip it with one command using tar ...
0
votes
2answers
1k views
Can't tar files: Refusing to read archive contents from terminal
I have host without dpkg so I need to install it.
I followed the instructions from here and here.
On both of these answers they wrote that I need to download the file:
wget http://security....
2
votes
1answer
183 views
Gzipped archives - Create using tar -czf and extract using only -xf?
Say I have a file.txt containing one line:
strawberry
Using
tar -czf archive.tar.gz file.txt
I can create a "gzipped archive" with the name archive.tar.gz, which is to my knowledge the equivalent ...
1
vote
4answers
66 views
Using an if statement to see if files are in a folder using a wildcard [duplicate]
I have to process multiple files that I dont have the entire name for. I am running an if statement using a wildcard. for example...
if [ "ESGSN_*.SMS.EDR" ]; then
echo "There are files to ...
1
vote
1answer
76 views
Logs gzipped and not gzipped
Recently I've changed logs configuration for letsencrypt, because there was no given one and I have files:
letsencrypt.log
letsencrypt.log.1
letsencrypt.log.10
letsencrypt.log.10.gz
letsencrypt.log....
-1
votes
1answer
138 views
Gzip verbosity goes to standard error
When I want to redirect any verbose output I usually do the following,
[arif@arif test]$ rsync -v /home/arif/storage . 1> stdout
[arif@arif test]$ cat stdout
storage
sent 177 bytes received 35 ...
1
vote
2answers
446 views
gzip multiple files and rename them
I have to gzip multiple files in a directory and rename them. I dont want to zip them into a single zip file. i.e.
gzip:
ABCDEPG01_20171120234905_59977
ABCDEPG02_20171120234905_59978
...
1
vote
0answers
19 views
How can I revert zip files which are zipped using “gzip -r” [duplicate]
I intended to zip a directory, so I used:
gzip -r myfolder
When I checked the output, it has been zipped all the files within the directory and its subdirectories. It wasn't what I explected
How ...
0
votes
3answers
917 views
Linux determine ZIP size without compression
I would like to make a ZIP file using zip command. Zip file would contain large number of directories containing large numbers of subdirectories, files etc.
Is there any way to calculate final size ...
9
votes
1answer
1k views
Uncompressed file estimation wrong?
I had a large (~60G) compressed file (tar.gz).
I used split to break it into 4 parts and then cat to join them back together.
However, now, when I am trying to estimate the size of the uncompressed ...
0
votes
1answer
169 views
NGINX gzip per virtualhost?
I know here are a lot of "similar" Q's, but anyway...
So – I have a nginx.conf with gzip on;. It's a new, clean Linux server, nothing yet done here.
Using curl – I can see gzip enabled:
$ curl -I -...
3
votes
2answers
478 views
Concatenate multiple zipped files, skipping header lines in all but the first file
I have a collection of gzipped files that I want to combine into a single file. They each have identical format. I want to keep the header information from only the first file and skip it in the ...
1
vote
2answers
2k views
Compression Utility Max Files Size Limit | Unix/Linux
In Unix/Linux is there any max files size limit that a compression utility ( gzip/compress) can compress. I remember years ago it was mentioned in the gzip page that it can compress files up to 4 gb. ...
4
votes
1answer
163 views
gunzip on Embedded machine
I'm using BusyBox 1.24.1 on my Embedded Linux machine (with kernel v3.10.31) and I get behavior I can't understand when unzipping files.
The gunzip command (often, but not always) fails,
with no ...
4
votes
3answers
2k views
Why does a gunzip to dd pipeline slow down at the end?
my command:
gunzip -c serial2udp.image.gz |
sudo dd of=/dev/mmcblk0 conv=fsync,notrunc status=progress bs=4M
my output:
15930949632 bytes (16 GB, 15 GiB) copied, 1049 s, 15.2 MB/s
0+331128 ...
0
votes
2answers
1k views
Does tar.gz do effective file compression of large files?
I wanted to compress a ~29GB file significantly, and used the tar command in the terminal like this :
tar cvzf file.tar.gz /path/to/directory
and it compressed it to ~26GB, so I looked up maximum ...
