Questions tagged [gzip]
A data compression program. Originally a GNU utility it has been cloned by most Unix variants.
0
votes
1answer
28 views
NGINX gzip per virtualhost?
I know here is a lot of "similar" Q's, but anyway...
So - I have a nginx.conf with gzip on;. It's new, clean Linux server, nothing yet done here.
Using curl - I can see gzip enabled:
$ curl -I -H "...
3
votes
2answers
29 views
Concatenate multiple zipped files, skipping header lines in all but the first file
I have a collection of gzipped files that I want to combine into a single file. They each have identical format. I want to keep the header information from only the first file and skip it in the ...
1
vote
2answers
25 views
Compression Utility Max Files Size Limit | Unix/Linux
In Unix/Linux is there any max files size limit that a compression utility ( gzip/compress) can compress. I remember years ago it was mentioned in the gzip page that it can compress files up to 4 gb. ...
4
votes
1answer
61 views
gunzip on Embedded machine
I'm using BusyBox 1.24.1 on my Embedded Linux machine (with kernel v3.10.31) and I get behavior I can't understand when unzipping files.
The gunzip command (often, but not always) fails,
with no ...
4
votes
3answers
222 views
Why does a gunzip to dd pipeline slow down at the end?
my command:
gunzip -c serial2udp.image.gz |
sudo dd of=/dev/mmcblk0 conv=fsync,notrunc status=progress bs=4M
my output:
15930949632 bytes (16 GB, 15 GiB) copied, 1049 s, 15.2 MB/s
0+331128 ...
0
votes
2answers
31 views
Does tar.gz do effective file compression of large files?
I wanted to compress a ~29GB file significantly, and used the tar command in the terminal like this :
tar cvzf file.tar.gz /path/to/directory
and it compressed it to ~26GB, so I looked up maximum ...
1
vote
1answer
23 views
perl symlink on gz file
I have a file file1.txt.gz and I am trying to create a symlink to a regular file as below using perl
symlink ("file1.txt.gz", orig_file1);
But this doesnt seem to work unless I add .gz suffix to the ...
0
votes
1answer
36 views
How could --compression be missing from my wget?
I have experience with the Windows version:
GNU Wget 1.19.4 built on mingw32.
But now on Ubuntu, I have:
GNU Wget 1.19.4, a non-interactive network retriever.
There is a relatively new option ...
1
vote
0answers
35 views
tar: Potential bug (?) when incremental archive selected - Deleting files in the target
I have quite interesting observation which perhaps is a manifest of a bug.
My use case is to:
take a backup of a folder, having a set of files from this folder added to --exclude-from file. I'm ...
3
votes
1answer
114 views
How to maintain sort order with xargs and gunzip
I'm attempting to extract the contents of some files by alphabetical (which in this case also means date and iteration) order and when I test the process first with ls:
$ find /opt/minecraft/wonders/...
0
votes
3answers
295 views
How to tar-gzip a directory and put results inside itself?
I would like to backup my entire home directory on Linux. Suppose this directory is the only one I have write permissions to.
How would I tar-gzip this directory and put result inside itself?
0
votes
1answer
269 views
How to turn a tar file to a tgz file?
I usually create a tgz file for my_files with the command tar -czvf my_files.tgz my_files, and extract them with tar -zxvf my_files.tgz. Now I have a tar file created with the command tar -cvf ...
3
votes
2answers
301 views
Archive big data into multiple parts
I'm working on big data and I need to archive a directory that is larger than 64 terabytes. I cannot create such large file (archive) on my file system. Unluckily, all proposed solutions for creating ...
0
votes
4answers
338 views
read first line from .gz compressed file without decompressing entire file
I have a huge log file compressed in .gz format and I want to just read the first line of it without uncompressing it to just check the date of the oldest log in the file.
The logs are of the form:
...
-2
votes
2answers
100 views
Trying to gzip qcow2 files to a /backup directory
I have some files in a directory; some of them are .qcow2 extensions. I want to backup only the files having .qcow2 extension.
Example: I have 5 files with .qc0w2 exxtension, they are: vm1.qcow2, ...
5
votes
2answers
456 views
How to correctly extract a .tar.gz with md5.sum on solaris 10
I am trying to extract a gcc-4.9.0.tar.gz downloaded from one of the gcc mirror site.
In order to check the md5 signature on it before I gunzip it I did
digest -a md5 -v gcc-4.9.0.tar.gz
which ...
0
votes
1answer
46 views
Grepping account names (read from a file) from a set of compressed files
This script:
while read ACCT ; do
echo $ACCT
zgrep -h ^$ACCT datafile.0[2-3]2015.gz >>/tmp/$ACCT.txt
done < /account_list.txt
I get no redirected output file
When I remove '>>/tmp/$...
0
votes
1answer
431 views
Unzipping a .gz file to get a text file but getting a binary file?
I have a .gz file called document.txt.gz file. It is supposed to be a text file which got compressed to a gzip file. When I uncompress, I am getting a corrupt file.
I ran the following commands:
$ ...
0
votes
0answers
42 views
file ending .gz0 - which program does this?
On my Solaris 10 boxes, I have compressed log files with the ending .gz0. gunzip cannot unzip them, however if I change the ending to .gz, gunzip will unzip them, and these files appear to be empty. ...
2
votes
1answer
45 views
Finding the Count for a String on Multiple Zipped Files in Multiple Directories (non aggregated)
I'm trying to figure out the number of lines that contain the strings "event" and "type". The files I want to search through are in multiple folders and are zipped. I'm able to get an aggregated count ...
-2
votes
3answers
185 views
How to tar zip directories which are 7 days older
I have a folder that contains a lot of xml files stored in it, the folder name has a format 'yyyymmdd' eg., 20180515. I wanted a script that can zip the folder of this kind, when the folder grows 7 ...
0
votes
1answer
217 views
How to merge two gz files into new outputfile with specific name
I have 96 *fastqc.gz raw read files from 24 samples. Each sample was sequenced on two lanees for each pair.
I would like to merge reads for each pair from both lanes into one output file with same ...
1
vote
2answers
156 views
compress file using /bin/gzip with particular patterns
I want to compress files individually with the particular pattern.
For example, I have directory as below :
/log/log1
-/1/log2/file.1
-/1/log2/file123
-/1/log2/file2.1
/log/log2
-/2/...
1
vote
1answer
112 views
Restore files from tar using rsync
I'm writing the software upgrade functionality for a piece of embedded equipment. Currently the root file system is upgraded by taking a new rootfs.tar.gz file and unpacking it to the root file ...
1
vote
1answer
36 views
How to remove parentheses from a compressed (preferably gzipped) text file
This question discussed how to remove parentheses from simple uncompressed text files.
The accepted answer suggested the following:
cat in_file | tr -d '()' > out_file
To my observation, however,...
1
vote
2answers
266 views
How to turn gzipped file into one tar.gz archive in-place?
I have a lot (~5 millions) of small files, each one compressed individually with gzip. I would like to turn them into one big tar.gz archive to save some space (metadata), but there are limitations:
...
0
votes
2answers
307 views
tar command create directory without 777 permission
I run into an error while extracting a tar file, the created directory are created with the chmod 666 instead of 777. Therefore it will not extract inside this folder.
Here is my command:
$umask 000 ...
1
vote
1answer
180 views
Reading partially downloaded gzip with an offset
Let's say that there is one huge db.sql.gz of size 100GB available https://example.com/db/backups/db.sql.gz and the server supports range requests.
So instead of downloading the entire file, I ...
1
vote
1answer
137 views
why does gzip not create files of equal size?
I have a file I need to split into smaller sizes (<24M when zipped)
Heres the file:
498775505 Mar 8 00:08 test.file
I split it:
split -b 125000k test.file test.file.
Now I have even sized ...
0
votes
1answer
225 views
gzip all files to root folder in .tar?
The command below will create a .tar.gz with another archive in it which contains the actual files.
Is it possible to prevent the additional archive from being created and put all files directly ...
-2
votes
1answer
273 views
bzip2 a tar folder
I just create a tar directory of 600G test.tar inside dir1 .
my question the only space available I have now 300G in dir1
I m looking for some ways to generate the compressed test.tar.bzip on the
...
0
votes
1answer
593 views
how to check/validate the tar.gz files before un-tar
when we uncompressed tar.gz file with tar xfz
tar xfz redhatPkgInstallation.tar.gz
we get the following errors
gzip: stdin: decompression OK, trailing garbage ignored
tar: Child returned status 2
...
1
vote
0answers
84 views
Can gzip change original file size?
/dev/sdf is a 32G SD. As far as I can tell, I had performed the following, but upon attempting to perform the final step and write to a SD, my file image appears to be one byte to large for the SD. ...
1
vote
1answer
114 views
Uncompressing *.Z files in a folder returns error
To estract a single .Z file from a given folder I use uncompress file.Z in a terminal and it works flawlessy. If, in the same folder, I want to extract all the .Z files I use uncompress "*.Z" or ...
4
votes
4answers
3k views
Sed for gzip files
I have a bunch of gz files and unzipped version of them contains the patterns A and B=1 (these are certainly on different lines where A appears first).
I want to write a command that gives me the ...
0
votes
1answer
58 views
Linux Mint kernel damage
First off: I am running Linux Mint 18.2 on an External HDD with Gnome 3 as my desktop.
Recently, I was trying to figure out how to compress my OS so I could move it to a different system. So, I ran:
...
0
votes
2answers
16 views
Why gunzipping a 644 file belonging to root is possible by user and gunzipped file belongs to user
A tar.gz file belonging to root, can be gunzipped by a user, since it is readable by group and public. However after gunzipping, the tar file owner is user, not root anymore. Is it a feature of the ...
2
votes
1answer
80 views
Read concatenated output of tgz archive (with plain text files), with filenames, without unpacking in bash?
I have a bunch of text files/logs, which I'd like to pack into a gzipped tar archive, say:
cd /tmp
echo "My Content One" > aaa.log
echo "My Content OneA" >> aaa.log
echo "My Content Two" >...
0
votes
1answer
78 views
Compression size is bigger than actual directory size
I am on trying to archive a directory (a Time Machine backup) using:
tar -czf /Volumes/My\ Drive/archive.tar.gz my_directory
The archive process is going on at the moment. I am monitoring the ...
0
votes
2answers
227 views
Merge possibly truncated gzipped log files
I have multiple log files from each day that I need to merge together. Each comes from a different server. The job that puts them there sometimes gets interrupted and files get truncated. In that ...
6
votes
1answer
3k views
Unzip gz archives with zip extension
I have a number of gzip archives; however they have a zip extension not gz:
***.zip
When I try to unzip them with unzip, I get not a zip archive error, and with gunzip I get unknown suffix: zip
...
0
votes
1answer
404 views
cat and split a file and upload to hdfs?
I am trying to achieve this scenario where I can split a gz file which is on local linux and upload to hdfs as part unzipped or zipped files without writing the split output to disk.
I am having ...
3
votes
1answer
114 views
*.tar.gz backup growing
I am doing a daily *.tar.gz-backup of graphite whisper databases (/var/lib/graphite) by using this script. The backup destination is a windows share mounted before doing the backup with CIFS.
The ...
0
votes
1answer
100 views
Disk performance between servers in NFS directory
I am having a two linux server set up.
ServerA has /apps/data.
ServerB has mounted on /data ServerAs path /apps/data with NFS.
How load of various operations is handled? by meaning:
Q:
By initiating ...
3
votes
2answers
182 views
How to make a dummy application reads gzip files (on-the-fly) without code changes
I'm looking for a general tool/library that overrides the OS fopen function to read gz files seamlessly on-the-fly without modification to an application.
Details: I have a legacy application that ...
2
votes
2answers
64 views
gzip *.txt vs gzip test.txt & gzip test2.txt &
I am looking to speedup gzip process. (server is AIX 7.1)
More specificly, the current implementation is with gzip *.txt and it takes up to 1h to complete.
(file extractions are quite big and we got ...
0
votes
1answer
71 views
Help with tar gz
I am running into the below issue.
I have two folders:
AD-VM and Jump-VM
In each of these folders, the contents are:
-rw-------. 1 root root 20G Jun 28 18:27 AD-VM-flat.vmdk
-rw-------. 1 root ...
21
votes
5answers
2k views
On-the-fly stream compression that doesn't spill over into hardware resources?
I have 200 GB free disk space, 16 GB of RAM (of which ~1 GB is occupied by the desktop and kernel) and 6 GB of swap.
I have a 240 GB external SSD, with 70 GB used1 and the rest free, which I need to ...
1
vote
1answer
19 views
Unable to read or unzip certain archives on one specific server
I am experiencing problems trying to zcat the contents of a particular gzip archive containing SQL text. The problem seems localised to one particular file on one server.
I have copied around 10 ...
4
votes
1answer
778 views
How to decompress only a portion of a file?
I have a large, repetitive text file. It compresses very well - it's about 3MB compressed. However, if decompressed, it takes 1.7GB. Since it's repetitive, I only need a fraction of the output to ...
