Questions tagged [gzip]
A data compression program. Originally a GNU utility it has been cloned by most Unix variants.
382
questions
0
votes
1
answer
27
views
"Permission denied" error when using sudo for writing image to /dev/sdb on Debian [duplicate]
I am confused as to why I am getting the following error:
$ sudo zcat firmware.A20-OLinuXino-Lime2.img.gz partition.img.gz > /dev/sdb
bash: /dev/sdb: Permission denied
Shouldn't sudo be able to ...
- 341k
1
vote
1
answer
6k
views
zcat / gzip error while piping out
I am using zcat to output the contents of a rather large .gz compressed text file. I am piping the output to grep and searching for a specific string. Below is the command I am running.
dylan@...
CommunityBot
- 1
21
votes
3
answers
38k
views
Use gzip to compress the files in a directory except for already existing .gz files
I have a directory of logs that I would like to set up a job to compress using gzip. The issue is I don't want to recompress the logs I've already compressed.
I tried using ls | grep -v gz | gzip, ...
- 69
5
votes
6
answers
4k
views
Estimate compressibility of file
Is there a quick and dirty way of estimating gzip-compressibility of a file without having to fully compress it with gzip?
I could, in bash, do
bc <<<"scale=2;$(gzip -c file | wc -c)/$(wc -...
- 111
0
votes
0
answers
26
views
vmlinuz to vmlinux ERROR
$ file vmlinuz
vmlinuz: Linux kernel x86 boot executable bzImage, version 4.14.244 (root@d0ea4514eda5) #1 SMP Thu Aug 31 01:23:02 PDT 2023, RO-rootFS, swap_dev 0x3, Normal VGA
I try to use ...
- 1
0
votes
0
answers
29
views
Recoverably recompress gzip files into zstd, preserving original checksums?
I need to archive a lot of gzip-compressed data. The problem is that compared to zstd, gzip is wasteful, both in terms of ratio and CPU time required to decompress the data. Because of that, I want to ...
- 1,361
10
votes
3
answers
8k
views
How to get trailing data of gzip archive?
I have a gzip archive with trailing data. If I unpack it using gzip -d it tells me: "decompression OK, trailing garbage ignored" (same goes for gzip -t which can be used as a method of detecting that ...
- 111
245
votes
4
answers
97k
views
Why are tar archive formats switching to xz compression to replace bzip2 and what about gzip?
More and more tar archives use the xz format based on LZMA2 for compression instead of the traditional bzip2(bz2) compression. In fact kernel.org made a late "Good-bye bzip2" announcement, 27th Dec. ...
- 416
0
votes
2
answers
140
views
extending recursive find to .tar, tar.gz, tar.bz2, and tar.xz on Debian 12
Continuing find a file within a tar.gz archive, how is it possible to automatically recursively search for files with given names, including the files inside the archives and compressed archives?
For ...
- 6,084
0
votes
2
answers
76
views
tar: ./.tar.gz: file changed as we read it | "Flagged" files are unrelated to the file tar is supposed to operate on
I am trying to use tar to recursively compress all files with the .lammpstrj extension within the directory tree starting at the directory whose path is stored in the variable home. home contains the ...
- 34.4k
7
votes
4
answers
11k
views
Decompress gzip file in place
I have a huge file (420 GB) compressed with gzip and I want to decompress it, but my HDD doesn't have space for storing the whole compressed file and its contents.
Would there be a way of ...
- 56
0
votes
1
answer
8k
views
"gzip: stdin: invalid compressed data--format violated" error when extracting a tgz archive
I have to open a tar file. I used the following command line on my Unix shell:
tar xfvz raw-GWA-data.tgz
I found the following error message:
gzip: stdin: invalid compressed data--format violated
tar:...
- 558k
0
votes
1
answer
67
views
How to verify size of pigz (parallel gzip) archive contents? [duplicate]
I created some pigz (parallel gzip) - home page - compressed archives of my SSD disk drives. (compiled version 2.8)
I called one of them 4TB-SATA-disk--Windows10--2024-Jan-21.img.gz which says the ...
- 29.3k
36
votes
1
answer
69k
views
Will tar -cvzf packed.tar.gz mydir take hidden files into account?
I need to create a tarball of a given directory. However, I need to make sure hidden files are included too (such as those beginning with .).
Will the following command automatically take the hidden ...
- 1
18
votes
7
answers
130k
views
tar: Unexpected EOF in archive
I was attempting to untar a .tar.gz file, but came across this error:
gzip: stdin: unexpected end of file
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: ...
- 1
2
votes
4
answers
223
views
awk appends column in .gz file as seen with cat -A, but changes column name in regular output
Using awk, to this table I want to add a column where first row is "INFO" and rest of the rows are all "1".
$ gunzip -c foo.gz | head
SNPID CHR BP Allele1 Allele2 Freq1 Effect ...
- 558k
205
votes
14
answers
412k
views
How to uncompress zlib data in UNIX?
I have created zlib-compressed data in Python, like this:
import zlib
s = '...'
z = zlib.compress(s)
with open('/tmp/data', 'w') as f:
f.write(z)
(or one-liner in shell: echo -n '...' | python2 -...
- 103
0
votes
0
answers
816
views
telegram-desktop: qt.svg: Error while inflating gzip file: SVG format check failed
I use telegram-desktop on arch linux in sway. It sometimes crashes, so I started it from terminal. This way I find out telegram spams this message:
qt.svg: Error while inflating gzip file: SVG format ...
- 1,491
0
votes
1
answer
2k
views
How to merge two gz files into new outputfile with specific name
I have 96 *fastqc.gz raw read files from 24 samples. Each sample was sequenced on two lanees for each pair.
I would like to merge reads for each pair from both lanes into one output file with same ...
CommunityBot
- 1
2
votes
2
answers
2k
views
How to modify a gzipped file with sed and then zip again the file?
I have a .vcf.gz file, with the following aspect:
#CHROM POS ID REF ALT
chr1 10894 chr1:10894:G:A G A
chr1 10915 chr1:10915:G:A G A
...
- 34.4k
0
votes
2
answers
673
views
How can I fix a .tar.gz being downloaded as html?
I'm trying to download this file https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-3.3.6/hadoop-3.3.6.tar.gz with wget. When I try to unpack it with tar I get the following error:
gzip: stdin:...
- 34.4k
34
votes
6
answers
23k
views
Fastest way of working out uncompressed size of large GZIPPED file
Once a file is gzipped, is there a way of quickly querying it to say what the uncompressed file size is (without decompressing it), especially in cases where the uncompressed file is > 4GB in size.
...
- 565
21
votes
7
answers
30k
views
Fastest and most efficient way to get number of records (lines) in a gzip-compressed file
I am trying to do a record count on a 7.6 GB gzip file. I found few approaches using the zcat command.
$ zcat T.csv.gz | wc -l
423668947
This works but it takes too much time (more than 10 minutes ...
- 565
5
votes
1
answer
5k
views
Difference between ar, tar, gzip, zip and when should I decide to choose which one?
I had a use case where I needed to pack a bunch of files into one. And all above commands does the same. I know gzip compresses my files but lets say space is not at all an issue for me then in that ...
- 34.4k
6
votes
1
answer
6k
views
How to uncompress a gzipped partition image and dd it directly to the destination partition without writing to current partition?
I've backed up a partition using sudo dd bs=8M if=/dev/sda2 | gzip > /someFolderOnSDB/sda2.img.gz.
The image is stored on a separate disk sdb.
When restoring it using gunzip -k /mnt/bkp/sda2.img.gz ...
- 1
-1
votes
1
answer
49
views
How can I evaluate tars and gzips for validity without expanding them?
LONG ago I wrote a backup script for our site and have updated it ever since. However, occasionally things go wrong and some of the older backups are now broken.
In days gone by I had used the utility ...
- 224
230
votes
9
answers
472k
views
tar: Removing leading `/' from member names
root@server # tar fcz bkup.tar.gz /home/foo/
tar: Removing leading `/' from member names
How can I solve this problem and keep the / on file names ?
- 467
0
votes
0
answers
1k
views
How to extract .gtar file?
I have a .gtar file, I wanted to extract it, but it is not getting extracted.
Below are the methods I have tried with its output. I supposed to be contained bunch of text files.
$ gtar -xf file.gtar
...
- 22.9k
20
votes
2
answers
53k
views
7zip, xz, gzip, tar, etc -- what are the differences? [closed]
what factors should be considered when choosing among 7zip, xz, gzip, tar, etc. for compressing and archiving files?
- 160
4
votes
2
answers
857
views
Rename .gz files according to names in separate txt-file
I want to rename .gz files according to names in separate txt-file.
I have a map with .gz files with the names:
trooper10.gz
trooper11.gz
trooper12.gz
etc.
and I have a separate txt-file with the ...
- 34.4k
1
vote
1
answer
556
views
gzip -t output "gzip: filename.gz: extra field of X bytes ignored"
I have a bunch of .gz files I'm checking the integrity after data transfer with gzip -t -v file
the output I'm getting is
gzip: C2_CRRA200017850-1a_H3LJWDSXY_L1_2.fq.gz: extra field of 6 bytes ignored
...
- 245k
218
votes
1
answer
363k
views
How do you extract a single folder from a large tar.gz archive?
I am using this command on a 5GB archive
tar -zxvf archive.tar.gz /folder/in/archive
is this the correct way to do this? It seems to be taking forever with no command line output...
- 131
2
votes
2
answers
11k
views
What's the difference between gzip and compress?
What is the difference between gzip and compress? What are the advantages of using compress function. I could see a size difference in the compressed file, but apart from this what are the main ...
- 3,750
1
vote
1
answer
2k
views
Normal gz file not extractable by tar [duplicate]
I have a gz archive but for some reason tar said that the format is incorrect even though I can double click it in mac Finder and extract it normally, and file command shows the same format just like ...
- 8,850
0
votes
1
answer
306
views
Extract first n bytes from .tar.gz and output as a .tar.gz in a single command
I have a .tar.gz as input and want to extract the first 128 MiB of it and output as a .tar.gz in a single command. I tried:
sudo tar xzOf input.tar.gz | sudo dd of=output bs=1M count=128 iflag=...
- 450k
1
vote
0
answers
156
views
Patch compressed file
is there any way to patch a file, that is compressed via gzip without decompressing it first? I tried
patch -n <(gzip -d file.gz) patch_to_file.diff > new_file
but I only got
patch: **** Can't ...
- 21
0
votes
1
answer
306
views
Archive verify `gzip -v -t` prints success to stderr or I'm doing something wrong? [duplicate]
I want to add gzip file verification to my backup script.
My backup script creates logs for stdout and stderr like this:
scp -i $BCKIDENTITYFILE $BCKLOG "$BCKREMOTEUSER"@"$BCKHOST":...
- 120
1
vote
0
answers
572
views
tar not working with sudo
I am trying to extract a file into /usr/local/ using
$ sudo tar -xvzf /home/vyom/Downloads/go1.19.4.linux-amd64.tar.gz -C /usr/local/
tar (child): gzip: Cannot exec: No such file or directory
tar (...
- 133
0
votes
3
answers
82
views
Is it wrong to do tar czf somename.tar some_*_files?
Is it wrong to do tar czf somname.tar some_*_files instead of tar czf somname.tar.gz some_*_files, then of course tar xf somename.tar? That is : without adding the .gz extension.
This question is ...
- 83.9k
0
votes
1
answer
209
views
How to backup many large files to single compressed file on S3
I have an application that has many thousands of files totaling over 10TB.
I'd need to backup this data somewhere (probably to AWS S3).
I'd like to:
compress data being backed up
save the backup as a ...
- 8,322
34
votes
4
answers
7k
views
GZip doesn't produce the same compressed result on macOS vs Linux
I have a few thousand files that are individually GZip compressed (passing of course the -n flag so the output is deterministic). They then go into a Git repository. I just discovered that for 3 of ...
1
vote
1
answer
77
views
Untar file tar compressed file to different directories [duplicate]
I have a .tar.gz file which may have the following files:
folder1/folder2/folder3/imp_folder1/file11.jpg
folder1/folder2/folder3/imp_folder1/file12.jpg
folder1/folder2/folder3/imp_folder2/file21.jpg
...
- 22.9k
0
votes
1
answer
2k
views
Why tar command uses gzip command through 'z' option?
In Linux Ubuntu about the 'tar' command for these versions:
tar -tzf /path/to/filename.tar.gz # Show the content
tar -xzf /path/to/filename.tar.gz # Extract the content
Observe both commands use ...
- 79.5k
0
votes
1
answer
2k
views
Unzip compressed dump and import via psql
I'm trying to dump a huge database and compress the dump in order to not have to wait hours till it's done.
I dump the database the following way:
pg_dump -Fc -U -v | gzip > db$(date +%d-%m-%y_%H-%...
- 34.4k
1
vote
3
answers
1k
views
Merge possibly truncated gzipped log files
I have multiple log files from each day that I need to merge together. Each comes from a different server. The job that puts them there sometimes gets interrupted and files get truncated. In that ...
- 908
3
votes
2
answers
992
views
gunzip multiple gz files with same compressed file name in multiple folders
I have a directory where there are multiple folders, each folder contains multiple .gz files with the same zipped file name "spark.log". How can I unzip all of them at once and rename them ...
- 558k
1
vote
1
answer
269
views
Sequentially extract files from a tar.gz file
I have a large tar.gz file with 122000 files that I'm downloading from the web to a compute resource that is limited to 14GB. The tar.gz is 3.3GB and the fully extracted archive is 29GB. I know I can ...
- 11
3
votes
1
answer
16k
views
Multithreaded xz, with gzip, pv, and pipes - is this the most efficient I can get?
I'm excited to learn that xz now supports multithreading:
xz --threads=0
But now I want to utilise this as much as possible. For example, to recompress gzips as xz:
gzip -d -k -c myfile.gz | pv | ...
- 1
2
votes
1
answer
280
views
Convert dos line endings in .gz zipped file faster
I have 120 .gz files (each about 5G) with dos line endings, my goal is to convert those to unix line endings, but I don't want to wait multiple days.
Here is my current approach:
function conv() {
...
- 558k
5
votes
3
answers
6k
views
How to check what compression type an RPM package uses?
I recently realized that the RPM packages shifted from gzip-compression to xz-compression a few years ago. I need to check is the compression type of an RPM package that I have. I also need to check ...
- 2,520