Questions tagged [gzip]
A data compression program. Originally a GNU utility it has been cloned by most Unix variants.
360
questions
2
votes
1
answer
6
views
gunzip multiple gz files with same compressed file name in multiple folders
I have a directory where there are multiple folders, each folder contains multiple .gz files with the same zipped file name "spark.log". How can I unzip all of them at once and rename them ...
- 131
1
vote
1
answer
35
views
Sequentially extract files from a tar.gz file
I have a large tar.gz file with 122000 files that I'm downloading from the web to a compute resource that is limited to 14GB. The tar.gz is 3.3GB and the fully extracted archive is 29GB. I know I can ...
- 11
2
votes
1
answer
26
views
Convert dos line endings in .gz zipped file faster
I have 120 .gz files (each about 5G) with dos line endings, my goal is to convert those to unix line endings, but I don't want to wait multiple days.
Here is my current approach:
function conv() {
...
- 23
0
votes
0
answers
25
views
use pigz compression on bacula
By default I can use gzip or lzo to compress files when I create backup with bacula.
"compression=GZIP" section: https://www.bacula.org/13.0.x-manuals/en/main/Configuring_Director.html
But ...
- 31
1
vote
0
answers
52
views
Compressing small files with gzip makes its size smaller than with bzip2, why? [closed]
I have a question, just a little thing couch my eyes
it's about compression with gzip and bzip2:
If I understood correctly - bzip2 requires more processing power but compresses files smaller and more ...
- 11
1
vote
1
answer
42
views
Why GZIP utility cares about extension?
Whenever I need to decompress a file which was compressed with Gzip, I need to rename the file with a .gz extension , I tried other compression utility like zip, bzip2 and they don't seem to care ...
- 13
-1
votes
3
answers
94
views
Gunzip extract *.gz files to another location
This code was written to extract multiple .tar files to another location and woks fine
ls -1 ${short_filename} \
| while read file; do \
tar -zxvf "$file" -C ${pentaho_temp_path}/${...
- 11
0
votes
2
answers
250
views
Pipe find list of files into xargs gzip and pipe again into pigz
I need to find files newer than x days, and then turn it into a gzip, but I want to do it using pigz.
For now I'm now doing it the slow way; this works:
find /path/to/src -type f -mtime -90 | xargs ...
- 23
0
votes
0
answers
257
views
Unable to decompress gzip file ("not in gzip format")
I've been tasked with diagnosing an issue with a webpage, and I've been able to trace the issue back to a gzip'ed JavaScript file, jquery.min.js.gz. To view the code, I've downloaded and attempted to ...
-2
votes
3
answers
837
views
Creating a tar.gz archive of multiple directories of different locations -- "tar: Cowardly refusing to create an empty archive"
I'm trying to create an achive:
$ cd /tmp
$ tar -czf test1.tar.gz -C ~/Downloads/dir1 -C ~/Documents/dir2 -C ~/dir3/dir4/dir5
... which is supposed not to preserve the full path of the directories in ...
- 19
8
votes
3
answers
1k
views
tar.gpg vs tar.gz.gpg
Is it redundant to compress a tar archive before encrypting it using GPG? From my understanding, GPG will compress the file that it encrypts. Is it preferable to do tar -> gpg instead of tar -> ...
- 2,144
0
votes
2
answers
146
views
Compress several files according to the pattern
I would like to know how to compress several files according to the pattern using gzip, tar, etc... that is, if I have these files:
server_regTCA.log.2021.02.12
server_regTCA.log.2021.02.13
...
0
votes
1
answer
183
views
Reading multiple zipped files in awk script
I have an awk script named text_processing.awk . I need to pass multiple zipped files named 01JAN21.txt.gz , 02JAN21.txt.gz etc. to this script for further processing. The output file takes the date ...
user511618
0
votes
1
answer
86
views
How to extract specific file from partially downloaded tar?
I have a large tar file but it could not be downloaded completely as the browser crashed when verifying the download. Is it possible to extract some files from this tar?
I am able to view the files ...
- 125
4
votes
3
answers
297
views
Un-concatenate gzipped file
The other day I was collecting some logs from a remote server and unthinkingly gzipped the files into a single file, rather than adding the directory to a tarball. I can manually separate out some of ...
- 43
5
votes
1
answer
353
views
Can gzip -t detect 100% of truncated-download errors?
Scenario: a single 1g CSV.gz is being written to an FTP folder. At the same time, my client machine connects over sFTP to that folder and attempts to pull it down.
Q: After fetching that file, ...
- 153
1
vote
1
answer
39
views
Archive directory with tar/gz to multiple locations and omit an additional copy
I have a simple backup script, which also creates a tar/gzip archive of local data to an external USB device and then copies that archive to a second USB device.
For example:
usb1="/mnt/usbone&...
2
votes
1
answer
344
views
How do I decompress a folder of files that I accidently saved as foldername.gz instead of foldername.tar.gz
I'm not sure how I did this. I had to wipe my dedicated server and start again, so I backed up everything to a remote VPS first; during this process, I compressed a folder full of other folders as ...
- 163
-1
votes
1
answer
208
views
Pick random line from a compressed text file with zcat/gz or altnernatives
I have a simple bash script that picks a random line from a text file. This is originally done with shuf file.txt | head -1 (the same result can be achieved with cat file.txt | shuf | head -1). I also ...
0
votes
1
answer
328
views
"gzip: stdin: invalid compressed data--format violated" error when extracting a tgz archive
I have to open a tar file. I used the following command line on my Unix shell:
tar xfvz raw-GWA-data.tgz
I found the following error message:
gzip: stdin: invalid compressed data--format violated
tar:...
0
votes
1
answer
30
views
What is the difference between Gzip and SZ error-bounded Lossy compression
I'm asking on the difference between Gzip and SZ error-bounded lossy compression and is there any relation between these compressors?
- 1
0
votes
1
answer
56
views
How to zgrep and then gzip multiple files individually?
I have the following code which searches for a term in multiple files (file1.json.gz, file2.json.gz, etc etc) using zgrep, and then using gzip it saves the output into a single file:
zgrep -i 'help' /...
- 1
3
votes
2
answers
320
views
Tarring-gzipping only hidden directories (or files + directories)
Based on this, to tar all hidden files in the current directory one can use
ls -A | egrep '^\.' | tar cvf ./test.tar -T -
However, how can one tar only all hidden directories or all hidden directories ...
- 261
1
vote
0
answers
99
views
How to tar-gzip a directory and put it in the /tmp
Hey im stuck on a assignment and i dont know how to solve it really because iam quite new with working in linux. I have to place a directory in a tar-ball and zip it using gzip.
At the moment this is ...
0
votes
1
answer
56
views
Is it efficient to archive-n-compress or compress-n-archive storage size-wise?
I'm trying to archive old files (1000+) and thinking of two options
gzip individual files and then create a zip
create gzip using tar cvzf on group of files
Not sure what would be most efficient ...
- 25.2k
0
votes
1
answer
120
views
Create set of fixed sized compressed files from a large number of files
I have set of large number of files to be archived in a linux server. I need to compress those files in to set of 1GB compressed files.
I am using gzip to compress the files but I don't know the final ...
- 101
0
votes
0
answers
197
views
Combine .gz file with .tsv file (columns) using command line
I have two files.
.gz file with with dimensions 12m x 5 columns
.tsv file (no compression) with the same 12m rows and 9 more columns
I normally would do this in R where I would read in both objects, ...
- 175
0
votes
2
answers
148
views
I cannot untar a zip file keep getting error
From the link below I try to follow the steps and I got the curl portion to work but when I type
tar zxf admit.tar.gz
I get
tar: Unrecognized archive format
tar: Error exit delayed from previous ...
0
votes
1
answer
150
views
1TB drive compressed shows only 3.8GB, what did I do wrong?
On Linux Mint 20.2 Cinnamon I would like to create a disk image of my secondary disk drive (SATA) containing Windows 10, not that it matters now, directly gzip'ed using the Parallel gzip = pigz onto ...
- 24.4k
0
votes
1
answer
27
views
File with div.gz extension (mistake?)
I tried downloading a zipped folder from Arxiv (https://arxiv.org/format/math/0606086 under DVI)
but it downloads as div.gz. I understand that this is the TeX output. I tried TeX and various unzip ...
- 103
0
votes
1
answer
38
views
Getting rid of spurious directories with tar -xzvf, while gunzipping
I want to trim the path of the gunzipped tarball so that some "spurious" leading directories are excluded. Let me explain with an example.
I have the following directory structure, as ...
- 3
1
vote
1
answer
169
views
How to split, add text for each split and gzip file
I want to take a huge file data.txt and:
Split it by 256m
Add the first 3 lines of data.txt to each split file
gzip it
data.txt looks like:
aaa
bbb
ccc
<data>
<data>
<data>
...
<...
- 1,205
3
votes
1
answer
585
views
Fastest way to find lines starting with string in gzip file
I have a flat database based on 65536 files, each one containing one word by line starting with two hexadecimal characters.
They look like this:
afword
46word2
Feword3
...
I make tens of thousands of ...
- 71
0
votes
0
answers
357
views
unzip xxx.csv.gz to PROCEDURES_ICD.csv
I would like to unzip a .csv.gz file to .csv. I cannot directly unzip because it will conduct an error: unable to expand (error 79: inappropriate file type or format). I tried the endorsed answer in ...
0
votes
1
answer
912
views
can't uncompress a gzip compressed data
I have extracted a file 'linux' from ubutu installation .iso file.
The command file linux gives me this output.
linux: gzip compressed data, was
"vmlinuz-5.4.0-42-generic.efi.signed", last ...
- 305
0
votes
1
answer
122
views
How to gzip all .BRIK in all Sub-Directories
I have a lot of files with .BRIK extension that take up a lot of space and need to be gzip. However, the files are scattered in multiple folders and few-level deep. I cannot find an answer to gzip *...
- 1
0
votes
1
answer
88
views
gzip, send to server, ungzip
I am working on a script to backup and gzip a set of MySQL tables, scp/ssh them to a different server and then unpack them
current script is:
#!/bin/bash
DATE=`date +"%d_%b_%Y_%H%M"`
...
0
votes
0
answers
39
views
help with the firmware file of the radio
help with the firmware file of the radio. I unpack the file with the 7zip program, with the same program I can not return the file back to its original state with the same file size.
https://cloud....
- 1
2
votes
1
answer
748
views
tar xf command giving file not found
I am trying to extract one file out of the whole tar.gz and it's not working. Below are the commands i tried to simulate the issue
mkdir test
touch test/version.txt
echo "1.0.2" > test/...
- 281
1
vote
1
answer
115
views
Modify the data type from .tar file and how to recover a corrupted one
Like the original question bellow, I need to recover a corrupted tar file. But after run:
file file.tgz
I got this return:
file.tgz: data
My question is about this return above and how to change ...
0
votes
0
answers
97
views
Compression vs. redundancy: do they cancel each other out?
Does it make sense to compress a tarball (or any kind of file, really), e.g., using gzip or bzip2, while at the same time creating redundancy files for it, e.g., a par2 file?
The context is that I am ...
- 101
1
vote
0
answers
27
views
What will happend if run this command `gzip -r filename .`?
I ran this gzip -r filename .
It zipped filename to fielname.gz and all the .(dotfiles/hidden files) in all the directories.
Is this the expected behaviour?
How to undo gz of .(dotfiles/hidden files)?...
1
vote
2
answers
112
views
Uncompressed files by content double gzipped by mistake
I have some files that I gzipped twice by mistake with no file extension
For example gzip -dc file outputs some binary gibberish for some of the files and for some other files it's okay
I would like ...
- 111
1
vote
1
answer
630
views
How to uncompress a gzipped partition image and dd it directly to the destination partition without writing to current partition?
I've backed up a partition using sudo dd bs=8M if=/dev/sda2 | gzip > /someFolderOnSDB/sda2.img.gz.
The image is stored on a separate disk sdb.
When restoring it using gunzip -k /mnt/bkp/sda2.img.gz ...
- 325
0
votes
1
answer
82
views
Compressing multiple files to .gz misbehaving
I had 100 images from a Raspberry Pi project and in order to transfer them to another computer, I used the "Compress" interaction after selecting all of the images and using the right click ...
- 111
0
votes
1
answer
27
views
What would the value of hyphen actually look like in this instance used to refer to stdin for this archive command?
I am following a guide to learn terminal commands and have encountered the use of hyphens to refer to stdin in the following command used to find all files named 'file-A' within the directory ...
- 115
14
votes
8
answers
13k
views
How to gzip 100 GB files faster with high compression
We have 100+ GB files on a Linux machine, and while trying to perform gzip using below command, gzip is taking minimum 1-2 hours to complete:
gzip file.txt
Is there a way we can make gzip to run fast ...
- 369
0
votes
2
answers
788
views
Gzip of folder while retaining original timestamp
I want to do gzip of folder, and also want to retain original timestamp for that folder. Can someone suggest any way around it.
i tried below mentioned code. The timestamp which i am getting the time ...
- 29
0
votes
1
answer
411
views
Split file from a streaming output with custom name and gzip
I have a script with a while loop that will print a text. I want to save it into files with custom name.
Script:
#!/bin/bash
while true
do
echo "Press [CTRL+C] to stop.."
done
I can run ...
- 303
1
vote
1
answer
544
views
Updating a large compressed file without creating temp file
I'm trying to figure out a way that I can update a compressed file (currently using zip, but am open to tar/gz/bz derivatives too) on a linux server without creating a temp file for the file to be ...
- 113