Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.
Sign up to join this community
Is gzip atomic?
What happens if I stop the gzip process while it's in the middle of gzipping a file?
If it's not atomic, and if I already pressed Ctrl+C on a gzip *.txt process, how do I safely resume?
(I am not just curious about how to resume, but also about whether gzip specifically is atomic.)
Is gzip atomic?
No. It creates a compressed file and then removes the uncompressed original.
Specifically, it does not compress a file in situ and there is a period of time while the file is being compressed where,
What happens if I stop the gzip process while it's in the middle of gzipping a file?
If you stop the gzip process with a catchable signal (SIGINT from Ctrl C, for example) it will cleanup partially created files. Otherwise, depending on the point at which it's stopped, you may end up with a partially compressed file alongside the untouched original.
If it's not atomic, if I already pressed Ctrl+C on a gzip *.txt process, how do i safely resume?
You delete the partially compressed version (if it still exists) and restart the gzip.
SIGINT or SIGTERM for which gzip installs signal handlers which remove the output file).
gzip mid flow there's always a tiny race condition there. Alternatively you can tell gzip always to overwrite target files, which sidesteps most of the cleanup issues.
It's not atomic (the Unix filesystem API doesn't really provide any way to perform atomic operations that affect multiple files), but it is fail-safe. The compressed file is a new file, it doesn't overwrite the original, and it doesn't delete the original file until it has completed creating the compressed file (this can actually cause an issue if you don't have enough disk space for both files).
If it gets an error or you interrupt the compression, the original file will remain unchanged. The partial compressed file will usually be removed.
There's no way to resume it in the middle, you just start it over from the beginning.
sed -i).
You don't need to worry about that because gzip creates a new .gz file, populates it with the compressed content, then deletes the original file. So if you stop the process in the middle, it won't affect your original file.
.txt files already successfully processed by gzip will have been replaced with .txt.gz compressed files, so you can safely run gzip *.txt again - only the files that haven't been processed yet will be compressed.
The file that was being processed by gzip at the time you pressed Ctrl-C will be unmodified - gzip won't replace it until after successfully compressing it.
No, it's very unatomic. This can get you in big trouble if you gzip a file that's being occasionally appended to, like a Web log.
Gzip reads, creates the .gz file (with current timestamp), copies the original file's timestamp, then deletes the original.
Certain interruptions may leave a stray, unfinished .txt.gz file right next to the .txt file. This then creates a data integrity issue: Which is the real file? Is this
.txt.gz? Or.txt file? Ortxt.gz, and a newly created .txt file?(This last happens when you go into your HTTP log directory and go gzip *).
I generally find it's prudent to sort this out by hand, unless you know exactly what happened because you just did it.
Fortunately gzip usually operates serially so you should only have this problem with one file. Paralleling gzip isn't a good idea - even though it'll use CPU more fully, it'll thrash the disk forcing it to read several files at once, greatly slowing down all gzip's. SSD or RAMdisk, on the other hand...
CTRL+Zinstead ofCTRL+C, then kill or resume the interrupted job (it answers with a numbern[--[n]+ Stopped-- gzip ...] then you can resume with%nor withfg, or withbg... in the same way you can kill it withkill %n).