Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.
Sign up to join this community
I run commands:
tar -cf myArchive.tar myDirectory/
gzip myArchive.tar
then I copy the file over a lot of unreliable mediums, and later I unpack it using:
tar -xzf myArchive.tar.gz
The fact that I compressed the tar-ball, will that in any way guarantee the integrity, or at least a CRC of the unpacked content?
tar itself does not write down a checksum for later comparsion. If you gzip the tar archive you can have that functionality.
tar uses compress. If you use the -Z flag while creating the archive tar will use the compress program when reading or writing the archive. From the gzip manpage:
The standard compress format was not designed to allow consistency checks.
But, you can use the -z parameter. Then tar reads and writes the archive through gzip. And gzip writes a crc checksum. To display that checksum use that command:
$ gzip -lv archive.tar.gz
method crc date time compressed uncompressed ratio uncompressed_name
defla 3f641c33 Sep 25 14:01 24270 122880 80.3% archive.tar
From the gzip manpage:
When using the first two formats (gzip or zip is meant), gunzip checks a 32 bit CRC.
Yes, the gzip file format contains a CRC-32 checksum that can be used to detect if the archive has been corrupted.
Of course, while the checksum lets gzip tell you that the archive is corrupted, it doesn't actually do anything to help you recover the data inside the archive. Thus, it's mostly useful for things like checking that an archive you just downloaded off the web really was downloaded correctly.
If you're actually worried about storing or transmitting your archives over unreliable media, you may want to consider using an archive format like par that actually provides error correction in addition to error detection. Of course, the down side of such formats is that the redundancy needed for error correction necessarily increases the file size somewhat.
tar does not have an integrity check.
Example:
$ echo JJJJJJJJJJJJJJJJJJ > b
$ tar cvf a.tar b
$ sed -i s/JJJJJJJJJJJJJJJJJJ/tttttttttttttttttt/g a.tar
$ tar xvf a.tar
$ cat b
tttttttttttttttttt
See, the content of a.tar archive changed, so the file b has completely different content, but tar did not notice this.
This is true for any tar, including tar-1.28 (latest) with both tar formats --format=gnu --format=posix.
The pax command (alternative tar reader) pax -r < a.tar also does not notice archive changes.
tar (at least without the -z option) does not do integrity checking. Also, the accepted answer states this. The question is: does gzip (or the use of the -z option) add integrity checking?
May 16, 2015 at 22:05
If tar finds errors when unpacking, it will print a message and exit with a non-zero exit value. This behavior is independent from the compression algorithm used after the tar file has been created.
If you want to verify that the file was successfully sent to the destination over an unreliable link, then create a md5 sum of the file prior to sending and verify the md5 sum after the reception.
tar itself. You could add an additional layer if needed: If the tar file's integrity is verified, then the content inside the tar archive is OK, too. But all that should be taken care of by the protocol used to transfer the data in the first place.
The PKZip package (win/dos) comes with a program called PKZipFix that can recover files from damaged archives. I have used this utility in the past, it can recover files from moderately damaged archives that would not decompress.
