1

I have a directory with a content. To ensure nothing will get lost with copying I compressed the directory and now I have a dir.tar.gz file.

The copying still troubles me and the files are damaged so I would like to find out if it is due to compressing or copying on the files, so I generated a crc32 of the compressed file.

I am not able to generate the crc32 of the original content of this directory which means I cannot use crc32 on it. Is it possible?

Also if I manage to compare these two numbers , if they are the same that means that files were not harmed?

  • "ensure nothing will get lost with copying" --> did you consider using rsync? "I am not able to generate the crc32 of the original content of this directory" --> do you mean you want a "recursive" crc32 of all the files, and then compare that after you extract from the copied tarball? – drewbenn Jul 7 '16 at 15:09
4

Depending on how "strict" you want the checksum to be, a typical Linux install provides at least three options:

  1. sum - this is a quick and simple checksum. It's not cryptographically secure and so deliberate tampering would be possible, but it should handle file copying errors well enough.

  2. md5sum (sometimes just md5). This calculates an MD5 based checksum on the file. This is slower than sum but more resilient. The MD5 algorithm has been broken, but it's not easy.

  3. sha1sum This calculates an SHA1 checksum; even slower but the strongest against tampering.

  4. crc32 This may not be present on all systems; it's a perl program.

In your use case I'd probably stick with sum.

By calculating the checksum on the tar.gz before transfer and after transfer you have reasonable assurance that the transfer was good (sum and crc32 are weakest, but good enough).

There's another possibility... since you're compressing the data then you can try and decompress the result at the other end with gzip -dc file.tar.gz > /dev/null. If this generates an error then the file is corrupt, otherwise it's good. Depending on the size of the file this might be easier because it doesn't require a second file to be transferred. It may not fit your use case, though.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.