2

I was wondering:

  1. Is it possible to create a checksum of a directory (using something like md5sum)
  2. Is it possible to recursively create checksum for each file inside the dir (and then print it out)?
  3. Or both?

I'm using bash

2

md5sum won't take directory as input, however

 tar cf - FOO | md5sum

will checksum it, if a file is change any place within FOO, checksum will change, but you won't have any hint of which file. The checksum will also change if any file metadata changes (permissions, timestamps, …).

You might consider using :

 find FOO -type f -exec md5sum {} \;  > FOO.md5

which will md5 every file individually, and save the result in FOO.md5. This makes it easier to check which file has changed. This variant only depends on file content, not on metadata.

  • If you have bad luck, both proposed commands may take a week. Regarding the second one: find(1) has been enhanced in 1989, so you may like to read again the man page and search for "execplus" (-exec cmd +). – schily Sep 10 '15 at 9:41
  • I suppose OP checksum file to monitor change on a "small" directory, but yes, if you have zillion of file and few Tera bytes, this could take long. – Archemar Sep 10 '15 at 9:46
  • Well, the main issue here is that tar traverses the whole tree and this may affect a lot of files unless you e.g. use the star -D option.For small trees, the command may be sufficient, but unless you carefully specify the archive format this may even not be helpful as modern TAR archive formats include the last file access time. – schily Sep 10 '15 at 9:57
  • Thanks a lot, this is what I was looking for. My directories have around 1gb with reasonable file count. The last thing lasts about a minute. – drigler Sep 10 '15 at 11:27
0

ZFS creates such a checksum but this is for internal use only.

In order to calculate a checksum from an object, you need to read this object and for directories, this is not granted to be possible.

Many filesystems return the EISDIR error when trying to read(2) a directory and ZFS in addition sets the "size" of the directory to the number of entries in that directory instead of the number of bytes (what you may expect when you like to read(2) a directory).

The fact that some filesystems still allow you to read(2) directories is a bug - a left over artefact from the 1970s when there was no readdir() call.

BTW: on my WOFS, all directories have size 0 as they don't have any file type content because the files instead say in their meta data "hey, I am in this directory". If you wonder what WOFS is: all other basic concepts from WOFS have been incorporated into ZFS.

  • Thanks for your answer, the zero-size didn't cross my mind. :) – drigler Sep 10 '15 at 11:21
0

1.Why do you need md5sum on directory? If you want to be sure of file consistency and/or integrity you can tar it and make hash of tar/tar.gz file.

2.If you have sub directories it will be a little harder. If you have files only, try this as example:

#!/bin/bash

for i in /home/{username}/Books/rhel/; do
  md5sum "$i"
done

Output example:

dd7a684cc8668d208ca5dcf00bc58e8d  Red_Hat_Enterprise_Linux-6-Deployment...
775602071a1ec5a1ac1a99cee9d065fa  Red_Hat_Enterprise_Linux-7-High_Av..
  • You're right, but I don't want to tar my files :) Thanks for your answer – drigler Sep 10 '15 at 11:21
0

On https://blake2.net/ they claim:

" BLAKE2 is a cryptographic hash function faster than MD5, SHA-1, SHA-2, and SHA-3, yet is at least as secure as the latest standard SHA-3. BLAKE2 has been adopted by many projects due to its high speed, security, and simplicity. "

Therefore, some tools are using it, e.g. rmlint https://github.com/sahib/rmlint .

Anyway, following https://unix.stackexchange.com/a/228758/9689 , you can also do :

find "${dirname}"/ -type f -exec b2sum -b -l 256 {} \;  > "${dirname}".blake2sum_l256

And later check with:

b2sum -c "${dirname}".blake2sum_l256

Or this to see only failing ones:

b2sum --quiet -c "${dirname}".blake2sum_l256

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.