I noticed the original post is rather old, however, I think this information can still be valuable to those looking for a solution to verify that files are copied correctly. Rsync might be the best method to copy data and answers given in this thread are good, however for those not that experienced with Linux, I will try to give a more detailed explanation.
Scenario: You just copied data from a disk to another, with lots of sub-directories and files. You want to verify that all the data is copied correctly.
First check that md5deep is installed by issuing the command
If you get a message saying something like 'command not found', then install md5deep by
apt-get install md5deep.
It's assumed you only want to deal with regular files. If you want to deal with other types of files, refer to the -o flag in the md5deep manual. (
Now you are good to go, and we assume that you copied files from
/mnt/backup, substitute these for any directories you are using.
First change to the source directory, this is the original source for the files you copied or backed up:
Then make a checksum of each file:
md5deep -rel -o f . >> /tmp/checksums.md5
This command explained:
-r enables recursive mode
-e displays progress indicator
-l enables relative file paths.
-o f only work on regular files (not block devices, named pipes etc.)
. tells md5deep to start in the current directory.
>> /tmp/checksums.md5 tells md5deep to redirect all output to
Note, if you want to overwrite content in previous versions of
> and not
Note that this command could take quite a while, depending on the io-speed and the size of the data. You could experiment with nice and/or ionice to increase performance of md5deep, but that's outside the scope of this answer.
When the creation of the check sums has finished, you now have a file that has entries similar to:
The first column is the md5 check sum, and the second column is the relative path to the file the checksum belongs to.
If you want to see how many files exists in the checksum file, issue the command:
wc /tmp/checksums.md5 -l
Now, you want to check that the copied data is correct:
md5deep -o f -reX /tmp/checksums.md5 . >> /tmp/compare.result
The only difference from when we created the checksums is -X which displays the current hash of a file if the entry in the checksums.md5 file does not match. So by the end of the test, if
/tmp/compare.result is blank, you can trust that all the files are copied correctly since the checksums matches.
Note that only files listed in the
/tmp/checksums.md5 file will be checked for a correct checksum, if there's any additional files in the
/mnt/backup directory, md5deep will not notify you about these.
You don't necessarily have to use redirection to store output files. Refer to the md5deep manual for further information.
You might have to run md5deep commands as root, depending on the permissions of the files you're handling.