Does rsync verify files copied between two local drives?

Question

I want to make a fresh new copy of a large number of files from one local drive to another.

I've read that rsync does a checksum comparison of files when sending them to a remote machine over a network.

1. Will rsync make the comparison when copying the files between two local drives?

2. If it does do a verification - is it a safe bet? Or is it better to do a byte by byte comparison?

Answer 1

rsync always uses checksums to verify that a file was transferred correctly. If the remote file already exists, rsync may skip updating the file if the modification time and size match the source file, but if rsync decides that data needs to be transferred, checksums are always used. So if you're making a fresh copy of a file, rsync will verify that the copying process was sound.

As for the verification itself, rsync uses MD4. While long considered obsolete for secure crytographic hashes, MD4 remains adequate for checking file corruption.

Source: the man page and eyeballing the rsync source code to verify.

Answer 2

rsync makes a checksum comparison before copying (in some cases), to avoid copying what's already there. The point of the checksum comparison is not to verify that the copy was successful. That's the job of the underlying infrastructure: the filesystem drivers, the disk drivers, the network drivers, etc. Individual applications such as rsync don't need to bother with this madness. All rsync needs to do (and does!) is to check the return values of system calls to make sure there was no error.

Answer 3

You also may use SafeFileManager. Find it at: http://mindbytez.com/sfm

It automates copy and bit to bit verify in one go.

Download the .jar file. Be sure to have installed java runtime jre7. Right click on the .jar - file, then click "open with java runtime".

Answer 4

rsync does not do the post-copy verification for local file copies. You can verify that it does not by using rsync to copy a large file to a slow (i.e. USB) drive, and then copying the same file with cp, i.e.:

time rsync bigfile /mnt/usb/bigfile

time cp bigfile /mnt/usb/bigfile

Both commands take about the same amount of time, therefore rsync cannot possibly be doing the checksum—since that would involve re-reading the destination file off the slow disk.

The man page is unfortunately misleading about this. I also verified this with strace—after the copy is complete, rsync issues no read() calls on the destination file, so it cannot be checksumming it. One more you can verify it is with something like iotop: you see rsync doing read and write simultaneously (copying from source to destination), then it exits. If it were verifying integrity, there would be a read-only phase.