3

I regularly backup some files to an external hard drive using rsync. Occasionally, I'll check the integrity of the files on disk with rsync in checksum mode to detect any bit rot.

This works fine, until today. My command is as follows:

rsync -nrvci source dest

-n for dry-run, -r for recursive, -v for verbosity, -c for checksum mode, and -i to see why a file is being changed.

This time, some files are being reported as having different checksums, which the itemized list shows with this string for all of the problem files: >fc.T......

This tells me according to the man page that the file is being transferred >, that it's a file f, that the checksum is different than the destination c, and that because of this the timestamp is going to be changed upon copy T. My assumptions could be wrong.

Anyways, when I run md5sum source/file dest/file, it matches. I've even gone and added --checksum-choice=md5 to my command to force rsync to use the md5 method to no avail, it still reports that the checksums are different.

What's going on with rsync and why is it seeing these files as different when they aren't?

1
  • This is just a guess, but the timestamp mismatch may be all rsync needs to declare the files as mismatched. My concern would be, what performed a write (on source or destination) that changed nothing (maybe a change was written, then rewritten to what it was before, but the timestamp was changed).
    – killermist
    Jul 30, 2021 at 9:20

2 Answers 2

2

I had the same issue, running rsync on my macbook, syncing files from a linux ubuntu server. Syncing two directories (with rsync -vrc), it would ignore all non-changed files, except for one; always the same one.

  • Using -i also reported >fc.T...... (or >fc........ if --times was provided).
  • md5sum gave the same results for both the source and destination file
  • Using another target-directory --> the same result
  • Renaming the file --> same result
  • Making the source file writable to the user doing the rsync --> same result
  • touch-ing the source file --> same result

After trying all this (and being very sure it was not some sort of problem with the storage of the source file: mds5um consistently gave the same checksum), I expect it is some problem in rsync itself.

After installing a newer rsync version (stock macos Ventura 13.2.1 still has rsync version 2.6.9 protocol version 29; used homebrew to install rsync version 3.2.7 protocol version 31), the problem disappeared.

0

I think there was a one-off issue with rsync or Veracrypt. When the medium was unmounted and remounted, the problem vanished. So "turn it off and back on" prevails again.

Edit: I had this problem again just over a month later. This time I was also able to solve it by running rsync with sudo instead of as a plain user.

1
  • 2
    Don't accept this sort of problem as inevitable. You should look through the various log files to see if there are errors with the disk, or the connection to it, which would indicate a poor cable, or low power, or electrical interference etc. Also run smartctl on the disk if possible. Errors should be reported by rsync as errors, not as a difference in data; that sort of silent corruption is not good.
    – meuh
    Jul 31, 2021 at 9:38

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .