rsync backup stragegy how to avoid syncing corrupted files?

Question

If I use rsync for an automated backup, how can I avoid it syncing files that have been corrupted at the source?

For example:

I have /source/*.* and it's periodically automatically synced to /destination/*.* where /source and /destination are different physical disks. Each time the files are updated, when rsync runs it copies the updated files to the destination.

How can I avoid it copying a file which has become corrupted due to HDD failure? Obviously I do not want the corrupted file copied to the backup drive.

There are two different modes for deciding what to sync.

the -c checksum swich which does a file checksum.

-c, --checksum
      This changes the way rsync checks if the files have been changed
      and  are in need of a transfer.  Without this option, rsync uses
      a "quick check" that (by default) checks if each file’s size and
      time of last modification match between the sender and receiver.
      This option changes this to compare a 128-bit checksum for  each
      file  that  has a matching size.  Generating the checksums means
      that both sides will expend a lot of disk I/O  reading  all  the
      data  in  the  files  in  the transfer (and this is prior to any
      reading that will be done to transfer changed  files),  so  this
      can slow things down significantly.

and the -u switch flag:

-u, --update
  This  forces  rsync  to  skip  any  files  which  exist  on  the
  destination  and  have  a  modified  time that is newer than the
  source  file.   (If  an  existing   destination   file   has   a
  modification time equal to the source file’s, it will be updated
  if the sizes are different.)

Clearly -c will not have the desired behaviour in this case. If the source file is corrupted it will have a different checksum and be copied to the backup.

My assumption is that -u will do exactly what I want. Because the corrupted file will have the same mtime as the version on the backup drive, if the source is corrupted it will not be copied across.

My question is, is using -u here to copy files an adequate backup strategy?

Alternatively, should I consider a different approach or a different tool entirely?

Answer 1

There is no way to identify files that have been "corrupted at source". What you may be able to do is to identify files that have been written to one or more disk blocks that can no longer be read, but this will only be a subset of the possible set matching your criterion.

As you have suggested, the -u (--update) flag will instruct rsync to avoid attempting to read and update any file either that has a newer modification time on the destination regardless of any other attribute, or that has the same modification time and size.

What might be better is to use rsnapshot for making your backup copies to an ext4 target (not NTFS or FAT/exFAT), and to use smartctl to implement SMART disk checking on your source.