If I use rsync for an automated backup, how can I avoid it syncing files that have been corrupted at the source?
For example:
I have /source/*.*
and it's periodically automatically synced to /destination/*.*
where /source
and /destination
are different physical disks. Each time the files are updated, when rsync runs it copies the updated files to the destination.
How can I avoid it copying a file which has become corrupted due to HDD failure? Obviously I do not want the corrupted file copied to the backup drive.
There are two different modes for deciding what to sync.
the -c
checksum swich which does a file checksum.
-c, --checksum
This changes the way rsync checks if the files have been changed
and are in need of a transfer. Without this option, rsync uses
a "quick check" that (by default) checks if each file’s size and
time of last modification match between the sender and receiver.
This option changes this to compare a 128-bit checksum for each
file that has a matching size. Generating the checksums means
that both sides will expend a lot of disk I/O reading all the
data in the files in the transfer (and this is prior to any
reading that will be done to transfer changed files), so this
can slow things down significantly.
and the -u
switch flag:
-u, --update
This forces rsync to skip any files which exist on the
destination and have a modified time that is newer than the
source file. (If an existing destination file has a
modification time equal to the source file’s, it will be updated
if the sizes are different.)
Clearly -c
will not have the desired behaviour in this case. If the source file is corrupted it will have a different checksum and be copied to the backup.
My assumption is that -u
will do exactly what I want. Because the corrupted file will have the same mtime as the version on the backup drive, if the source is corrupted it will not be copied across.
My question is, is using -u
here to copy files an adequate backup strategy?
Alternatively, should I consider a different approach or a different tool entirely?
--link-dest
into a sequence of backup directories. That way you would back up your corrupted files, but still have an older backup that wasn't corrupted. Likewise if you use a proper backup solution likerestic
orborgbackup
which by default saves a separate "snapshot" per backup. Not an answer since I'm not showing how to actually do things.--link-dest
option torsync
could be used to hard link files that have not changed, rather than copying the again (indeed, this is the point of this option). Bothrestic
andborgbackup
would additionally do deduplication of data. I'm usingrestic
, and backing up five computers at home for over a year in a single backup repository uses much less space than say, Time Machine on macOS (which is using a similar approach torsync --link-dest
), doing the same job.