0

Now 5 years after this question was orignially asked, is there a way for rsync to do the following?

  • same time and same size ► skip file (no transfer, no checksum)
  • different sizes ► transfer file (no checksum)
  • different times and same size ► perform checksum ► transfer only if checksums differ

In my case, I have a network share that is accessible through multiples OS (one that uses UTC and another that uses RTC - I do not have admin/root access on either, so I cannot change it). Modifying the file in one OS changes the time so that it is a few hours ahead/behind the "real" time, depending on which OS I browse the files. If I modify a file, the timestamp will be "incorrect".

  • rync's option -u, --update This forces rsync to skip any files which exist on the destination and have a modified time that is newer than the source file. (If an existing destination file has a modification time equal to the source file’s, it will be updated if the sizes are different.) -- This is not exactly what you want, but should help when the time differs 'one way'. So if you 'dry run' rsync with swapped source and target you should be able to catch time differences both ways. – sudodus Jun 18 '18 at 4:16
0

On the one hand, the linked question is very confusing. The answer by 9mjb is correct.

On the other hand, if you are using rsync on a local mount of a network filesystem, there is no way it can checksum the remote file without downloading the whole file anyway! So you are going to be stuck with slowness here. Assuming you have less network bandwidth available than the disk speed.

Except I don't understand what problem you are trying to explain about times, either. If you use -a, it should transfer the exact times from the source system. If you don't use -a, it won't transfer the original times from the source filesystem, therefore the times will not match exactly for subsequent transfers anyway. It sounds like you don't want -u behaviour, but then you can just not use -u :-).


rsync is efficient by default. (At the cost of "if time and size both match, the chance that the files are different is insignificant, I'm willing to take the risk not to transfer").

[rsync] is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use.

Rsync finds files that need to be transferred using a "quick check" algorithm (by default) that looks for files that have changed in size or in last-modified time.

The important thing to understand is that the rsync delta-transfer algorithm, and the "quick check", are separate.

It sounds like you don't want the behaviour of --checksum. --checksum disables the "quick check" part. In that case, don't use --checksum.

-c, --checksum

This changes the way rsync checks if the files have been changed and are in need of a transfer. Without this option, rsync uses a "quick check" that (by default) checks if each file’s size and time of last modification match between the sender and receiver. This option changes this to compare a 128-bit checksum for each file that has a matching size. Generating the checksums means that both sides will expend a lot of disk I/O reading all the data in the files in the transfer (and this is prior to any reading that will be done to transfer changed files), so this can slow things down significantly.

The sending side generates its checksums while it is doing the file-system scan that builds the list of the available files. The receiver generates its checksums when it is scanning for changed files, and will checksum any file that has the same size as the corresponding sender’s file: files with either a changed size or a changed checksum are selected for transfer.

Note that rsync always verifies that each transferred file was correctly reconstructed on the receiving side by checking a whole-file checksum that is generated as the file is trans‐ ferred, but that automatic after-the-transfer verification has nothing to do with this option’s before-the-transfer "Does this file need to be updated?" check.

0

Amazingly, this does not look possible with a built-in rsync option!

This may work:

rsync -an --info=name src dest \
    | rsync -an --info=name --checksum --files-from - src dest

Explanation:

The first rsync command uses the default rsync mode of checking time and size. It outputs any directory or filename with a different time or size. Because of the -n flag, no files are transferred.

The second rsync command takes the list of files with a changed time or size from the first rsync and then runs rsync again on just those files with the --checksum argument.

The second command has -n for dry run mode, so only the file names will be printed. Remove -n from the second command to actually transfer the files.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.