1

I want rsync to copy everything from "copy_to_home" directory (which is on a NTFS-formatted hard drive) to the users home directory. I don't want it to delete anying, but it should replace files on the receiving side if they're not the same as the files on the sending side.

This is what the command looks like:

rsync --modify-window=1 -hh --progress -v -r copy_to_home/ ~/

My problem is that whenever executing this command, rsync always seems to replace every single file in ~ despite the files having not been changed. The --update option wouldn't do that, but it doesn't replace modified files on the receiving side.

2

If Quora Feans' answer does not help, you can add the option -i or --itemize-changes to get rsync to explain why it is updating the files. It prints a string formed of chars YXcstpoguax where c, for example, means the checksum differs. You will probably find codes p permissions differ or o owner differs. These are usually fixed by using the -a option to preserve such attributes. If you still see diffs, try using --size-only to only have files updated if their size differs (ignoring the timestamp).

  • Nice! Your answer (--itemize-changes) helped me find out that I just had to add --times (or -t)! — From the man page: "a missing -t or -a will cause the next transfer to behave as if it used -I, causing all files to be updated" – Joschua Dec 27 '15 at 17:05
2

What you need is archiving the files. That is, the -a option. Dry run it as:

sudo rsync -avr --dry-run --delete <sourcedir/> <targetdir>

And, for real:

sudo rsync -avr --delete <sourcedir/> <targetdir>

Notice the / at the end of sourcedir.

  • You're right, although I didn't want to delete on the receiving side and --times seems to be enough. – Joschua Dec 27 '15 at 17:09
1

Poor reliability of matching timestamps whenever Windows formatted file systems are involved, can cause files to be constantly misunderstood as being different and thus rsync ends up copying everything. One workaround to this poor timestamp reliability when it comes to Windows formatted file systems, is to not compare on timestamp at all, but compare on contents. Rsync has the -c option, so in your command prompt you could try:

$ rsync -c --progress -v -r copy_to_home/ ~/

Explanation

  • without -c or --checksum, rsync normally determines whether to update an item or not by comparing file size as well as modification times, which, as you have seen, is not as reliable when it involves files and directories from Windows NTFS despite any attempts to get it to work better as you have done with --modify-window=1
  • if we only omit timestamp comparison while comparing size, there can still be files that will have the same file size before and after a change and thus would be missed if rsync merely compares file sizes
  • specifying -c makes rsync run checksums on both source files and destination files to make the comparison and decision. So in this manner, with checksums, rsync is deciding whether to transfer a file or not based on its actual contents, not its modification times or sizes, thus ensuring nothing is missed nor unnecessarily copied

Disadvantages

  • the larger the files the slower it runs, so if you are using rsync with gigabyte-sized files of course it would be slow. It would however be an improvement over rsync unnecessarily copying gigabyte sized files, if that is your current situation
  • otherwise if you have typically just small textual configuration files in your home directory that you wish to rsync, then for modern computers, the delay using checksum may not even be noticeable
  • or if you do have gigabyte-sized files and this process is slow, then use cron to schedule the command to run during off-peak times

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.