1

I'm encountering some file corruption when using rsync (that is, the file contents do not match the source file). I can understand the file becoming corrupt if it is being written to while being synced, but I don't think it is being written to. Could corruption occur in these other cases?

1. The file is being synced by multiple rsync processes concurrently.

This seems like an unnecessary and possibly dangerous thing to do. But let's suppose it's necessary. I imagine that the two rsync processes sync the source to two separate temporary files. When each is done, it performs an atomic rename to replace the destination file with the temporary. I don't see how this could cause corruption. At worst, I can see the earlier rsync finishing last and causing the destination file to be synced to a potentially older version of the file.

2. The source file is moved/renamed onto during syncing.

That is, another file is renamed to have the same name as the file currently being synced from. Supposing the rsync process on the source end opens the source file once and only once, I don't see how this can cause corruption. When a file is renamed to take the place of the source, the file that rsync has open doesn't change. It should continue operating on the original, unmodified (assuming no process is writing to it) source.

I'm probably making incorrect assumptions about how rsync works. Feel free to correct any mistakes.

  • rsync is capable of a many different overwriting strategies, which one are you using? – Jasen Apr 14 '15 at 9:21
0

What exactly is the question here?

You should add some detail on your invocation pattern of rsync to get any meaningful assistance. I've used rsync for years without problems. I think it's safe to say that any apparent corruption is due to improper usage of this versatile tool. If you don't pass -c, rsync skips based on modification time and size. Even if you pass -c, rsync won't update if you also passed -u and the file is newer on the receiver. If you pass --inplace, rsync updates destination files in-place. I'm just giving these as examples to show that there are a lot of interactions between the options of rsync and you really need to add details about how you are invoking rsync.

I can understand the file becoming corrupt if it is being written to while being synced, but I don't think it is being written to.

Rather than guessing, how about running inotify (e.g., on the containing directory of the file) to determine for sure that the file is only updated by one process at any time?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.