I have set up two servers to synchronise using rsync daemons and cron job every 5 minutes.

  • Server A has a daemon that makes a folder available via alias.
  • Server B has a daemon that makes a folder (same path) available via alias.
  • Server A has a cron job that syncs the folder (same path) to server A's daemon alias.
  • Server B has a cron job that syncs the folder (same path) to server A's daemon alias.
  • The rsync command is given the parameters -urtzv.
  • The server clocks are synchronized.
  • The cron job runs every 5 minutes (at the 5 minute tick on clock face), so both servers would execute at the same time (approximately).

Are there any potential problems in doing this? Specifically:

  • Would there be data corruption because both servers are executing rsync at the same time? Or do the daemons intelligently handle conflicts?
  • Would there be data corruption if (for example) server A is transferring to server B for more than 5 minutes (in a single rsync run), but then server B executes rsync the other way about 5 minutes after the long-running rsync on server A was started?
  • In last case, would there be data corruption if, in addition to that, another process on server A was also writing into the file for more than 5 minutes?

I read that running rsync twice in parallel (on same server and source/destination) should be safe in terms of data integrity, though it might degrade performance. I also learnt that rsync stores the incoming data in temporary files and only moves them into the permanent files once transfer is complete. But I'm unsure of the behavior when rsync is setup on two servers to sync both ways at the same time.

If this is better asked in Server Fault, please let me know.

  • Why do you set up rsync on both for the same folder?! Usually you set it up on one server and say to synchronize both computers. Then the rsync system does it right... – Alexis Wilke Aug 22 '16 at 4:17
  • So there is potential for data corruption in my current setup? I should probably change it, but is it known to be unsafe? – ADTC Aug 22 '16 at 4:50
  • I'd be worried about rsync's temporary files getting synced back and forth, mostly. (And while we're on unwanted advice ;), lsyncd may be interesting to do, it will use inotify instead of a frequent cronjob, that might make often-changing files safer to sync.) – Ulrich Schwarz Aug 22 '16 at 5:26
  • I think the safest is to do the rsync both ways from one of the computers. Run a first time from A to B, then next from B to A. If you do not use the --delete, I do not think it will be much of a problem that way. If files can get deleted... That's more complicated! – Alexis Wilke Aug 22 '16 at 5:27
  • 1
    Dictionaries playing a joke...convoluted. – Rui F Ribeiro Aug 24 '16 at 13:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.