1

There are two RHEL 7.2 linux servers located in different places. Both have same directory structure. Requirement is to keep certain directories of both servers in sync, i.e any modification in server1 should get reflected in server2 and vise versa, but if some file gets deleted locally it should not get deleted from remote server also. If some files get modified while link between those two servers is down, it should be copied as soon as link gets established.To implement this following script is made :

#!/bin/bash
EVENTS="CREATE,MOVED_TO,MODIFY"
inotifywait -e "$EVENTS" -m -r --format '%w%f' --fromfile list.txt|
while read FILE; do
    echo $FILE
    returnvalue=1
    while [[ $returnvalue -ne 0  ]]
    do
        rsync -azr $FILE backupserver:/$FILE
        returnvalue=$?
        echo $returnvalue
        if [[ $returnvalue -ne 0 ]]
        then
            sleep 60
        fi
   done
done

list.txt contains all the directories to be monitored. This is script is running in both the servers.

Problem : Whenever some modification is made in server1, it detects and copies it to server2 with return code 0. But that is detected in server2 as new modification and it tries to copy back to server1, as that file is present in server1 rsync returns an error code 23, so programs gets stuck.

What is the best solution to this problem?

Note: We cannot use --ignore-existing option as same file exists in both places which needs to be updated if content gets changed.

  • IMHO the best solution is to use NFS (or any other shared filesystem) – Romeo Ninov Nov 25 '16 at 12:40
  • That was thought as an option but as places are far away link goes down frequently causing stale file system. – Sushovan Nov 28 '16 at 4:19
0

** first thought this might be a rsync option issue ** I think differently now!

Answer: use a local lock to claim the right to make a copy

I would suggest a simple test, if you do not have one, of one directory, - and first - just use the defaults, and see where that does not work as desired.

** deleted rsync stuff **

OK - backup 4 steps and look again.

Maybe the problem is the loop logic:

  • ServerA has a change (to a LARGEFILE), it starts to copy it.
  • Now, on ServerB, LARGEFILE is new/modified DURING the rsync/rcp copy, and tries to copy it back 'immediately'

I suspect this 'immediacy' lies at the root cause of problem.

My suggestion is to add a "LOCK" file of some type:

  • place the lock locally
  • check for lock remote, if found at remote, remove local lock and ignore local inotify
  • if not found at remote, start copy of file, remove lock when copy finshes
  • i suspect you need rcp/scp as the actual 'copy mechanism' as this seems to be for individual files, not directories or reposistories that could be setup as an rsync "area" – Michael Felt Nov 25 '16 at 13:43
0

You might replace your wait on MODIFY events by instead waiting on close_write ( file closed after being opened writeable), which will ensure you do not try to rsync files that are still changing.

0

The problem was due to temporary file creation as suggested in other answers. It got resolved when I added --temp-dir=/tmp as an rsync option.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.