I think rsync is the wrong tool, as are find and mv. My recommendation is instead to make use of a software configuration management system. These include Subversion, Git, Mercurial, Bazaar among others. All of which can easily handle changes of tree structure.
In the structure you describe, you have tree structure A on your client system and a repository tree structure B on a secondary location, possibly local, possibly remote.
Where you have updates to both, you now have competing changes which have to be applied in the correct order for your two repositories to remain consistent. If they are applied out of order you get into the situation you are in now, where changes in one cannot be applied directly to the other because the structures no longer exist.
As it stands, there are no options for rsync which can allow it to automatically know which changes have to be applied to bring both repositories to consistency. That isn't what it's designed to do. It can certainly make one repository look exactly like the other but it requires that only one side is changed at a time. Example being alternating changes to A, then B, then A. At any point, you need to designate one of the structures A or B as the master and sync changes in only one direction at a time.
I also don't believe there's a simple commandline-fu command which will achieve the result you're looking for, so now you're into the realm of shell programming.
Where you have changes only to the tree structure of B and only to the file contents of A, it's a relatively simple task to look for the file names of the changed files and acquire the new paths of those files in B, then modify the tree structure of A to match. This only works if the file names are unique.
The pseudo code which would bring the structure of A into consistency with B would look something like:
generate list of file names in A and their paths
For each of the names in A
find that same name in B
If the path of A is the same as B
continue to the next file
if not then
create the directory structure in A
move the file to the new location.
if the old path in A is now empty
delete the directory.
repeat
check if the parent directory is now empty, then delete it.
until a non empty directory
Once the tree structures are in sync, then A can be copied directly to exactly the same paths in B. The --update option of rsync can be used to overwrite older files with newer ones in both directions.
Some example shell code to copy locally changed files into the existing repository, using find as the file name selector.
#!/bin/bash
set -xv
localRepo=/tmp/a
remoteRepo=/tmp/b
rm -rf $localRepo $remoteRepo
mkdir -p $localRepo/1/2/ $localRepo/1/3/
mkdir -p $remoteRepo/2/1/ $remoteRepo/3/1/
echo a12 > $localRepo/1/2/file
echo b21 > $remoteRepo/2/1/file
echo a13 > $localRepo/1/3/file1
echo b31 > $remoteRepo/3/1/file1
echo ex1
cat $localRepo/1/2/file $remoteRepo/2/1/file
echo ex2
cat $localRepo/1/3/file1 $remoteRepo/3/1/file1
localFileNameList=$(find $localRepo -type f -mtime -1 | xargs -L 1 basename)
for localFileName in $localFileNameList
do
localFilePath=$(find $localRepo -name $localFileName | xargs dirname)
backFile=$(find $remoteRepo -name $localFileName)
repoDir=$(dirname $backFile)
cp $localFilePath/$localFileName $repoDir
done
echo ex1
cat $localRepo/1/2/file $remoteRepo/2/1/file
echo ex2
cat $localRepo/1/3/file1 $remoteRepo/3/1/file1
To import a file system into subversion for example as one of the simpler to use SCMs:
e.g.
mkdir /tmp/svn
svnadmin create /tmp/svn/reponame
cd /tmp/b
svn import -m "The initial import " file:///tmp/svn/reponame
Adding 2
Adding 2/1
Adding 2/1/file
Adding 3
Adding 3/1
Adding 3/1/file1
Then check the repo out and make local changes.
$ cd /tmp
$ svn checkout file:///tmp/svn/reponame
A reponame/2
A reponame/2/1
A reponame/2/1/file
A reponame/3
A reponame/3/1
A reponame/3/1/file1
Checked out revision 1.
/tmp:
$ cd reponame/
/tmp/reponame:
$ ls -ltr
total 8
drwxrwxr-x 3 css1971 css1971 4096 Apr 11 12:04 3
drwxrwxr-x 3 css1971 css1971 4096 Apr 11 12:04 2
/tmp/reponame:
$ svn move 3 4
A 4
D 3
D 3/1
D 3/1/file1
/tmp/reponame:
Commit the changes back to the repo.
$ svn commit -m "renamed dir"
Deleting 3
Adding 4
Committed revision 2.
From this point, use the svn tool as part of your normal workflow to manipulate the repository.
Useful commands:
svn import
svn update
svn commit
svn del
svn cp
svn mv
Command reference:
http://svnbook.red-bean.com/en/1.7/svn.ref.html
./flat/
which holds a copy of all the files, but in one flat list. The copies would be hard links, so take no space. You do the same on the local machine. You can then rsync from the local flat dir to the remote flat dir. This will update all the remote files as rsync preserves remote hard links if you use--inplace
. The flat dirs could be created by script just before the rsync and removed after.git
or other version control software.