2

From the docs for rsync's -L flag:

-L, --copy-links    transform symlink into referent file/dir

This question is about how to "reverse" a transfer that was originally done using rsync -L ....


It's easiest for me to explain what I mean here by "reverse" with a simple concrete example.

The setup

Suppose I have directories /tmp/SOURCE/ and /tmp/TARGET/, as shown below:

$ /usr/bin/tree -aF /tmp/SOURCE/ /tmp/TARGET/
/tmp/SOURCE/
├── A.d/
│   ├── w
│   └── x
├── B.d/
│   └── z
├── C.d/
│   ├── w -> ../A.d/w
│   └── y -> ../A.d/x
└── D.l -> B.d/
/tmp/TARGET/

In particular, note the following details:

  • the contents of /tmp/SOURCE/C.d are relative symlinks pointing to regular files in /tmp/SOURCE/A.d;
    • the basenames of symlink /tmp/SOURCE/C.d/w and of its referent are the same;
    • the basenames of symlink /tmp/SOURCE/C.d/y (namely y) and of its referent (namely, x) are different;
  • /tmp/SOURCE/D.l is a relative symbolic link to /tmp/SOURCE/B.d;
  • /tmp/TARGET/ is currently empty.

From this initial state, I run the sort of commmand mentioned in this post's title, the sort of command I want to "reverse" (as explained further below):

$ rsync -L -a /tmp/SOURCE/C.d /tmp/SOURCE/D.l /tmp/TARGET

After I run this command, /tmp/TARGET/ looks like this:

$ /usr/bin/tree -aF /tmp/TARGET/
/tmp/TARGET/
├── C.d/
│   ├── w
│   └── y
└── D.l/
    └── z

Note in particular that, although under both /tmp/SOURCE and /tmp/TARGET the relative paths

C.d/w
C.d/y
D.l/z

exist, only those under /tmp/TARGET are "real" paths. By a "real path" $P I mean that, roughly speaking, the output of readlink -f $P is again $P (or, more precisely,${P:a}). E.g.

$ readlink -f /tmp/TARGET/C.d/w
/tmp/TARGET/C.d/w

but

$ readlink -f /tmp/SOURCE/C.d/w
/tmp/SOURCE/A.d/w

The problem

In the current context, what I mean by "reversing" the previous rsync -L command is basically to copy the regular files under /tmp/TARGET/ onto their originals under /tmp/SOURCE, leaving the structure of /tmp/SOURCE unchanged.

In particular, after this "reverse transfer", the symlinks under /tmp/SOURCE should remain symlinks, and should point to exactly the same referents as they did to before.

I'm looking for a way to carry out such "reversal" that is comparable in complexity/difficulty to the rsync -aL command used to perform the "forward" transfer in the first place.


One shortcoming of this simple example is that it is so simple that one is tempted to solve the problem with brute force. E.g.

$ rsync /tmp/TARGET/C.d/w /tmp/SOURCE/A.d
$ rsync /tmp/TARGET/C.d/y /tmp/SOURCE/A.d/x
$ rsync /tmp/TARGET/D.l/z /tmp/SOURCE/B.d

IOW, in such a simple situation, this brute force approach is not very much more difficult than it was to run the original rsync -L transfer. In general, however, the brute force strategy can be vastly more difficult to carry out, when compared to the original transfer.

This example is oversimplified also because the source and target hosts are the same. In general, they won't be, so please avoid solutions that rely on this feature of the example.


FWIW, the following script creates the set-up of the toy example above.

BASEDIR=/tmp/test1

# uncomment the following line before running this script a second time
# rm -rf $BASEDIR/SOURCE $BASEDIR/TARGET

mkdir -p $BASEDIR/SOURCE/{A,B,C}.d $BASEDIR/TARGET
touch $BASEDIR/SOURCE/{A.d/{w,x},B.d/z}
ln -s ../A.d/w $BASEDIR/SOURCE/C.d/w
ln -s ../A.d/x $BASEDIR/SOURCE/C.d/y
ln -s B.d $BASEDIR/SOURCE/D.l
1

There are options to allow rsync to not disturb symbolic links on the remote when they refer to directories, but it doesn't seem to have an option to not replace symbolic links to files by real files.

So what you might do is initially preserve the symbolic links to files in SOURCE in some way, eg by renaming them or listing them into a file, then do the rsync preserving directory symbolic links, then restore the file symbolic links, copying the data from the real file to the restored symbolic link and then removing the real file. Here's a potential script:

find /tmp/SOURCE -type l ! -xtype d -exec mv {} {}.lnk \;
cd /tmp/TARGET || exit
rsync -a -R --no-implied-dirs --keep-dirlinks . /tmp/SOURCE/
find /tmp/SOURCE -type l ! -xtype d |
while read fname
do base=${fname%.lnk}
   cp "$base" "$fname" 
   mv "$fname" "$base"
done

I leave you to cope with weird filenames properly. I'm not sure that the -R --no-implied-dirs is necessary, but I'll let you experiment with that.

  • Thanks, that worked nicely. In my informal tests, I found that the -R --no-implied-dirs was not necessary. Also unnecessary is the cd to /tmp/TARGET. Instead, I just replaced the . argument in the rsync command with /tmp/TARGET/. This is better anyway, since, in general, the (original) target directory may be on a remote host. – kjo Jun 26 '17 at 13:59
1

I don't think you can do this directly within rsync. The --keep-dirlinks (-K) option will keep symlinked directories untouched, but symlinked files will still be overwritten.

The closest solution I can find for this, is a little (bash) script

( cd TARGET && find . -type f -print0 | sed 's!^./!!' ) |
    while IFS= read -d $'\0' -r f
    do
        s=$(readlink -f SOURCE/"$f")
        rsync -a TARGET/"$f" "${s:-f}"
    done

This takes each file in TARGET one at a time, checks for a symlink target in SOURCE and applies rsync to that directly.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.