rsync only missing directories

Question

Is there a way to tell rsync to sync only source directories that are missing on a destination (whole directories, not just any missing file)?

Considering S as the set of source directories, and D the set of destination directories, I want to:

• copy the entire directory s in S if s not in D.
• skip entirely the directory s (indep. of the files it contains) if s in D.

Of course it's possible to list the directories on both sides, rsync the list from the destination, do some perl and generate a list of all directories that need to be copied, but it would be better if it were possible with just one invocation of rsync.

For example, if source and destination are on the same server, one could do:

src=/some/where
dst=/else/where
cd /tmp
(cd $src; find . -type d) | sort > a
(cd $dst; find . -type d) | sort > b
comm -23 a b | perl -e 'L: while(<>) {chomp; $p=$_; while ($p=~s,/[^/]+$,,) { next L if $n{$p}; } $n{$_}++; s,^./,/,; print "$_/***\n"}' > tocopy
rsync -vmazn --include-from tocopy --exclude '*' $src/ $dst/

(without the -n if for real).

PS: note that the perl "one-liner" above (more or less) strips subdirs of dirs that we decide to copy (as those subdirs are subsumed in the copy of their parent). That recipe ends up with the minimal set of dirs to copy.

Answer 1

Whatever you want, I can't think of a way to do this with rsync alone.

If you truly want to entirely skip directories that exist on the destination side, then the test isn't recursive, only the toplevel directory or directories matter. For each argument to copy (dirN in rsync --flag-that-you-hope-exists dir1 dir2 dir3 destination/), either the directory exists on the target side and there is nothing to do, or the source directory doesn't exist and you want to copy it whole.

cd /path/to/source
for x in dir1 dir2 dir3; do
  [ -e "/path/to/target/$x" ] || cp -Rp -- "$x" /path/to/target/
done

If you actually want to recurse into directories that exist on the target side, and only copy files when a directory is missing, here's a method using find to traverse the source directory and check each corresponding target directory.

cd /path/to/source
find . -type d -exec sh -c '
  if [ -e "$0/$1" ]; then exit 0; fi
  cp -Rp -- "$1" "$0/$1"
  exit 1
' /path/to/target {} \; -prune

If the source or the target is remote, mount it with SSHFS.

Answer 2

I was looking for a solution for almost exactly the same problem, but I needed only the first level directories to be checked. I.e. I wanted to copy contents from srcDir to dstDir skipping the directories from srcDir which already exist in dstDir (without checking their date, size or content). I was surprised that rsync aparently does not have a way to do it. So I came up with the following solution.

cd "$srcDir"
for f in *
do
    if [[ -d "$f" ]]
    then
        if [[ ! -d "$dstDir/$f" ]]
        then
            cp -a "$f" "$dstDir"
        fi
    fi
done
cd ..