30

I want to rsync multiple sources and I wonder the best way to achieve that.

e.g.

/etc/fstab
/home/user/download

I thought about 3 solutions :

  • Solution 1

multiple call to rsync

rsync -a /etc/fstab bkp
rsync -a /home/user/download bkp

con : harder to have agreggated stat

  • Solution 2

create a tobackup folder that contains symlink, and use -L options

sync -aL /home/user/tobackup bkp

con : content to backup must not contain symlinks

  • Solution 3

move files into to backup and create symlink in original location

rsync -a /home/user/tobackup bkp

con : some manual config

Which one do you recommend ?

Is there a better way ?

29

You can pass multiple source arguments.

rsync -a /etc/fstab /home/user/download bkp

This creates bkp/fstab and bkp/download, like the separate commands you gave. It may be desirable to preserve the source structure instead. To do this, use / as the source and use include-exclude rules to specify which files to copy. There are two ways to do this:

  • Explicitly include each file as well as each directory component leading to it, with /*** at the end of directories when you want to copy the whole directory tree:

    rsync -a \
        --include=/etc --include=/etc/fstab \
        --include=/home --include=/home/user --include='/home/user/download/***' \
        --exclude='*' / bkp
    
  • Include all top-level directories with /*/ (so that rsync will traverse /etc and /home when looking for files to copy) and second-level directories with /*/*/ (for /home/user), but strip away directories in which no file gets copied. This is more convenient because you don't have to list parents explicitly. You could even use --prune-empty-dirs --include='*/' instead of counting the number of levels, but this is impractical here as rsync would traverse the whole filesystem to explore directories even though none of the include rules can match anything outside /etc and /home/user/download.

    rsync -a --prune-empty-dirs \
        --include='/*/' --include='/*/*/' \
        --include=/etc/fstab \
        --include='/home/user/download/***' \
        --exclude='*' / bkp
    
  • 1
    Great, but how to keep the directory structure inside bkp? – Rodrigo Sep 28 '17 at 18:23
  • I mean, bkp/etc/fstab and bkp/home/user/dowlonad? Or maybe /bkp/fstab and bkp/download? – Rodrigo Sep 28 '17 at 18:32
  • @Rodrigo I've edited my answer – Gilles Sep 28 '17 at 18:46
8

I really like Gilles' answer, however, I'd like to add that in my view the requirement to sync multiple folders while preserving the directory structure is best met by passing multiple source arguments in conjunction with the --relative option.

In this case, we could have something as follows:

rsync -aR /etc/fstab /home/user/download bkp

which would result in bkp/etc/fstab and bkp/home/user/download.

The best part about this is that (I believe since rsync v. 2.6.7) we can in essence control how much of the directory structure we want to replicate at the receiver.
(See the documentation on the --relative option here)

So e.g. if we did this

rsync -aR /home/./user1/download /home/./user2/download bkp

we would end up with bkp/user1/download and bkp/user2/download.

1

This also works - curly braces, containing comma separated list of sources.

rsync -vap --progress --stats root@server:{/etc,/root/backups,/home/ultralazer} /mnt/bigdrive

Somewhat similar to what happens when you invoke curly braces sytnax with cp and certain other utilities:

cp -vr /etc /root/backups /home/{ultralazer,zerocool} /mnt/bigdrive

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.