1

I want to be able to sync files from multiple directories to one directory. If I remove a directory from the source list, then I want that directory and all of its files to be removed from the destination.

Here's a quick sample:

mkdir dir1
touch dir1/file1
mkdir dir2
touch dir2/file2
mkdir dir3
rsync -a dir1 dir2 dir3

dir3 now contains "dir1/file1" and "dir2/file2".

If I rsync again, and remove dir2 from the list:

rsync -a dir1 dir3

Then I would like dir2/file2 and dir2 to be removed from the destination.

The --delete option to rsync only handles deletes between dir1 and dir3/dir1, so dir2 remains (even if using --delete).

I've looked through the various option in the man page for rsync, and I don't see any option that tells rsync to "combine" all the sources and consider them as "one" tree so that "--delete" would do what I want it to do. However, I'm pretty confident I'm not the first person to want to do something like this.

Ideas?

  • I think it's not possible using the syntax that you want. I think you nail it with your last paragraph. rsync doesn't "know" about dir2, so it won't delete it. – Sparhawk May 28 at 0:47
  • It's just not clear to me why rsync doesn't have an option which makes it consider all the sources as one master source. That would solve the problem. However, that may be addressed below by Freddy... – Jason K May 28 at 2:15
2

The --delete flag obviously doesn't work if you sync multiple source directories to one destination directory. Instead of syncing dir1 and dir2 to dir3 you could sync the content of the parent dir of dir1 and dir2.

Example (using rsync flag -i to see the changes):

$ mkdir -p dir12/dir1 dir12/dir2 dir3
$ touch dir12/dir1/file1 dir12/dir2/file2
$ rsync -ai dir12/ dir3
cd+++++++++ dir1/
>f+++++++++ dir1/file1
cd+++++++++ dir2/
>f+++++++++ dir2/file2
# BEFORE: dir1 and dir2 were synced to dir3
$ rm -rf dir12/dir2
$ rsync -ai --delete dir12/ dir3
*deleting   dir2/file2
*deleting   dir2/
.d..t...... ./
# AFTER: dir3/dir2 is gone

If you don't have a common parent directory, then dir12/dir2 could be a symlink pointing to the real directory. With option -L (--copy-links) in the first rsync call you would get the same directory structure in dir3 with dir2 as directory (instead of a symlink). And after deleting symlink dir12/dir2 and syncing again with --delete the directory dir3/dir2 would be gone like in the example.

EDIT:

I fiddled a bit around and may have found a hack:

$ mkdir empty dir1 dir2 dir3
$ touch dir1/file1 dir2/file2
$ rsync -ai dir1 dir2 dir3
cd+++++++++ dir1/
>f+++++++++ dir1/file1
cd+++++++++ dir2/
>f+++++++++ dir2/file2
$ rsync -ai --delete empty/ dir1 dir3
*deleting   dir2/file2
*deleting   dir2/
.d..t...... ./

Use empty/ as a placeholder for all directories you want to delete.

  • The question is unclear, but I assumed that dir1 and dir2 were not necessarily in the same directory. I thought about -L, but that potentially conflicts with -a from the question. – Sparhawk May 28 at 1:19
  • Yes, that's why I thought he could create a (fake) parent directory and use symlinks instead. For the example with symlink dir12/dir2 I used rsync -aiL dir12/ dir3 and that seems to work, but I haven't done any further testing with more complex setups. But... if OP knows that dir2 will be excluded from the next sync, it might be easier to manually rm -rf the directory on the destination side. – Freddy May 28 at 1:30
  • Yeah manually rm makes more sense to me. They have to change the rsync command anyway. – Sparhawk May 28 at 2:04
  • I think there's some promise to using -L and I will experiment with that tomorrow. Basically, I have a tool that is using rsync. One user specifies the directories that they want synced when the tool is run. A different user runs the tool that actually does the syn. The initial user can change the directory lists to be synced at any time. However, the syncing user should see the "end" result - a perfect copy with no "extra" directories. If I build a directory containing links to all the directories first, this would work. I just wish I didn't need that extra step. – Jason K May 28 at 2:22
  • As it happens, I don't think -L will work after all. After the rsync completes, the system ensures that all synced files are mode 644 and directories 755. This would mean that someone could symlink to a file they couldn't read, then after the sync, because of -L and the permission 'fixing' they would have access to the file. I'll have to think this through some more. – Jason K May 28 at 3:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.