1

I have a setup where I use rsync to mirror some source directory to a (remote) destination except for some files and locations defined witch rsync exclude patterns:

# Useless files
- thumbs.db
- *.~
- *.tmp
- /*/.cache
- /*/.local/share/trash
# Already rsynced somewhere
- .dropbox
# Medias files
- *.avi
- *.mkv
- *.wav
- *.mp3
- *.bmp
- *.jpg
# System files
- /hiberfil.sys
- /pagefile.sys

This script runs on Windows and Linux stations and uses the --delete parameter to delete extraneous files from dest dirs.

The thing is, when I "update" the exclude patterns (eg. add an exclusion pattern for Ogg files: *.ogg) I have to re-run the rsync to remove any existing Ogg file from the destination.

I was wondering if I could easily apply the exclude rules (some can be complex because it uses wildcards) on a given directory so that I don't have to re-run the rsync from the source to clean my destination directory.

So far I've come to the following which deletes files and directories using basename matching:

dirToClean="/var/somedir"
excludeFile="file_to_exclude"

# Delete files and dir of $dirToClean whose basename matches an exclude pattern from $excludeFile
for fileToDelete in $(grep --extended-regexp "^- .*" $excludeFile | sed 's/^- \(.*\)$/\1/'); do
    find "$dirToClean" -iname "$fileToDelete" -exec rm {} \;
done

Maybe I could run rsync using the same directory as source and destination?

2 Answers 2

0

Don't do this. It is a very bad idea to try to replicate what rsync include/exclude patterns do without using rsync. The fact that some of the patterns can be complex is even more reason not to attempt it. Use rsync itself to guarantee consistent behavior and minimize surprises.

From the rsync manual page:

--existing, --ignore-non-existing

          This  tells rsync to skip creating files (including directories)
          that do not exist yet on the destination.   If  this  option  is
          combined  with  the  --ignore-existing  option, no files will be
          updated (which can be useful if all you want to do is to  delete
          extraneous files).

So you should run with --delete --delete-excluded --ignore-existing --ignore-non-existing and rsync will remove extraneous files and not update or remove any other files.

3
  • I was very aware mimicking rsync would be a bad idea ;) So you're saying I can clean my /var/somedir with run rsync --delete --delete-excluded --ignore-existing --ignore-non-existing --exclude-from="file_to_exclude" /dev/null /var/somedir?
    – CDuv
    Aug 28, 2014 at 8:00
  • Actually, both rsync --include-from=file_to_exclude --recursive --ignore-existing --ignore-non-existing --delete --delete-excluded /var/somedir /var/somedir and rsync --include-from=file_to_exclude --recursive --delete-excluded /var/somedir /var/somedir works... So it seems "--ignore-existing --ignore-non-existing" and "--delete" are not needed.
    – CDuv
    Aug 30, 2014 at 12:43
  • Write it up as an answer, explain why your way works better, gain some rep.
    – Kyle Jones
    Aug 30, 2014 at 19:50
0

Following Kyle Jones advice to use available rsync options to do the job (instead of coding one) I've found

rsync --include-from=file_to_exclude --recursive \
--delete-excluded \
/var/somedir/ /var/somedir/

to work just fine. I've also tried using --ignore-existing --ignore-non-existing --delete on multiple scenarios but the result was the same as without theses 3 options.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .