rsync: Use filters to exclude top-level directory but include some of its subdirectories

Question

I want to back up my /home directory with rsync. I have read rsync's man page and decided to use filter rules for this task.

What I would like to achieve: Exclude all files and directories in the Repos directory but keep all pull_all.sh files and output directories --- regardless where they are located within the Repos directory.

So far, I have ended up with following filter list, but this backs up only the pull_all.sh files but not the output directories:

# Files prefixed with "+ " are included. Files prefixed with "- " are excluded.
#
# The order of included and excluded files matters! For instance, if a folder
# is excluded first, no subdirectory can be included anymore. Therefore,
# mention included files first. Then, mention excluded files.
#
# See section "FILTER RULES" of rsync manual for more details.


# Included Files

# TODO: This rules do not work properly!
+ output/***
+ pull_all.sh
- Repos/**

# Excluded Files

- .android
- .cache
...

I use the filter list in my script run_rsync.sh:

#!/bin/bash

date="$(date +%Y-%m-%d)"
hostname="$(hostname)"

# debug_mode="" # to disable debug mode
debug_mode="--list-only"

# Note: With trailing "/" at source directory, source directory is not created at destination.
rsync ${debug_mode} --archive --delete --human-readable --filter="merge ${hostname}.rsync.filters" --log-file=logfiles/$date-$hostname-home.log --verbose /home backup/

Unfortunately, the existing StackExchange threads have not solved my problems:

• https://stackoverflow.com/questions/8270519/rsync-exclude-a-directory-but-include-a-subdirectory
• Using Rsync include and exclude options to include directory and subdirectory but exlude files in subdirectory

What's going wrong here?

[Update] Here is an example how the home directory looks like and which files to keep and which files to ignore:

user@hostname:~$ tree /home/ | head
/home/
└── user
    ├── Desktop                -> keep this
    │   ├── file1              -> keep this
    │   └── file2              -> keep this
    ├── Documents              -> keep this
    ├── Repos
    │   ├── pull_all.sh        -> keep this
        ├── subdir1
        │   ├── output         -> keep this
        ├── subdir2
            ├── another_subdir
                ├── output     -> keep this
        ├── subdir3            -> do not keep (because does not contain any "output")
        ├── file3              -> do not keep

Answer 1

Slightly restating what I've interpreted as your requirements,

• Include all pull_all.sh files regardless of where we find them
• Include all output directories and their contents regardless of where we find them
• Exclude the Repos directory, other than what we have already stated
• Include everything else

This can be specified as follows

rsync --dry-run --prune-empty-dirs -av

    --include 'pull_all.sh'
    --include 'Repos/**/output/***'

    --include '*/'

    --exclude 'Repos/***'

    /home backup/

Some notes

• The --include '*/' is required so that rsync will consider heading down into the Repos directory tree (to look for pull_all.sh files), which would otherwise be excluded by the final --exclude statement.
• The three different uses of * are different:
  • * matches anything except / characters
  • ** matches anything including / characters
  • dir/*** is a shortcut equivalent to specifying dir/ and dir/**.
• The --prune-empty-dirs flag stops rsync creating empty directories, which is particularly important as we need to process the Repos directory tree looking for pull_all.sh and output items.
• Remove --dry-run when you are happy with the results.

Answer 2

You can use "+" for the wanted sub directory and "-" for the rest:

rsync -auv "${src}" "${dst}" \
--filter=+_"/parent-dir/to-sync-child-dir/" \
--filter=-_"/parent-dir/*"