3

I want to back up my /home directory with rsync. I have read rsync's man page and decided to use filter rules for this task.

What I would like to achieve: Exclude all files and directories in the Repos directory but keep all pull_all.sh files and output directories --- regardless where they are located within the Repos directory.

So far, I have ended up with following filter list, but this backs up only the pull_all.sh files but not the output directories:

# Files prefixed with "+ " are included. Files prefixed with "- " are excluded.
#
# The order of included and excluded files matters! For instance, if a folder
# is excluded first, no subdirectory can be included anymore. Therefore,
# mention included files first. Then, mention excluded files.
#
# See section "FILTER RULES" of rsync manual for more details.


# Included Files

# TODO: This rules do not work properly!
+ output/***
+ pull_all.sh
- Repos/**

# Excluded Files

- .android
- .cache
...

I use the filter list in my script run_rsync.sh:

#!/bin/bash

date="$(date +%Y-%m-%d)"
hostname="$(hostname)"

# debug_mode="" # to disable debug mode
debug_mode="--list-only"

# Note: With trailing "/" at source directory, source directory is not created at destination.
rsync ${debug_mode} --archive --delete --human-readable --filter="merge ${hostname}.rsync.filters" --log-file=logfiles/$date-$hostname-home.log --verbose /home backup/

Unfortunately, the existing StackExchange threads have not solved my problems:

What's going wrong here?

[Update] Here is an example how the home directory looks like and which files to keep and which files to ignore:

user@hostname:~$ tree /home/ | head
/home/
└── user
    ├── Desktop                -> keep this
    │   ├── file1              -> keep this
    │   └── file2              -> keep this
    ├── Documents              -> keep this
    ├── Repos
    │   ├── pull_all.sh        -> keep this
        ├── subdir1
        │   ├── output         -> keep this
        ├── subdir2
            ├── another_subdir
                ├── output     -> keep this
        ├── subdir3            -> do not keep (because does not contain any "output")
        ├── file3              -> do not keep
1

2 Answers 2

5

Slightly restating what I've interpreted as your requirements,

  • Include all pull_all.sh files regardless of where we find them
  • Include all output directories and their contents regardless of where we find them
  • Exclude the Repos directory, other than what we have already stated
  • Include everything else

This can be specified as follows

rsync --dry-run --prune-empty-dirs -av

    --include 'pull_all.sh'
    --include 'Repos/**/output/***'

    --include '*/'

    --exclude 'Repos/***'

    /home backup/

Some notes

  • The --include '*/' is required so that rsync will consider heading down into the Repos directory tree (to look for pull_all.sh files), which would otherwise be excluded by the final --exclude statement.
  • The three different uses of * are different:
    • * matches anything except / characters
    • ** matches anything including / characters
    • dir/*** is a shortcut equivalent to specifying dir/ and dir/**.
  • The --prune-empty-dirs flag stops rsync creating empty directories, which is particularly important as we need to process the Repos directory tree looking for pull_all.sh and output items.
  • Remove --dry-run when you are happy with the results.
3
  • I have updated my answer to address your revised question. Commented Mar 1, 2019 at 12:07
  • Thanks a lot. :) The updated answer works like a charm. I guess the key was to add --include '*/'. Otherwise, rsync would not have traversed the Repos directory. Commented Mar 4, 2019 at 6:33
  • Yes, that's right. I've added some notes that I hope make it clearer what's going on. Commented Mar 4, 2019 at 9:32
0

You can use "+" for the wanted sub directory and "-" for the rest:

rsync -auv "${src}" "${dst}" \
--filter=+_"/parent-dir/to-sync-child-dir/" \
--filter=-_"/parent-dir/*"
1
  • 1
    The OP already had + and - rules in their question, using a filter rules file rather than arguments on the command. The problem was that they couldn't get a combination to achieve their required set of files to be copied. I don't see how a generic answer with src, dst, parent_dir, etc. addresses the question that was asked. You would need to rewrite your answer to show how to apply filter rules to the situation Commented Jul 27, 2023 at 7:10

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .