2

I want to back up my /home directory with rsync. I have read rsync's man page and decided to use filter rules for this task.

What I would like to achieve: Exclude all files and directories in the Repos directory but keep all pull_all.sh files and output directories --- regardless where they are located within the Repos directory.

So far, I have ended up with following filter list, but this backs up only the pull_all.sh files but not the output directories:

# Files prefixed with "+ " are included. Files prefixed with "- " are excluded.
#
# The order of included and excluded files matters! For instance, if a folder
# is excluded first, no subdirectory can be included anymore. Therefore,
# mention included files first. Then, mention excluded files.
#
# See section "FILTER RULES" of rsync manual for more details.


# Included Files

# TODO: This rules do not work properly!
+ output/***
+ pull_all.sh
- Repos/**

# Excluded Files

- .android
- .cache
...

I use the filter list in my script run_rsync.sh:

#!/bin/bash

date="$(date +%Y-%m-%d)"
hostname="$(hostname)"

# debug_mode="" # to disable debug mode
debug_mode="--list-only"

# Note: With trailing "/" at source directory, source directory is not created at destination.
rsync ${debug_mode} --archive --delete --human-readable --filter="merge ${hostname}.rsync.filters" --log-file=logfiles/$date-$hostname-home.log --verbose /home backup/

Unfortunately, the existing StackExchange threads have not solved my problems:

What's going wrong here?

[Update] Here is an example how the home directory looks like and which files to keep and which files to ignore:

user@hostname:~$ tree /home/ | head
/home/
└── user
    ├── Desktop                -> keep this
    │   ├── file1              -> keep this
    │   └── file2              -> keep this
    ├── Documents              -> keep this
    ├── Repos
    │   ├── pull_all.sh        -> keep this
        ├── subdir1
        │   ├── output         -> keep this
        ├── subdir2
            ├── another_subdir
                ├── output     -> keep this
        ├── subdir3            -> do not keep (because does not contain any "output")
        ├── file3              -> do not keep
1

Slightly restating what I've interpreted as your requirements,

  • Include all pull_all.sh files regardless of where we find them
  • Include all output directories and their contents regardless of where we find them
  • Exclude the Repos directory, other than what we have already stated
  • Include everything else

This can be specified as follows

rsync --dry-run --prune-empty-dirs -av

    --include 'pull_all.sh'
    --include 'Repos/**/output/***'

    --include '*/'

    --exclude 'Repos/***'

    /home backup/

Some notes

  • The --include '*/' is required so that rsync will consider heading down into the Repos directory tree (to look for pull_all.sh files), which would otherwise be excluded by the final --exclude statement.
  • The three different uses of * are different:
    • * matches anything except / characters
    • ** matches anything including / characters
    • dir/*** is a shortcut equivalent to specifying dir/ and dir/**.
  • The --prune-empty-dirs flag stops rsync creating empty directories, which is particularly important as we need to process the Repos directory tree looking for pull_all.sh and output items.
  • Remove --dry-run when you are happy with the results.
  • I have updated my answer to address your revised question. – roaima Mar 1 at 12:07
  • Thanks a lot. :) The updated answer works like a charm. I guess the key was to add --include '*/'. Otherwise, rsync would not have traversed the Repos directory. – Schwefelsaeure Mar 4 at 6:33
  • Yes, that's right. I've added some notes that I hope make it clearer what's going on. – roaima Mar 4 at 9:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.