1

I am trying to use rsync to copy a specific set of subdirectories to back them up. My directory structure looks something like this:

/path/to/data/foo1
/path/to/data/foo2
/path/to/data/bar1
/path/to/data/bar2

I just want to copy the 'foo' folders, ignoring the 'bar' ones.

I also want to ignore any .zip files inside those folders.

I have tried many combinations of arguments in rsync, but can't get it to do exactly what I want.

I can do the transfer as I want to using two commands:

rsync --dry-run -a --human-readable --stats --progress --exclude"*.zip" "/path/to/data/foo1" "/mnt/backup/drive/"
rsync --dry-run -a --human-readable --stats --progress --exclude"*.zip" "/path/to/data/foo2" "/mnt/backup/drive"

Form reading elsewhere, I expect something like this to work

rsync --dry-run -a --human-readable --stats --progress --include="data/foo*" --exclude="*.zip" --exclude="*" "/path/to/data" "/mnt/backup/drive"

But this also seems to capture the 'bar' folders.

The ordering of the --include and --exclude arguments is confusing me. As is the need to specify the 'data' folder in the --include argument is not the functionality I would expect.

What is the obvious flag that I'm missing!

Thanks for your input.

Ben

1

It will work for you to run rsync in this way:

rsync --dry-run -av --human-readable --stats --progress --exclude="*.zip" /path/to/data/foo* /path/to/copy/to/

perhaps you'd like to shorter the command, in this way:

rsync  -avhn --stats --progress --exclude="*.zip" /path/to/data/foo* /path/to/copy/to/

quoting from man rsync

-a, --archive archive mode; equals -rlptgoD (no -H,-A,-X)

-v, --verbose increase verbosity

-h, --human-readable output numbers in a human-readable format

-n, --dry-run perform a trial run with no changes made

when you are ready run the command without -n

it's also possible to save a logfile with this argument :

--log-file=

so the rsync command above will become:

rsync  -avhn --stats --progress --exclude="*.zip" --log-file=logfile.log /path/to/data/foo* /path/to/copy/to/

the logfile will be written in the same directory where you run the command, if you need to save it in another location, you must use an absolute path. for example:

--log-file=/path/to/logfile

0

You've already got a suitable answer, and that's fine. I wanted to address the other part of your question that asks why the --include functionality doesn't do what you expect.

First of all, let's look at how the --include and --exclude filters operate.

  • aaa will apply to any file or directory named aaa
  • /aaa will apply to any file or directory named aaa at the top of the source path
  • aaa/ will apply to any directory called aaa

You can combine these, so a path of bbb/ccc/ indicates any directory ccc that is a child of a directory bbb anywhere in the source tree, but /bbb/ccc/ is a directory ccc inside a directory bbb that is tied to the top of the source tree.

Include and exclude operations are processed from left to right (the first operation is more important that the second, and the second is more important than the third).

Now let's look at your specific example:

rsync -ah --include="data/foo*" --exclude="*.zip" --exclude="*" "/path/to/data" "/mnt/backup/drive"

The filter rules say, in order:

  1. Include all files or directories whose name begins with foo that are in a directory called data somewhere below the source path.
  2. Exclude all *.zip files (or directories).
  3. Exclude everything we haven't already mentioned.

The first include would match /path/to/data/data/foo1 but it would not match /path/to/data/foo1. I suspect this is the primary misunderstanding.

You can match your /foo* directories with a very similar solution:

rsync -ah --include="/foo*/" --exclude="*.zip" --exclude="*" "/path/to/data/" "/mnt/backup/drive"

But actually, if I were writing this I would probably include the /foo* directories in the source path specification (notice the foo* is unquoted so the shell can get at the wildcard) and only worry about excluding the unwanted zip files:

rsync -ah --exclude="*.zip" /path/to/data/foo*/ /mnt/backup/drive

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.