rsync keep specific folder with its contents and the structure, exclude any files outside the folder

Question

my file system looks like:

source/

folder1/
  subfolder/
    contents
  other_subfolders
  file1
  file2
folder2/
  subfolder/
    contents
  other_subfolders
  file1
  file2
folder3/
  subfolder/
    contents
  other_subfolders
  file1
  file2
file1
file2

What I want to copy is pretty much only this:

folder1/
  subfolder/
    contents
folder2/
  subfolder/
    contents
folder3/
  subfolder/
    contents

i.e. I only want to keep

• folder structure
• copy over subfolders with their contents
• ignore any files outside subfolders or any other folders outside subfolders

Currently, I'm trying:

$cd destination
$rsync -atvr --include="*/subfolder/" --exclude="*" source/ .

By putting in --exclude="*" I hope to exclude everything except what I put in include, which is */subfolder/, that is, a folder called subfolder being inside any highest-level folder in the source...

However, no files are being ever copied. Why? What am I missing?

I also tried

$cd destination
$rsync -atvr --include="*/subfolder/" --exclude="*/*" source/ .

But that did this:

folder1/
  subfolder/
folder2/
  subfolder/
folder3/
  subfolder/
file1
file2

i.e. kept files in source (outside folders), created the desired subfolders, but did not copy anything inside them.

Edit: at Siva's request, after putting in rsync -av --include='/' --include='/subfolder**' --exclude='*' source/ .

I get:

To access the system:

1) If not using ssh-keys, please enter your TACC password at the password prompt 2) At the TACC Token prompt, enter your 6-digit code followed by .

Password:
TACC Token Code:
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "C.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "C.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "C.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "C.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "C.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "C.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "C.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "C.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
receiving incremental file list
./

sent 59 bytes  received 104 bytes  10.52 bytes/sec
total size is 3,242  speedup is 19.89

But I promise, all those warnings have nothing to do with the problem at hand and pop up every time I try to access the server in whatever way (ssh, scp, rsync, ...).

Answer 1

You can avoid the potential complexity of --include and --exclude by using relative paths to specify the source fileset you want to copy

Scenario

We create your example directory and file structure underneath a topmost directory src. We will be copying the selection to the topmost directory dst

mkdir -p src/folder{1,2,3}/{subfolder,other_subfolders} dst
touch src/folder{1,2,3}/subfolder/contents src/{folder{1,2,3}/,}file{1,2}
find src | LC_ALL=C sort

rsync --dry-run -avR src/./folder*/*/* dst/

Output (from rsync)

sending incremental file list
folder1/
folder1/subfolder/
folder1/subfolder/contents
folder2/
folder2/subfolder/
folder2/subfolder/contents
folder3/
folder3/subfolder/
folder3/subfolder/contents

The /./ marks the point at which the remainder of the source path is to be used at the destination. Remove --dry-run to make it perform the chosen operations.

Answer 2

From man:

if the pattern contains a / (not counting a trailing /), then it is matched against the full pathname, including any leading directories.

Try as,

 rsync -av --include='/' --include='/subfolder**' --exclude='*' source/ destination/