1

I know there are a billion questions of this out there. But I'm positive I have tried many of them and i am not able to make this work, so please, don't just mark this as duplicate

I have a file system looking like

1_counts/
|_________sample1/
          |__________boring_file1
          |__________boring_file2
          |__________boring_dir1/
                     |__________boring_file1
                     |__________boring_file2
          |__________dir/
                     |__________boring_file1
                     |__________boring_file2
                     |__________another_dir/
                                |__________file1
                                |__________file2
                                |__________file3
                                |__________boring_dir/
                                           |__________boring_file
                                |__________boring_file.RData

And I have several "sample" directories.

I need to sync file 1, 2 and 3 on another_dir/ . I want to keep the file structure (I dont have the sub/directories in the destination), I just dont want to copy everything.

I started by trying to get everything under dir/another_dir:

rsync -r -v --dry-run --include="dir/" \
--include="dir/another_dir/" \
--exclude="*" 1_counts/* .

This doesnt return any files with the message [sender] hiding directory sample_1 because of pattern *. Same with

rsync -r -v --dry-run --include="dir/" \
--include="dir/another_dir/***" \
--exclude="*" 1_counts/* .

This option (here on referred as solution 1) retrieved all contents in dir/another_dir/:

rsync -r -v --dry-run --include="dir/" \
--include="dir/another_dir/***" \
--exclude="*/*" 1_counts/* .

Tbh, I guessed it. I have no idea why I need */* for the exclude.

If I try

rsync -r -v --dry-run --include="dir/" \
--include="dir/another_dir/" \
--exclude="*/*" 1_counts/* .

I get only the dir/another_dir directories, not the contents. As expected.

And if i do

rsync -r -v --dry-run --include="dir/" \
--include="dir/another_dir/*" \
--exclude="*/*" 1_counts/* .

I get only the dir/ directories, no contents. I guess this is also expected (Second answer here) but I am confused why i dont get another_dirs also... A mystery.

Anyway, now I can use solution 1 to copy everything from 1_counts/sample1/dir/another_dir. Now I'm trying to exclude the boring_file.RData and dir/another_dir/boring_dir.

I tried

rsync -r -v --dry-run --include="dir/" \
--include="dir/another_dir/***" \
--exclude="*.RData" \
--exclude="boring_dir/" \
--exclude="*/*" 1_counts/* .

This doesn't work. Everything is still included. I thought it was something to do with the path, so i also tried

rsync -r -v --dry-run --include="dir/" \
--include="dir/another_dir/***" \
--exclude="dir/another_dir/*.RData" \
--exclude="dir/another_dir/boring_dir/" \
--exclude="*/*" 1_counts/* .

Doesn't work either. I'm running out of options and I'm confused on why some parts of this works...

I would really appreciate any input on this.

2

At this point you are very close to the solution:

rsync -r -v --dry-run --include="dir/" \
--include="dir/another_dir/***" \
--exclude="*.RData" \
--exclude="boring_dir/" \
--exclude="*/*" 1_counts/* .

The thing is that rsync uses the first matching pattern, so by including everything under another_dir, you're effectively including the boring stuff and .RData files. You only have to change the order of the filter rules:

rsync -r -v --dry-run --include="dir/" \
--exclude="*.RData" \
--exclude="boring_dir/" \
--include="dir/another_dir/***" \
--exclude="*/*" 1_counts/* .

Because order matters, people put rules to exclude files by extension at the beginning, and rules to exclude all at the end.

1
  • 1
    Thank you so much! This solved it. And now I kinda understand what is going on :-). – compuTE Mar 2 at 18:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.