2

I have a bash script where I'm trying to copy some folders and archives in local with rsync. I have something like this:

RootFolder
  A
  B
  C 
  D
     D1
     D2
     D3
     .
     .
     .
     DN 
  E 
  F

What I want/do:

  • Clean the Destiny Folder before start which I do with option --delete
  • Exclude some type of archives which I do with option --exclude.
  • Exclude some folders which I do with option --exclude.

So until here I will have a copy Folders A, B, D and E.

But now I need to copy Only some subfolders in D and the list will change according to a list.

So Imagine I want:

 A 
 B 
 D 
   D2 
   D4
   D8
 E 

I don't know how to do it as I've tried with --include but I'm doing something wrong.

I'm doing something like:

 rsync -av --delete --include=/D/D2 --include=/D/D4 --include=D8 --exclude=/C --exclude=/F RootFolder/ DestinyFolder

But It doesn't work well..... I've thought about doing some iterations or maybe I don't know if I could put all thee includes in a variable and pass it to rsync.

I have another doubt which is: As I get the destination Folder by param, what happens if the user uses "../" and I have the --delete option?? Will it delete the current folder? And if user just writes "/" will it delete everything?

Edit: Having:

RootFolder
  A
     File.txt
     .HidenFile
  B
  C 
  D
     D1
     D2
       File.txt
       .HidenFile
     D3
     .
     .
     .
     DN 
  E 
  F

I have something like

rsync -av --delete  --include=/D/D2 --include=/D/D4 --include=D8 --exclude=*.txt  --exclude=.* --exclude=/C --exclude=/F RootFolder/ DestinyFolder
4
  • As for your last paragraph: always check your variables before you use them, especially if you get them from user input. And quote them whenever applicable. Feb 26, 2019 at 9:11
  • @WeijunZhou thank you! what do you mean by quoting?
    – Megasa3
    Feb 26, 2019 at 9:17
  • 1
    "$var" vs $var. One of my favorite questions: Security implications of forgetting to quote a variable in bash/POSIX shells Feb 26, 2019 at 9:24
  • Your latest edit shows that you want to exclude files matching *.txt and .*. But there are no other files in your example left to copy. Which ones do you really want, and which do you want to ignore? Please make this an explicit statement in your question (use words, not code, to describe the requirement). Feb 26, 2019 at 10:49

1 Answer 1

2

The options for --include and --exclude are applied from left to right. The first options are always the strongest, so you can end a list with --exclude='*/' to stop any further processing of directories.

Once you use --exclude='*/' though, you will disable implicit processing of all files and directories, and you will need to state explicitly which files and folders to include.

  • You want to include A, B, and E with all their files and folders,
  • You want D and its files but not directories,
  • You want D/D2, D/D4, /D/D8 and all their files and folders,
  • Nothing else.

We can therefore construct the include/exclude statements like this

rsync --dry-run -av --delete

    --include '/A' --include '/A/**'
    --include '/B' --include '/B/**'
    --include '/E' --include '/E/**'

    --include '/D'

    --include '/D/D2' --include '/D/D2/**'
    --include '/D/D4' --include '/D/D4/**'
    --include '/D/D8' --include '/D/D8/**'

    -exclude '*/'

    src/ dst/

Since you are using bash, which is a shell that can handle arrays, you can build up the set of --include instructions safely. Here's an example:

# It's an array
includes=()

# Start with the fixed set of known directories
for known in A B E
do
    includes+=('--include' "/$known" '--include' "/$known/**")
done

# Special case
includes+=('--include' '/D')

# Now the changeable set of subdirectories. You might get this list elsewhere
for subdirs in D2 D4 D8
do
    includes+=('--include' "/D/$subdir" '--include' "/D/$subdir/**")
done

# Put it all together
rsync --dry-run -av "${includes[@]}" --exclude '*/' src/ dst/
8
  • Thanks. What happends if I have /A/file.txt and I have an --exclude=*.txt ? Will work still with your --include '/A' --include '/A/**' ? I'll edit my post to add this info and make it clear. And yes, its a bash script
    – Megasa3
    Feb 26, 2019 at 10:00
  • @Megasa3 why not try it? The --dry-run option that I've included in all my examples shows what would happen without actually making any changes. Feb 26, 2019 at 10:02
  • Oh! So so sorry I didn't noticed.. I've made my edit but I'm going to try this now. Thanks!
    – Megasa3
    Feb 26, 2019 at 10:19
  • It starts printing all that is supossed to move like /A /.hidenFile1 /.hidenFile2 /.hidenFileN but just do it with A Folder and if I look into destination, there is nothing.. The /src and/dst are correct as I'm using the same as with my example.
    – Megasa3
    Feb 26, 2019 at 10:42
  • Remember that --dry-run doesn't actually do any work. It only shows you what would happen. When you're happy with what it intends to do, just remove that option. Feb 26, 2019 at 10:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .