3

The following snippet of code is part of a larger Bash script I'm working on. My intention is to pass input variables as arguments to Rsync.

I managed to make most of it work, except for the "--exclude" parameter's arguments:

INCLUDE="/.vim*,/.git*,/.ssh,.bash_aliases,.bashrc"  

rsync ${DRYRUNARG} -avzhP --delete ${SOURCE_DIRS} \                                                 
    --include=${INCLUDE} \                                                                          
    --exclude={/VirtualBox_VMs/*/Snapshots,/snap,/Downloads,/.*} \                                  
    ${MNTPNT}${DEST_DIR}

This code works perfectly for what I need, but I'd still like to see those directories that have to be excluded defined by a variable.

The resulting code that I need should be something like this:

INCLUDE="/.vim*,/.git*,/.ssh,.bash_aliases,.bashrc"
EXCLUDE="/VirtualBox_VMs/*/Snapshots,/snap,/Downloads,/.*"       

rsync ${DRYRUNARG} -avzhP --delete ${SOURCE_DIRS} \                                                 
    --include=${INCLUDE} \                                                                          
    --exclude=${EXCLUDE} \                                  
    ${MNTPNT}${DEST_DIR}

The code above doesn't work. The command executes without error but fails to exclude anything. The funny part is that it does work if I define the EXCLUDE variable with just one directory, even though it follows the same structure used for the INCLUDE variable.

I'm sure it's some silly mistake I'm making. Where could it be?

6
  • 1
    You can't give the --exclude option a comma-separated list of directories. If you want to exclude multiple directories you need to either use a pattern that matches all the directories or (more likely) use more than one --exclude option. As an aside, if you add set -x at the top of your script you'll see the command Bash is trying to execute (useful for debugging issues like these).
    – rkhff
    Mar 23 '20 at 19:03
  • 1
    The problem is I actually can give the --exclude option a comma-separeted list of directories if I use brace expansion just like the snippet I supplied. It works perfectly. The problem starts when I try to assign this same list to a variable and then pass it to rsync.
    – Livewire
    Mar 23 '20 at 19:05
  • 1
    And your --exclude is subject to brace expansion before the resulting patterns are matched. But brace expansion is the first thing to happen when the code is parsed (before variable substitution, in particular), it won't happen on the contents of variables.
    – xenoid
    Mar 23 '20 at 19:06
  • @xenoid This is interesting. Is there a way I could go around that?
    – Livewire
    Mar 23 '20 at 19:10
  • 1
    You can put the result of the brace-expansion (and pattern -expansiion) in an array declare -a EXCLUDES={/VirtualBox_VMs/*/Snapshots,/snap,/Downloads,/.*}) and then generate the string from that.
    – xenoid
    Mar 23 '20 at 19:13
2

Do not demote a list of separate entries to a single string. This makes it difficult to parse out the different entries and you have to deal with the issues of entries potentially containing the character used as the delimiter, etc. Also, rsync does not know how to parse comma-delimited lists of include/exclude patterns.

Instead, use arrays:

#!/bin/bash

INCLUDE=( '/.vim*' '/.git*' /.ssh .bash_aliases .bashrc )
EXCLUDE=( '/VirtualBox_VMs/*/Snapshots'  /snap /Downloads '/.*' )

Then insert --exclude and --include in front of each item in these lists, creating new lists:

exclude_opts=()
for item in "${EXCLUDE[@]}"; do
    exclude_opts+=( --exclude="$item" )
done

include_opts=()
for item in "${INCLUDE[@]}"; do
    include_opts+=( --include="$item" )
done

Then simply use these new lists in your call to rsync (assuming that SOURCE_DIRS is some array of source directories):

rsync ${DRYRUNARG:+"$DRYRUNARG"} -avzhP --delete \
    "${include_opts[@]}" \
    "${exclude_opts[@]}" \
    "${SOURCE_DIRS[@]}" \
    "$MNTPNT/$DEST_DIR"

Here, $DRYRUNARG will be used quoted if it's set and not empty, otherwise it will not be part of the command line at all.

1

One way would be to use the variable as a file input for --exclude-from for example:

EXCLUDE_DIRS="VirtualBox_VMs
Downloads
Snapshots"

rsync  -arv --exclude-from=<(printf "%s" "$EXCLUDE_DIRS";)  $SOURCE $DESTINATION
0

I've been trying the same. And find the issue baffling.

The best I could do was (using /bin/bash) turn the comma separate list into list of separate --exclude:

INCLUDE="/.vim*,/.git*,/.ssh,.bash_aliases,.bashrc"
EPATHS=("/VirtualBox_VMs/*/Snapshots" "/snap" "/Downloads" "/.*")     

for path in "${EPATHS[@]}"; do
   EXCLUDE="$EXCLUDE --exclude=$path"
done

rsync -avzhP --delete ${SOURCE_DIRS} \                                                 
    --include=${INCLUDE} \                                                                          
    $EXCLUDE             \                                  
    ${MNTPNT} ${DEST_DIR}

This is an incomplete and inelegant solution, but it works.

The lingering issue is that I cannot get it to work with whitespaces in pathnames. When I tried to add quotes to the concatenation, e.g.

EXCLUDE="$EXCLUDE --exclude=\"$path\""

bash seemed to gum it up with more single-quotes.

As noted before, set -x was my friend here.

1
  • The only issue here is that you use EXCLUDE as a single string. Make it an array instead with EXCLUDE+=(--exclude="$path"). Then use that array as "${EXCLUDE[@]}" in the call to rsync. Remember to quote you variable expansions properly.
    – Kusalananda
    Sep 6 '20 at 6:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.