Auto-expansion problem with array elements containing an '*' (asterisk)

Question

I'm trying to write me a find script that should later be able to read a list of directories to be excluded from an external file. Whilst I can accomplish that part myself, it's the annoying array expansion that makes the task difficult. First, some "preparations" in order to obtain a suitable sample directory tree:

$ mkdir tmp && cd tmp
$ mkdir excl1_dirx excl2_dirx excl3_dirx
$ touch excl1_dirx/dummy1.txt excl2_dirx/dummy2.txt excl3_dirx/dummy3.txt
$ mkdir excl1_diry excl2_diry excl3_diry
$ touch excl1_diry/dummy4.txt excl2_diry/dummy5.txt excl3_diry/dummy6.txt
$ touch dummy00.txt dummy01.txt

If the script works, only dummy00.txt and dummy01.txt may be shown.

#!/bin/bash
excl_d=("excl*_dirx" "excl*_diry")

find_str=" . -type f ! ( "

for ((i=0 ; i<$((${#excl_d[*]})) ; i++)); do
   if [[ $i > 0 ]]; then
     find_str+=" -o "
   fi

  find_str+=" -path \"./${excl_d[i]}/*\""
done

find_str+=" )"

# this is just for debugging
echo "[debug] value of str = find $find_str"

find $find_str

First of all: why so (seemingly) complicated in the line before the "done"? Well, bash sometimes likes to annoy the users by doing things they don't expect. Without those quotes, it will expand each array element; e. g. excl*_dirx will become excl1 dirx excl2 dirx excl3 dirx, which (obviously) breaks the -path line! And that although I've used a pair of double quotes per array element, which was actually supposed to prevent bash from doing its expansion mischief!

However, the best is yet to come: the second last line - when the ( ) are escaped to \( \) - will work fine in the shell, but not in a standalone script. Even though it won't throw any errors, the result in the latter implementation will be wrong.

I've tried all kinds of combinations with single and double quotes in the

find_str+=" -path \"./${excl_d[i]}/*\""

line, but I just can't get it to work even though it looks absolutely perfect when displayed as in the second last line. It seems that bash internally treats my escaped quotes \" differently from non-escaped ones. Ah, and quotes inside quotes work near everywhere but for some reason will get filtered out when using that += operator.

Not only am I looking for an explanation for this behavior but also for a solution how it could be made work in a standalone script. It must really be a stupid mistake I've made. :-/

Answer 1

The problem here is that find_str is a string, which is then used as a list of strings. jw013's comment is correct, and do read I'm trying to put a command in a variable, but the complex cases always fail!. You're not putting the whole command in a variable, but the problem arises as soon as you try to stuff multiple words in a string variable.

In Bourne/POSIX shells, this is a necessary evil. But in ksh/bash/zsh, there's a better way: use an array.

#!/bin/bash
excl_d=("excl*_dirx" "excl*_diry")
find_str=( . -type f \! \( )
for ((i=0 ; i<$((${#excl_d[*]})) ; i++)); do
   if [[ $i > 0 ]]; then
     find_str+=( -o )
   fi
  find_str+=( -path "./${excl_d[i]}/*" )
done
find_str+=( \) )
find "$find_str[@]}"

There's an easier way of expressing such a filter.

#!/bin/bash
exclude_patterns=("excl*_dirx" "excl*_diry")
exclude_args=()
while [[ ${#exclude_patterns} -gt 0 ]]; do
  exclude_args+=( -path "./${exclude_patterns[1]}/*" -prune -o )
  shift exclude_patterns
done
find "${exclude_args[@]}" -type f

Answer 2

Here you are:

#!/bin/bash
excl_d=("excl*_dirx" "excl*_diry")

find_str=". "

for ((i=0 ; i<$((${#excl_d[*]})) ; i++)); do
    if [[ $i > 0 ]]; then
    find_str+=" -o "
    fi

    find_str+=" -path \"./\${excl_d[i]}\" -prune "
done

find_str+=" -o -type f -print "

# this is just for debugging
echo "[debug] value of str = find $find_str"

eval "find ${find_str}"

The problem here is that when bash executes a line in the script find ${find_str} it then executes find with just one argument, i.e. argc argument of main function in find will be equal 2. eval in contrast composes the string that then passes tokenizing in bash.