Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I'm trying to write me a find script that should later be able to read a list of directories to be excluded from an external file. Whilst I can accomplish that part myself, it's the annoying array expansion that makes the task difficult. First, some "preparations" in order to obtain a suitable sample directory tree:

$ mkdir tmp && cd tmp
$ mkdir excl1_dirx excl2_dirx excl3_dirx
$ touch excl1_dirx/dummy1.txt excl2_dirx/dummy2.txt excl3_dirx/dummy3.txt
$ mkdir excl1_diry excl2_diry excl3_diry
$ touch excl1_diry/dummy4.txt excl2_diry/dummy5.txt excl3_diry/dummy6.txt
$ touch dummy00.txt dummy01.txt

If the script works, only dummy00.txt and dummy01.txt may be shown.

#!/bin/bash
excl_d=("excl*_dirx" "excl*_diry")

find_str=" . -type f ! ( "

for ((i=0 ; i<$((${#excl_d[*]})) ; i++)); do
   if [[ $i > 0 ]]; then
     find_str+=" -o "
   fi

  find_str+=" -path \"./${excl_d[i]}/*\""
done

find_str+=" )"

# this is just for debugging
echo "[debug] value of str = find $find_str"

find $find_str

First of all: why so (seemingly) complicated in the line before the "done"? Well, bash sometimes likes to annoy the users by doing things they don't expect. Without those quotes, it will expand each array element; e. g. excl*_dirx will become excl1 dirx excl2 dirx excl3 dirx, which (obviously) breaks the -path line! And that although I've used a pair of double quotes per array element, which was actually supposed to prevent bash from doing its expansion mischief!

However, the best is yet to come: the second last line - when the ( ) are escaped to \( \) - will work fine in the shell, but not in a standalone script. Even though it won't throw any errors, the result in the latter implementation will be wrong.

I've tried all kinds of combinations with single and double quotes in the

find_str+=" -path \"./${excl_d[i]}/*\""

line, but I just can't get it to work even though it looks absolutely perfect when displayed as in the second last line. It seems that bash internally treats my escaped quotes \" differently from non-escaped ones. Ah, and quotes inside quotes work near everywhere but for some reason will get filtered out when using that += operator.

Not only am I looking for an explanation for this behavior but also for a solution how it could be made work in a standalone script. It must really be a stupid mistake I've made. :-/

share|improve this question
1  
Don't put commands in variables - they weren't designed to work that way. What are you really trying to do? Is your end goal to just print a file list or do you plan to do something with that list? –  jw013 Sep 28 '12 at 16:38
    
1. I did not. I merely put parameters for find in a variable but as you can see in the very last line, I'm actually calling find natively. -- 2. Yes, I want to get me a file list w/excluding a bunch of directories to find files which are at locations where they don't belong. (e. g. *.mp3 files in a location where I have my office stuff). So it will be the "good" locations which are excluded so my list doesn't get cluttered with them. –  syntaxerror Sep 28 '12 at 16:41

2 Answers 2

up vote 1 down vote accepted

The problem here is that find_str is a string, which is then used as a list of strings. jw013's comment is correct, and do read I'm trying to put a command in a variable, but the complex cases always fail!. You're not putting the whole command in a variable, but the problem arises as soon as you try to stuff multiple words in a string variable.

In Bourne/POSIX shells, this is a necessary evil. But in ksh/bash/zsh, there's a better way: use an array.

#!/bin/bash
excl_d=("excl*_dirx" "excl*_diry")
find_str=( . -type f \! \( )
for ((i=0 ; i<$((${#excl_d[*]})) ; i++)); do
   if [[ $i > 0 ]]; then
     find_str+=( -o )
   fi
  find_str+=( -path "./${excl_d[i]}/*" )
done
find_str+=( \) )
find "$find_str[@]}"

There's an easier way of expressing such a filter.

#!/bin/bash
exclude_patterns=("excl*_dirx" "excl*_diry")
exclude_args=()
while [[ ${#exclude_patterns} -gt 0 ]]; do
  exclude_args+=( -path "./${exclude_patterns[1]}/*" -prune -o )
  shift exclude_patterns
done
find "${exclude_args[@]}" -type f
share|improve this answer
    
In one word: FANTASTIC! I would never have been able to pull all that off by myself (yet), because I'm still learning the advanced stuff like functions and arrays (after considering myself safe enough in the basics, which took way longer than I had expected). Thank you very much, your approach looks as if taken from a textbook. –  syntaxerror Sep 29 '12 at 2:24

Here you are:

#!/bin/bash
excl_d=("excl*_dirx" "excl*_diry")

find_str=". "

for ((i=0 ; i<$((${#excl_d[*]})) ; i++)); do
    if [[ $i > 0 ]]; then
    find_str+=" -o "
    fi

    find_str+=" -path \"./\${excl_d[i]}\" -prune "
done

find_str+=" -o -type f -print "

# this is just for debugging
echo "[debug] value of str = find $find_str"

eval "find ${find_str}"

The problem here is that when bash executes a line in the script find ${find_str} it then executes find with just one argument, i.e. argc argument of main function in find will be equal 2. eval in contrast composes the string that then passes tokenizing in bash.

share|improve this answer
1  
Thanks for trying, but no, that does not fix the problem. I still get also files in directories that are in my exclusion list. Though those braces can cure a lot, they are no universal remedy either. –  syntaxerror Sep 28 '12 at 17:11
    
Please see answer to jw013's comment. (2.)) BTW, I did copy-paste the script and tested it, and there were no errors. –  syntaxerror Sep 28 '12 at 17:15
    
Thanks for the update, but it still does not work. Same as before: there are files shown which are inside directories in my exclusion list. Ahem, can you please try to use your updated script on my sample directory tree at the top of this page, thus pre-testing it before attempting any more of these wild experiments? I've already spent too much time with trial-and-error myself, that's why I asked here. Sorry. –  syntaxerror Sep 28 '12 at 17:32
    
@syntaxerror once more. checked on your files –  Serge Sep 28 '12 at 17:40
1  
@syntaxerror Yes, and this is why I did not proposed you that from the very beginning and pointed you the -prune option: if you do the thing like you presented in your Q then you have some reasons to do so. I do not make any assumptions on someone's experience. The figures (I mean reputation, e.t.c.) are meaningless in a sense –  Serge Sep 29 '12 at 2:40

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.