I am trying to run find to exclude a list of directories that I have in a file. This file is used by a different command as well so I am stuck with the format of the file, which is each line contains the directory name "as is" followed by a slash, e.g.

My Files/

I tried various things such as xargs which I could not get to work but finally settled on pre-processing the blacklist to produce the arguments to find like this:

find . -type f $(cat .blacklist.txt | while read -r line; do printf "! -path './%s*' -prune " "$line"; done)

However, whenever I encounter a directory name in the blacklist which has spaces in it, I get the error:

find: paths must precede expression: `<Last Part Of Name>/*''

I used set -x to try to figure out what is going on, and it appears that the separate parts of the directory name are being individually quoted. For instance, suppose I have a directory name in the list called "My Files". I see this output from set -x for this particular line of the file:

++ printf '! -path '\''./%s*'\'' -prune ' 'My Files/'

So far so good I think. But when I see the output of this in the final assembled command, it looks like this:

'!' -path ''\''./My' 'Files/*'\''' -prune 

The problem is obvious: it's turning my directory name which I worked so hard to keep in one piece, into two pieces! But I can't see why it would be doing this. I have tried a few variations, for instance instead of:

printf "! -path './%s*' -prune " "$line"

I think that somehow I need to make sure that the resulting output of the path is quoted when it comes out of the printf, but I have tried all of these and none of them worked:

printf "! -path \"./%s*\" -prune " "$line"
printf "! -path "'./$line*'" -prune "
printf "! -path '"./$line*"' -prune "

But none of these stopped the individual words in the directory name from being split apart.

I also tried this:

printf "! -path ./%s* -prune " "$line"

This avoids splitting the directory name, but as it's no longer quoted it is getting expanded into all the sub-directories under that path which is wrong and also breaks the command as now there are multiple sub-directories where only one directory is expected.

It also does not seem to matter if I use %s followed by the argument as a separate parameter to printf, or if I substitute %s with $line directly.

  • You should probably be assembling the arguments into an array - see for example Using linux 'find' command's multiple name feature in a variable? Apr 21 at 0:48
  • @steeldriver aside from the fact that I can't seem to get the completed array out of the loop when I'm done (it's a sub-process, and all the solutions to get the results out seem to destroy the whole point of having subprocesses) how do I get the values into the array properly in the first place? With set -x I'm still seeing the parts of the directory name get split up.
    – Michael
    Apr 21 at 1:31
  • @steeldriver I found a fix but it's basically to construct the entire command including the find and then eval the entire string, but I'm not sure I understand the implications of eval well enough to know if there is a security hole here...
    – Michael
    Apr 21 at 1:46
  • Related, if not a duplicate - unix.stackexchange.com/a/559604/100397
    – roaima
    Apr 21 at 16:15

4 Answers 4


and it appears that the separate parts of the directory name are being individually quoted.

I wouldn't say that. What happens is that when you print e.g. ! -path './My Files*' -prune from a command substitution, it gets word-split into !, -path, './My, Files*', -prune as the word-splitting step doesn't process syntax like quotes, but only looks at whitespace characters (or what ever you've set IFS to). That Files*' would also act as a glob, but you likely don't have filenames that end in a single quote.

The quotes you see in the set -x output are there because Bash makes that output valid as input to the shell for display. The escaped quote there is the one you printed from printf.

Here, you need to produce !, -path, ./My Files*, -prune as distinct arguments to find. One way to do that is to collect the arguments in an array.

filename='./My Files'
args+=( ! -path "$filename*" -prune )
# etc.
find . -type f "${args[@]}"

But you likely really need that as

find . -type f ! -path './somedir/*' ! -path './otherdir/*'

(without the -prune, walking the whole tree including the listed directories, just ignoring anything in them), or like Stéphane Chazelas shows in their answer,

find . \( -path ./somedir -o -path ./otherdir \) -prune -o -type f -print

(not even descending into the listed directories, you need to escape () for the shell).

And if the excluded names can contain characters that are special for find's pattern matching (*?[), you need to escape those with a backslash, along with any backslashes.

If we use the former pattern, and ignore the escaping:

args=(-type f)
while read -r filename; do
    args+=( ! -path "./$filename/*" )
done < excluded.txt
find . "${args[@]}"

the better way is a bit more involved, something like this:

while read -r escaped; do
    if [ "$first" != 1 ]; then
        args+=( -o )
    args+=( -path "./$escaped" )
done < <(sed -e 's/[[?*\]/\\&/g' < excluded.txt)
find . \( "${args[@]}" \) -prune -o -type f -print

(here, also using a process substitution (<(cmd...)) to pipe the list of filenames through sed for escaping)

The key when building the array is to put the arguments in the assignment exactly as you'd put them in a command, and not e.g. add any extra quotes. Then when using the array, you must just the "${args[@]}" syntax, noting the quotes.


If the intention is to find regular files, skipping the ones in directories whose paths followed by a / are listed in .blacklist.txt one per line, then with that file containing:

my dir/
 my [other] dir?/sub dir/

for instance, you'd need to call find with these arguments:

  1. find
  2. .
  3. (
  4. -path
  5. ./my dir
  6. -o
  7. -path
  8. ./ my \[other\] dir\?/sub dir ([ and ? escaped as otherwise they're treated specially by find's -path; escaping ] shouldn't be necessary in most find implementations but won't harm)
  9. )
  10. -prune # prune the ones matched above
  11. -o # anything else:
  12. -type
  13. f # files of type "regular"
  14. -print

So you need for each line:

  • to remove the trailing /
  • to escape the characters special to -path: \?*[]
  • prepend ./
  • prepend a -path line

Then put a -o argument in between each, and collect the resulting lines into a list to pass to find.

To do that splitting, in the bash shell, you can do it with readarray -t and in zsh with the f parameter expansion flag. You could use split+glob like you did by omitting the quotes around $(...), but you'd need first to set $IFS to newline only and disable the glob part which we don't want here (not done in zsh) with set -o noglob

In bash:

readarray -t args < <(
  sed 's|/$||
-path' .blacklist.txt
find . '(' "${args[@]}" ')' -prune -o -type f -print

In zsh:

find . '(' ${(f)"$(
  sed 's|/$||
-path' .blacklist.txt)"} ')' -prune -o -type f -print

zsh has the b parameter expansion flag to escape glob operators, and in that shell, you could use globbing with the Nullglob qualifier to reduce the blacklist to directories that actually exist and Prepend the -o and -path arguments.

() {
  find . '(' $@[2,-1] ')' -prune -o -type f -print
} ./${(f)^"$(<.blacklist.txt)"}(Ne['REPLY=${(b)REPLY%/}']P[-o]P[-path])

Here, we're P prepending both -o and -path to every file using the P glob qualifier, passing the list to an anonymous function and and passing the second to last argument to find ($@[2,-1]). Given that you likely want to omit .blacklist.txt itself from the result, you could also do:

find . '(' -path ./.blacklist.txt ./${(f)^"$(<.blacklist.txt)"}(Ne['REPLY=${(b)REPLY%/}']P[-o]P[-path]) ')' -prune -o -type f -print

If you have GNU find or compatible, an alternative to -path that doesn't require having to do that escaping, removing the trailing / or prepending ./ is to use -samefile:

find . '(' -samefile .blacklist.txt ${(f)^"$(<.blacklist.txt)"}(NP[-o]P[-path]) ')' -prune -o -type f -print

I'd do this with perl's File::Find module because constructing long find commands with lots of predicates is a PITA compared to just writing a simple procedural script.


use strict;
use File::Find;
use autodie qw(open);

# load the blacklist into an array
my @blacklist;
open(my $BL,"<","blacklist.txt");
while(<$BL>) {
  # Assume one directory per line. Paths are treated
  # as regular expressions, not as literal strings.
  push @blacklist, $_;

# generate a regexp to match all the entries in the array
our $blacklist_re = "^(?:" . join("|",@blacklist) . ")";
#print "$blacklist_re\n";

find { wanted => \&wanted, preprocess => \&prune }, '.';

sub wanted { -f && print "$File::Find::name\n" };
sub prune  { return grep { ! -d || ! m/$blacklist_re/ } @_ };

Updated 2023-04-24 - this version actually prunes the unwanted directories by using a preprocess subroutine, so that File::Find doesn't descend into them.

The preprocess subroutine prune is called every time find enters into a directory (including any directories given as args to the find function, . in this case).

The filenames (including dirs, sockets, device nodes etc) in a dir are passed as an array to prune, which returns an array of filenames which either aren't directories OR don't match the blacklist regexp. This uses perl's built-in grep function - this is not the external /bin/grep command, it operates on lists/arrays instead of files. See perldoc -f grep.

Any filenames NOT in the array returned by prune are excluded from further processing by find, so the wanted subroutine has been simplified to just a -f test - wanted never even sees pruned directories or files.

Sample Run:

With a blacklist.txt file containing the following:

foo bar
bar baz

and creating those directories and some dummy files:

mkdir 'foo bar' 'bar baz' 
touch 'bar baz/a.txt' 'foo bar/b.txt' 'foo bar/c.txt'
touch d.txt

so that the directory structure looks like this:

├── bar baz
│   └── a.txt
├── blacklist.txt
├── d.txt
├── ff.pl
├── foo bar
    ├── b.txt
    └── c.txt

Running the script (after saving it as, e.g., ff.pl, and making it executable with chmod +x ff.pl) produces the following output:

$ ./ff.pl 

i.e. the directores in blacklist.txt are excluded from the output.

  • oh wow, thanks! I didn't know about %q. I'll test this out as soon as I get a check and then accept.
    – Michael
    Apr 21 at 4:29
  • %q won't help here, the output of that sloop is split with split+glob which doesn't understand quotes (and you don't want the glob part). Those ANDed together ! -path ... -prune likely don't make sense either Apr 21 at 7:34

What I'd probably do (assuming bash and GNU find):

find . -exec fgrep -qx \{} $(sed 's#/$##' .blacklist.txt) \; -prune -o -type f -print

It is not very efficient if the blacklist is long, but it works with most filenames. (Most: e.g. filenames with newlines in them are still an issue.)

  • There's nothing specific to bash or GNU find in there, but that won't work as you're passing the edited contents of .blacklist.txt subject to split+glob to fgrep while you want to pass the path to a file containing that edited contents, so you'd need zsh and something like -exec grep -Fqx '{}' =(sed 's|/$||' .blacklist.txt) ';' Apr 22 at 8:56
  • 1
    Note that it's not efficient even if the blacklist is short as it spawns one process and executes one fgrep (likely a lot more expensive than looking up text in that file unless the blacklist is huge) for each file. Apr 22 at 8:59
  • It may not be efficient, but it does work (I tested it) and it is relatively easy to write. Apr 22 at 21:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .