find files not matching list of filename patterns

Question

I find myself needing to find and identify extraneous files (out of about 900K files on a 2T drive). There are lots of files that I want to keep, and I have filename patterns for these known good files. What I want is to locate those files that don't fit any of the patterns.

How do I find files not matching a list of filename patterns?

I can run find to get a list of all files, and I could use grep -v on the result, using a list of patterns stored in a file. Is this the canonical method, or do you have a concise way to find these non-conforming files?

---

Clarification - based upon the answers, here is a little more information. I expect to have numerous patterns (>20, maybe >100), I want to store them in a file and certainly want an easy way to add new ones. I would prefer to avoid directly editing a large list of find parameters (fragile), but building that list might work.

Answer 1

find(1) is powerful enough to do what you need. Simply collect all of the conforming names into an expression using parentheses, then negate it to show non-conforming file names. For example, to show all files not named *.txt, *.bz2, or *.zip:

$ find . \! \( -name \*.txt -o -name \*.bz2 -o -name \*.zip \)

You can use -not instead of \! with GNU and BSD find. It isn't POSIX compliant, but it doesn't require an escape to prevent the shell from interpreting it.

To build the expression from patterns in a file, it's a small matter of shell scripting:

#!/bin/sh
set --
while IFS= read -r pattern
do
    set -- "$@" -o "$pattern"
done < .fnpatterns
if [ $# -ne 0 ]; then
  shift
  set -- -not \( "$@" \)
fi
find . "$@"

This expects a file in the current directory called .fnpatterns with one pattern per line. To mimic the one-liner above, it would need to contain:

*.txt
*.bz2
*.zip

Note that the shell script escapes the * characters in the patterns for you.

You can make this arbitrarily complex. Some ideas:

• Add -type f to the find command so it only shows normal files, not directories.

• Pass the pattern file name in as an argument instead of expecting it in a fixed location

• Keep the pattern file where it is, but add -o -name .fnpatterns to the built-up find command so it doesn't show up in the output. (This would also avoid the need for the shift hack to "eat" the lead -o in the built-up expression.)

• Add actions to the find command via -exec or similar.

• Allow blank lines or comments in the patterns file

Answer 2

Since you mention Perl...

#!/usr/bin/perl

use strict;
use warnings;
use File::Find qw{find};

my %patterns;
while (<>) {
  chomp;
  $patterns{$_}++;
}

die "No pattern supplied\n" unless keys %patterns;

find( 
    sub{
           my $matches_a_pattern=0;
           for my $pattern (keys %patterns){
               my $glob_pattern = $pattern;
               for($glob_pattern){
                   s/\./\\./g;
                   s/\*/.*/g;
                   s/\?/./g;
               }
               $matches_a_pattern++ if ( /\Q$pattern\E/ or /$glob_pattern/);
           }

           print "$File::Find::name\n" unless $matches_a_pattern;
     }
    , '.' )

Invoke this as

/path/to/my/script file_with_patterns

Replace the . at the end with the top of the tree you want to walk.