I have a lookup file containing filenames. I also have a directory containing files, some of which have the names that correspond to some of the names in the directory.

How can I:

  1. Move exact matching files to a new directory or
  2. Remove non-matching files from the existing directory

Also, how can I do this recursively through a top-level directory and its subdirectories?

  • Is this tagged bash for any specific reason or it was actually meant to be "shell script"?
    – njsg
    Jan 28, 2013 at 16:08
  • I lack the time to make a more appropriate answer right now, but I suppose you could use bash's globstar for each entry in the file (for x in $(cat list); do ls **/$x; done), although you'd have to test each name in the expansion of **/$x to rule out directories, or give the names to find and have it move the matches with mv, the problem is, if you have more than one match for the same name, how would you handle (1.)?
    – njsg
    Jan 28, 2013 at 16:12
  • I don't expect there to be more than one match but it's certainly worth considering. I suppose, only one should be kept (the first one found perhaps) since I think we can assume that the content is the same if the filenames are the same.
    – user31333
    Jan 28, 2013 at 16:20

2 Answers 2


Perhaps something along the following lines:

sed -e 's;^.*$;\^&\$; list-of-names > list-of-patterns

rm $(find . -type f | grep -v -f list-of-patterns)

Be careful, this will surely break if the filenames contain spaces or characters that the shell considers special. Please check what this would delete before running it for real!

(There might be a way of sidestepping the pattern file, but off the top of my head I don't see it).

  • safer way: 1) move the matching files in their new destination 2) delete the remaining (after user confirmation to proceed) Jan 28, 2013 at 16:34
  • Right. But then you have to munge the paths, and that is more complicated than just finding out the files to delete. In any case, do not do this unless you know what is going to happen, and consider backing up the full original just in case.
    – vonbrand
    Jan 28, 2013 at 16:40
  • another thing: your find will display names as : ./name1 ./name2 ./subdir/name3 and you are trying to match it to '^somename$' : it won't work. Maybe in that case recreate list-of-patterns to be : ^.*/&$ instead of ^&$ Jan 28, 2013 at 17:44
  • @OlivierDulac, Brain fart. Need \< and \> instead of ^ and $. Thanks!
    – vonbrand
    Jan 28, 2013 at 17:46
  • I really recommend the way I put it, as otherwise if a subdir match one of the filename, you end up deleting all that subdir's content... iow 'careful with that axe, eugene' (replace axe with rm ^^ it's a ZZtop song) Jan 28, 2013 at 17:48

The following bash script (warning: untested!) moves files from under /directory/with/the/files to under /matched/files. Only files whose name (sans directory) is in /path/to/lookup.file are moved. Beware that if there are symbolic links to directories in /directory/with/the/files, they are traversed recursively as if they were directories themselves.

shopt -s globstar
set -f; IFS=$'\n'
typeset -A filenames
for x in $(cat /path/to/lookup.file); do filenames[$x]=1; done
set +f; unset IFS
cd /directory/with/the/files
for x in ./**; do
  if [[ -d "$x/." ]]; then
    : # skip directories and symbolic links to directories
  elif [[ -n ${filenames[${x##*/}]} ]]; then
    # the file is matched, move it under /matched/files
    mkdir -p "/matched/files/${x%/*}"
    mv "$x" "/matched/files/$x"
    # the file isn't matched


  • shopt -s globstar enables the ** glob to traverse directories recursively.
  • set -f; IFS=$'\n' turns off globbing and sets newlines as the sole word delimiter, so that the unquoted command substitution $(cat /path/to/lookup/files) is split at newlines only.
  • filenames is an associative array.
  • for x in ./** takes care that $x always contains a directory part (this way ${x%/*} is the directory part of $x, with . for the toplevel directory), and does not begin with - (so it doesn't risk being treated as an option).
  • ${x##*/} expands to the filename part of $x, i.e. $x without the directory part.
  • Thanks. when I save this as a shell script, I get fileextractor.sh: 1: fileextractor.sh: shopt: not found fileextractor.sh: 3: fileextractor.sh: typeset: not found fileextractor.sh: 4: fileextractor.sh: Syntax error: "fi" unexpected (expecting "done")
    – user31333
    Jan 29, 2013 at 9:24
  • @user31333 That's a bash script, you need to run it with bash (start with a #!/bin/bash line). Jan 29, 2013 at 9:33
  • Yep I did that, and I run it like this sh ./fileextractor.sh from the script's directory. Still get the output above though???
    – user31333
    Jan 29, 2013 at 9:59
  • Ok it runs now. However, I get ./fileextractor.sh: line 17: syntax error near unexpected token fi'`
    – user31333
    Jan 29, 2013 at 10:31
  • @user31333 If you don't have anything to do in the else part, either add a : (no-op command) or remove else. Jan 29, 2013 at 12:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.