Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a lookup file containing filenames. I also have a directory containing files, some of which have the names that correspond to some of the names in the directory.

How can I:

  1. Move exact matching files to a new directory or
  2. Remove non-matching files from the existing directory

Also, how can I do this recursively through a top-level directory and its subdirectories?

share|improve this question
    
Is this tagged bash for any specific reason or it was actually meant to be "shell script"? –  njsg Jan 28 '13 at 16:08
    
I lack the time to make a more appropriate answer right now, but I suppose you could use bash's globstar for each entry in the file (for x in $(cat list); do ls **/$x; done), although you'd have to test each name in the expansion of **/$x to rule out directories, or give the names to find and have it move the matches with mv, the problem is, if you have more than one match for the same name, how would you handle (1.)? –  njsg Jan 28 '13 at 16:12
    
I don't expect there to be more than one match but it's certainly worth considering. I suppose, only one should be kept (the first one found perhaps) since I think we can assume that the content is the same if the filenames are the same. –  user31333 Jan 28 '13 at 16:20
add comment

2 Answers 2

Perhaps something along the following lines:

sed -e 's;^.*$;\^&\$; list-of-names > list-of-patterns

rm $(find . -type f | grep -v -f list-of-patterns)

Be careful, this will surely break if the filenames contain spaces or characters that the shell considers special. Please check what this would delete before running it for real!

(There might be a way of sidestepping the pattern file, but off the top of my head I don't see it).

share|improve this answer
    
safer way: 1) move the matching files in their new destination 2) delete the remaining (after user confirmation to proceed) –  Olivier Dulac Jan 28 '13 at 16:34
    
Right. But then you have to munge the paths, and that is more complicated than just finding out the files to delete. In any case, do not do this unless you know what is going to happen, and consider backing up the full original just in case. –  vonbrand Jan 28 '13 at 16:40
    
another thing: your find will display names as : ./name1 ./name2 ./subdir/name3 and you are trying to match it to '^somename$' : it won't work. Maybe in that case recreate list-of-patterns to be : ^.*/&$ instead of ^&$ –  Olivier Dulac Jan 28 '13 at 17:44
    
@OlivierDulac, Brain fart. Need \< and \> instead of ^ and $. Thanks! –  vonbrand Jan 28 '13 at 17:46
    
I really recommend the way I put it, as otherwise if a subdir match one of the filename, you end up deleting all that subdir's content... iow 'careful with that axe, eugene' (replace axe with rm ^^ it's a ZZtop song) –  Olivier Dulac Jan 28 '13 at 17:48
add comment

The following bash script (warning: untested!) moves files from under /directory/with/the/files to under /matched/files. Only files whose name (sans directory) is in /path/to/lookup.file are moved. Beware that if there are symbolic links to directories in /directory/with/the/files, they are traversed recursively as if they were directories themselves.

#!/bin/bash
shopt -s globstar
set -f; IFS=$'\n'
typeset -A filenames
for x in $(cat /path/to/lookup.file); do filenames[$x]=1; done
set +f; unset IFS
cd /directory/with/the/files
for x in ./**; do
  if [[ -d "$x/." ]]; then
    : # skip directories and symbolic links to directories
  elif [[ -n ${filenames[${x##*/}]} ]]; then
    # the file is matched, move it under /matched/files
    mkdir -p "/matched/files/${x%/*}"
    mv "$x" "/matched/files/$x"
  else
    # the file isn't matched
    :
  fi
done

Explanations:

  • shopt -s globstar enables the ** glob to traverse directories recursively.
  • set -f; IFS=$'\n' turns off globbing and sets newlines as the sole word delimiter, so that the unquoted command substitution $(cat /path/to/lookup/files) is split at newlines only.
  • filenames is an associative array.
  • for x in ./** takes care that $x always contains a directory part (this way ${x%/*} is the directory part of $x, with . for the toplevel directory), and does not begin with - (so it doesn't risk being treated as an option).
  • ${x##*/} expands to the filename part of $x, i.e. $x without the directory part.
share|improve this answer
    
Thanks. when I save this as a shell script, I get fileextractor.sh: 1: fileextractor.sh: shopt: not found fileextractor.sh: 3: fileextractor.sh: typeset: not found fileextractor.sh: 4: fileextractor.sh: Syntax error: "fi" unexpected (expecting "done") –  user31333 Jan 29 '13 at 9:24
    
@user31333 That's a bash script, you need to run it with bash (start with a #!/bin/bash line). –  Gilles Jan 29 '13 at 9:33
    
Yep I did that, and I run it like this sh ./fileextractor.sh from the script's directory. Still get the output above though??? –  user31333 Jan 29 '13 at 9:59
    
Ok it runs now. However, I get ./fileextractor.sh: line 17: syntax error near unexpected token fi'` –  user31333 Jan 29 '13 at 10:31
    
@user31333 If you don't have anything to do in the else part, either add a : (no-op command) or remove else. –  Gilles Jan 29 '13 at 12:38
show 5 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.