Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a directory with 1000+ files. In a text file, I have about 50 filenames, one per line. I'd like to delete all the files in the directory whose filenames don't correspond with an entry on the list. What's the best way to do this? I started a shell script, but couldn't determine the proper command to determine in the filename is on the list. Thanks.

share|improve this question

4 Answers 4

up vote 5 down vote accepted

I realize that any question asking how to delete files must be taken with great care. My first answer was too hasty I didn't take the fact that the filelist could be malformed to be used with egrep. I edited the answer to reduce that risk.

That should work for the files that has no space in the name:

First rebuild your filelist to be sure to match the exact file name:

sed -e 's,^,^,' -e 's,$,$,'  filelist  > newfilelist 

build the rm commands

cd your_directory
ls | egrep -vf newfilelist   | xargs -n 1 echo rm  >  rmscript

Check if the rm script suits you (You can do it with "vim" or "less").
Then perform the action :

sh -x rmscript

If the files have spaces in their name (if the files have the " in the name that will not works) :

ls | egrep -vf newfilelist  | sed 's,^\(.*\)$,rm "\1",' > rmscript

of course the filelist should not be in the same directory

EDITED :

The Nathan's file list contained names that where matching all the files in the directory (like "html" matches "bob.html"). So nothing was deleted because egrep -vf absorbed all the stream. I added a command to put a "^" and a "$" around each file name. I was lucky here that Nathan's file list was correct. Would have it been DOS formated with CR-LF ended lines or with additional spaces, no files would have been preserved by the egrep and all been deleted.

share|improve this answer
    
When I run the preview command, I get one line with "rm". When I run the actual command, I get an error message about missing arguments for rm. Do I need special syntax to use the results from ls | egrep in the xargs input? –  Nathan Apr 30 at 14:16
    
@Nathan you must cd to your directory first. No special syntaxt. ls provides the directory file names, egrep -vf filelist filter your 50 file names. I'm afraid you deleted all your files. –  Emmanuel Apr 30 at 14:27
    
@Emamanuel I'm running the command from the directory that contains files to be deleted. –  Nathan Apr 30 at 14:31
    
@Nathan are all your files deleted ? –  Emmanuel Apr 30 at 14:31
    
no, they're still there. –  Nathan Apr 30 at 14:32

Pre-construct the arguments to find:

{
  read -r
  keep=( -name "$REPLY" ) # no `-o` before the first one.
  while read -r; do
    keep+=( -o -name "$REPLY" )
  done
} < file_list.txt
find . -type f ! \( "${keep[@]}" \) -exec echo rm {} +

Use the echo parts to see what would be constructed. Remove the echo parts to actually run it.

Update: Demonstration:

##
# Demonstrate what files exist for testing.
# Show their whitespace:
~/foo $ printf '"%s"\n' *
" op"
" qr"
"abc"
"def"
"gh "
"ij "
"k l"
"keep"
"m n"

##
# Show the contents of the "keep" file,
# Including its whitespace:
~/foo $ cat -e keep
keep$
abc$
gh $
k l$
 op$

##
# Execute the script:
~/foo $ { read -r; keep=( -name "$REPLY" ); while read -r ; do keep+=( -o -name "$REPLY" ); done } < keep
~/foo $ find . -type f ! \( "${keep[@]}" \) -exec rm {} +

##
# Show what files remain:
~/foo $ printf '"%s"\n' *
" op"
"abc"
"gh "
"k l"
"keep"
share|improve this answer
    
i like this one best as it removes the need fore filelist –  eyoung100 Apr 30 at 14:56
    
+1 from me, although it doesn't deal very well with spaces. Perhaps some single quotes (') should be added i.e. keep=( -name \'"$REPLY"\' ) and keep+=( -o -name \'"$REPLY"\' ). –  Cristian Ciupitu Aug 26 at 0:36
    
the above is dangerous, because you can delete accidentally files. –  davidva Aug 26 at 1:28
    
@CristianCiupitu doesn't it? I added a demo showing that it deals very well with whitespace. –  kojiro Aug 26 at 13:13
    
@davidva Under what circumstances? Any time you automate deleting things you run the risk of making a mistake, but within the parameters of the question I think my demo proves this approach is sound. –  kojiro Aug 26 at 13:15

If you put the contents of the directory into a file like so:

cd <somedirectory>
ls >> filelist

Open filelist with a text editor, and remove all the files except the ones YOU WANT TO DELETE. That's bolded because it's the opposite approach to the answer above

Try this:

while read p || [[ -n $p ]]; 
echo $p
done < filelist

If you see your list of files output to the screen replace echo with rm -v, like so:

while read p || [[ -n $p ]]; 
rm -v $p
done < filelist
share|improve this answer

Run the below script.

  1. Initially I am finding all the files that are present inside the directory and storing the output to another file all_files.
  2. We have a file which has the list of files that should NOT be deleted (not_to_be_deleted_files).
  3. I am adding the file names not_to_be_deleted_files and files_to_be_deleted to the end of not_to_be_deleted_files as we need these 2 files.
  4. Now, I am finding the files that needs to be deleted using linux join command and redirecting the output to files_to_be_deleted file.
  5. Now, in the final while loop I am reading all the file names in files_to_be_deleted and removing the files mentioned in that file name.

The script is as below.

find /home/username/directory -type f | sed 's/.*\///' > all_files
echo all_files >> not_to_be_deleted_files
echo not_to_be_deleted_files >> not_to_be_deleted_files
echo files_to_be_deleted >> not_to_be_deleted_files
join -v 1 <(sort all_files_listed) <(sort files_not_to_be_deleted) >   files_to_be_deleted
while read file
rm  "$file"
done < files_to_be_deleted

P.S: Probably, if you wish this to be saved as a script and run it, you can add the script name also using echo scriptname >> not_to_be_deleted_files.

Though it is not required, I prefer to do it because there will be no regrets later. I tested for a small set of files and it worked in my system. However, if you want to be sure, try in a test directory first and then remove the files in the original directory.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.