2

I have two folders (w/ subfolders) containing files that both contain essentially the same content.

Example:

  • /folder/files/data/new
  • /folder/files/data/old

Is there way to delete all files in /new that are already in /old by strictly filename (NOT date modified) to prevent me from re-reading duplicate files.

  • 2
    Hi, definitely there is way using shell script to achieve what you are trying to do. But, it would always be better to let people know what you have tried so far and where you are struck. – Ramesh Mar 29 '14 at 20:30
  • I tried enedil's script below without success. – Guest Mar 29 '14 at 20:57
  • By filename, or by path? E.g. if you have new/foo/somefile and new/bar/somefile, are these duplicates or unrelated files? – Gilles Mar 30 '14 at 0:27
  • 1
    You should use checksums to compare files instead of using just the filenames. – S182 Mar 30 '14 at 1:44
  • +1 @Serge files can't be duplicates by mere file names, size & checksums(MD5/SHA1, etc) must be verified. – SparKot Mar 30 '14 at 7:45
5

The code below will search through not just the old directory but also its subdirectories and delete files from the new directory that have the same name and path. This will work with all manor of hostile file names (spaces, newlines, etc.):

cd /folder/files/data/old || { echo "Failed to get in correct directory" ; exit 1 ; }
find . -type f -exec rm -f ../new/{} \;

The argument -type f tells find to look for files, not directories. The -exec ... \; argument tells find that, once it finds a file in old, it should run the command rm -f on its counterpart in the new subdirectory.

Because rm is given the -f option, it will silently ignore files that don't exist. This eliminates the need for testing for the existence of the file in new before attempting to delete it. Also, it will not prompt before deleting files. This is very handy when deleting a large number of files. However, be sure that you really want to delete the files before running this script. For protection, the script will exit with an error with an error if the cd command fails.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.