Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

On my Mac OS X 10.6.7, iTunes duplicated every single file in my music library. Now I have 3,840 files instead of 1,920. The problem is that iTunes didn't simply duplicate the whole folder, but each file in each folder, adding a 1 to the duplicate. It looks like this:

├── album1
│   ├── track1.mp3
│   ├── track1 1.mp3
│   ├── track2.mp3
│   └── track2 1.mp3
└── album2
    ├── track1.m4a
    └── track1 1.m4a

Now rather than clicking on 1,920 duplicates, I want to enter some smart line in the Terminal to get rid of the extra files:

find path/to/music/ -name "* 1.*" -delete

Though this will work in most cases, it will mess up things in the following situation:

└── album2
    ├── track 1.mp3
    └── track 1 1.mp3

Maybe I should exclude "* 1 1.*" and later on rename those files removing the extra 1?

How do I do this?

Is there a simple solution to this?

share|improve this question
    
Please specify the environment. Is this OS X? –  Michael Kjörling Apr 8 '13 at 14:13
    
You are right — see edit! I am asking this on UNIX & LINUX because I think it's more likely you guys know... –  AvL Apr 8 '13 at 14:14
1  
Nothing wrong with asking here about using the OS X terminal, as OS X is a Unix system. It just helps if you specify the environment because otherwise people are likely to assume that you are on Linux, with a GNU userland. (The mention of iTunes jarred with this, which prompted my request for clarification.) –  Michael Kjörling Apr 8 '13 at 14:23

2 Answers 2

up vote 4 down vote accepted

Something like this, haven't tested; it's in bash so you may have to convert some syntax:

IFS=$'\n'              # so that only newlines separate words, not spaces
set -f                 # disable globbing
FILES=$(find path/to/music/ -name "* 1.*")

for FILE in ${FILES}; do
    if [[ -f "${FILE% 1.*}" ]] ; do
        echo "Matched ${FILE}."
        # rm "${FILE}" # Uncomment me once you have confirmed it would do what you intend.
    fi
done

${FILE% 1.*} strips the last match of the 1.* syntax from the end, [[ -f ... ]] checks whether that file would exist; therefore, it would remove the files for which the file without the syntax at the end would exist. Please test before uncommenting rm, to be sure it is correct.

share|improve this answer
    
Thanks a lot! Actually I simply tested if there would be a conflict with find . -name "* 1 1.*" -print. This listed 5 files, which I manually renamed after cleaning the folder with: find . -name "* 1.*" \! \( -name "* 1 1.*" \) -delete –  AvL Apr 8 '13 at 14:49

I'd also add that to successfully find (and remove) exact file duplicates, you have to compare files by hash:

#!/bin/sh -eu
find "${1:-.}" -type f ! -empty -print0 | xargs -0 md5 -r | \
    awk '$1 in a{sub("^.{33}","");printf "%s\0",$0}a[$1]+=1{}' | \
    xargs -0 rm -v --

Save this as deldupes.sh and run it giving dirname as 1st parameter (otherwise $PWD would be used).

Tested on OS X, works for long whitespaced filenames.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.