6

On my Mac OS X 10.6.7, iTunes duplicated every single file in my music library. Now I have 3,840 files instead of 1,920. The problem is that iTunes didn't simply duplicate the whole folder, but each file in each folder, adding a 1 to the duplicate. It looks like this:

├── album1
│   ├── track1.mp3
│   ├── track1 1.mp3
│   ├── track2.mp3
│   └── track2 1.mp3
└── album2
    ├── track1.m4a
    └── track1 1.m4a

Now rather than clicking on 1,920 duplicates, I want to enter some smart line in the Terminal to get rid of the extra files:

find path/to/music/ -name "* 1.*" -delete

Though this will work in most cases, it will mess up things in the following situation:

└── album2
    ├── track 1.mp3
    └── track 1 1.mp3

Maybe I should exclude "* 1 1.*" and later on rename those files removing the extra 1?

How do I do this?

Is there a simple solution to this?

  • Please specify the environment. Is this OS X? – a CVn Apr 8 '13 at 14:13
  • You are right — see edit! I am asking this on UNIX & LINUX because I think it's more likely you guys know... – AvL Apr 8 '13 at 14:14
  • 1
    Nothing wrong with asking here about using the OS X terminal, as OS X is a Unix system. It just helps if you specify the environment because otherwise people are likely to assume that you are on Linux, with a GNU userland. (The mention of iTunes jarred with this, which prompted my request for clarification.) – a CVn Apr 8 '13 at 14:23
4

Something like this, haven't tested; it's in bash so you may have to convert some syntax:

IFS=$'\n'              # so that only newlines separate words, not spaces
set -f                 # disable globbing
FILES=$(find path/to/music/ -name "* 1.*")

for FILE in ${FILES}; do
    if [[ -f "${FILE% 1.*}" ]] ; do
        echo "Matched ${FILE}."
        # rm "${FILE}" # Uncomment me once you have confirmed it would do what you intend.
    fi
done

${FILE% 1.*} strips the last match of the 1.* syntax from the end, [[ -f ... ]] checks whether that file would exist; therefore, it would remove the files for which the file without the syntax at the end would exist. Please test before uncommenting rm, to be sure it is correct.

  • Thanks a lot! Actually I simply tested if there would be a conflict with find . -name "* 1 1.*" -print. This listed 5 files, which I manually renamed after cleaning the folder with: find . -name "* 1.*" \! \( -name "* 1 1.*" \) -delete – AvL Apr 8 '13 at 14:49
5

I'd also add that to successfully find (and remove) exact file duplicates, you have to compare files by hash:

#!/bin/sh -eu
find "${1:-.}" -type f ! -empty -print0 | xargs -0 md5 -r | \
    awk '$1 in a{sub("^.{33}","");printf "%s\0",$0}a[$1]+=1{}' | \
    xargs -0 rm -v --

Save this as deldupes.sh and run it giving dirname as 1st parameter (otherwise $PWD would be used).

Tested on OS X, works for long whitespaced filenames.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.