Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a massive music library (all mp3), but I some of the music is almost the same but:

  1. Maybe one or two second longer
  2. About 97% the same as another song
  3. Or another bitrate.

Is there a way to find these duplicates? As mentioned they don't have the same size, name, or SHA1-hash.

share|improve this question
up vote 2 down vote accepted

There's an open source acoustic fingerprinting system called Echoprint, you might want to check I want to deduplicate a big collection. I haven't used it before (just stumbled across it), but it sounds awesome to be able to do this locally.

share|improve this answer
Independent of this, if your files are properly tagged (I am not sure if by name you mean filenames and/or ID3 tags), there are also deduplicating tools which work based on tags. – Adrian Frühwirth Apr 5 '13 at 20:15

Use MusicBrainz's Picard. It scans the music to generate a 'fingerprint' and then compares it against an online database to identify it (pulling all the info like Artist, Album, etc).

After it's identified the songs, it can organize them into a custom directory & name structure, and you can use that to find the duplicates.

Fully compatible with linux.

share|improve this answer
Thanks, I will try that now. – Hans Groeffen Apr 5 '13 at 16:19
This is the definite way to go, works great. – slm Apr 5 '13 at 16:24
Ok, I think I have a problem. None of my music is commercial. Most of it is CC-licensed downloaded from Jamendo. It says it can't find any matches. – Hans Groeffen Apr 5 '13 at 16:36
@HansGroeffen then that would be a problem. You are allowed to publish fingerprints of previously unknown music, so you could do that, but you should make an effort to validate that the submission is accurate. Since you have mismatched names, that might not be a good idea. I'm not sure if there's a way to just use it for it's fingerprinting and comparison capabilities. – Patrick Apr 5 '13 at 16:41
The MusicBrainz database that Picard uses is user-editable. You can create new entries for artists, works (songs) and releases (albums/EPs), and then use Picard to re-tag/move the files. It's a bit of effort, but it works - and it's helpful to others. – naught101 May 26 '15 at 2:57

I'd recommend using beets with its duplicates plugin; beets is similar to picard in terms of functionality (it can lookup ID3 info from the musicbrainz database and organize your files based on that info), but goes beyond picard's abilities with some of its plugins.

Personally I use them together--picard for cleaning up ID3 tags and beets for organizing the files and cleaning up duplicates.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.