Is it possible to find duplicate files on my disk which are bit to bit identical but have different file-names?
In Debian or Ubuntu, you can install it with
To run a check descending from your filesystem root, which will likely take a significant amount of time and memory, use something like
As asked in the comments, you can get the largest duplicates by doing the following:
This will break if your filenames contain newlines.
Another good tool is
On debian-based systems, youcan install it with:
You can also do this manually if you don't want to or cannot install third party tools. The way most such programs work is by calculating file checksums. Files with the same md5sum almost certainly contain exactly the same data. So, you could do something like this:
Sample output (the file names in this example are the same, but it will also work when they are different):
This will be much slower than the dedicated tools already mentioned, but it will work.
Short answer: yes.
Longer version: have a look at the wikipedia fdupes entry, it sports quite nice list of ready made solutions. Of course you can write your own, it's not that difficult - hashing programs like
If you believe a hash function (here MD5) is collision free on your domain:
Wanna identical file names grouped? Write a simple script
This is basic idea. Probably you should change
Have a look on this wikipedia link displaying available open source software for this task.
I will add that the GUI version of fslint is very interesting, allowing to use mask to select which files to delete. Very useful to clean duplicated photos.
On Linux you can use:
The 2 last work on many systems (windows, mac and linux) I 've not checked for FSLint
protected by Community♦ Jan 14 at 12:14
Thank you for your interest in this question.
Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).
Would you like to answer one of these unanswered questions instead?