4

In Unix, is there any way to compare every text file in a directory to every other text file in the directory, and then sort each pair of files by similarity (using the diff utility)? There are already some command-line Unix programs (such as fdupes) that can find duplicate files in a directory, but I'm wondering if it's possible to find similar files using a shell script as well.

Improve this question
9

1 Answer 1

Reset to default
1

I think this question is too broad on so many levels. Term "difference" depends on type of data and its container: txt, mp3, avi, jpg. For each of them you need to execute their own processing methods. For instance, text or source code files may require only diff utility. Music, video and image files require fuzzy logic and computer learning algorithms.

Improve this answer
5
  • By the way, for example geeqie image viewer has some algorithms for comparing images.
    – peterph
    Commented Jan 8, 2013 at 13:43
  • What I really want to do is group files by binary similarity - that's why I posted a link to binsort here.
    – Anderson Green
    Commented Jan 8, 2013 at 17:42
  • Also, there is a program called diff which can be used to find the difference between two files. Also, I'm mainly concerned with text and source code files here.
    – Anderson Green
    Commented Jan 8, 2013 at 17:43
  • Also, it would probably be easier to compare files of just one file type, instead of comparing files of all types. I think text files and source code files would probably be the easiest to compare.
    – Anderson Green
    Commented Jan 8, 2013 at 17:51
  • I've narrowed down the question's scope to discuss only text files and source files, so the question is no longer too broad.
    – Anderson Green
    Commented Jan 8, 2013 at 17:53

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .