18

Diff is a great tool to display the changes between two files. But how to display the similarities of two text files (while ignoring the differences)?

I.e. sample input:

a:
Foo Bar
X
Hello
World
42

b:
Foo Baz
Hello
World
23

Pseudo output (something like this):

@@ 2,3
=Hello World

Just sorting both files and using comm is not enough, because in that case the line information is lost.

22

How about using diff, even though you don't want a diff? Try this:

diff --unchanged-group-format='@@ %dn,%df 
  %<' --old-group-format='' --new-group-format='' \
  --changed-group-format='' a.txt b.txt

Here is what I get with your sample data:

$ cat a.txt 
Foo Bar
X
Hello
World
42
$ cat b.txt 
Foo Baz
Hello
World
23
$ diff --unchanged-group-format='@@ %dn,%df
%<' --old-group-format='' --new-group-format='' \
  --changed-group-format='' a.txt b.txt
@@ 2,3
Hello
World
  • 2
    You can avoid embedding a literal newline like this: ...%df'$'\n''%<'... – Dennis Williamson Aug 24 '10 at 2:54
  • 1
    You can also do it like this: ... --unchanged-group-format="@@ %dn,%df%c'\012'%<" ... (Note the double quotes.) – Dennis Williamson Aug 24 '10 at 3:12
  • Great stuff! I did not know these options, because I just looked at the diff man page ... – maxschlepzig Aug 24 '10 at 18:17
  • I'm using diff --version diff (GNU diffutils) 2.8.1 And I get the following error: diff: conflicting output style options diff: Try `diff --help' for more information. – Sujay Apr 21 '14 at 12:09
  • I was getting "error: diff: conflicting output style options diff" because I had a diff alias defined. Use which diff to see if this is your problem. – justinjhendrick Feb 28 '17 at 3:08
12
grep -Fxf file1 file2

-F means match plain strings (not regexps), -x means only whole-line matches, -f means take 'patterns' (i.e. lines) from the file named as its argument

  • 3
    Aren't -f and -F exchanged?. At least in my grep version is like that. I need to provide file2 input to -f argument, like cat file1 | grep -Fxf file2, and then works. – Birei Dec 15 '12 at 20:01
  • This is not worked for me. – Chaminda Bandara Mar 8 at 11:45
6

comm can be used. man comm for all the options but you'll want to use comm -12 ... to show only lines that exist in both inputs.

As people have pointed out, you need to pass your input through sort first.

  • 1
    Hm, that works only for common lines that are at the same line number in both files. – maxschlepzig Aug 23 '10 at 15:02
  • 2
    comm seems to be for sorted files only and not to give that useful output for the usecase of the OP. His example: $ comm -12 a b Hello World comm: file 1 is not in sorted order comm: file 2 is not in sorted order – Marcel Stimberg Aug 23 '10 at 15:04
  • @maxschlepzig: you should sort your files before passing them to comm. – Hemant Aug 23 '10 at 15:05
  • 2
    By sorting you get rid of all information on the position of the common lines, though. You wouldn't sort files before comparing them with diff, either. – Marcel Stimberg Aug 23 '10 at 15:17
6

I don't think there is a single command that does what you want it to do. You can try to combine the output of diff with grep, though. If your text files contain none of the characters |,<,>, the following gives you somewhat useful output:

$ diff --side-by-side a b | grep -n -v "[|<>]"
3:Hello                             Hello
4:World                             World
  • Try this: diff --width=155 --left-column --side-by-side a b | grep -n -v '|' | sed 's/ *($//' – Dennis Williamson Aug 24 '10 at 2:48
  • that looks better -- but you have to include < and > in the grep to get also rid of added lines in either file. – Marcel Stimberg Aug 24 '10 at 7:46
2

Dick Grune wrote a family of tools for this kind of thing:

http://dickgrune.com/Programs/similarity_tester/

There are versions that parse the syntax of various languages, so that things like renamed variables can be seen as unchanged.

It is packaged as similarity-tester in Debian and Ubuntu.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.