35

Diff is a great tool to display the changes between two files. But how to display the similarities of two text files (while ignoring the differences)?

I.e. sample input:

a:
Foo Bar
X
Hello
World
42

b:
Foo Baz
Hello
World
23

Pseudo output (something like this):

@@ 2,3
=Hello World

Just sorting both files and using comm is not enough, because in that case the line information is lost.

Improve this question

5 Answers 5

Reset to default
33 
grep -Fxf file1 file2

-F means match plain strings (not regexps), -x means only whole-line matches, -f means take 'patterns' (i.e. lines) from the file named as its argument

Improve this answer
3
  • 1
    This is not worked for me.
    – Chaminda Bandara
    Mar 8, 2019 at 11:45
  • @ChamindaBandara I’ve tried it with both the Mac’s built-in (POSIX/BSD) grep, where it fails, and with GNU grep, where it works as advertised. People tend to forget that the two are different.
    – Frungi
    Dec 30, 2020 at 0:53
  • @ajgringo and frungi standard grep accepts the given options, which means the suggested edit which proposed a "GNU grep" qualification would have made this answer worse rather than better.
    – Jeff Schaller ♦
    Dec 30, 2020 at 3:35
26

How about using diff, even though you don't want a diff? Try this:

diff --unchanged-group-format='@@ %dn,%df 
  %<' --old-group-format='' --new-group-format='' \
  --changed-group-format='' a.txt b.txt

Here is what I get with your sample data:

$ cat a.txt 
Foo Bar
X
Hello
World
42
$ cat b.txt 
Foo Baz
Hello
World
23
$ diff --unchanged-group-format='@@ %dn,%df
%<' --old-group-format='' --new-group-format='' \
  --changed-group-format='' a.txt b.txt
@@ 2,3
Hello
World
Improve this answer
5
  • 3
    You can avoid embedding a literal newline like this: ...%df'$'\n''%<'...
    – Dennis Williamson
    Aug 24, 2010 at 2:54
  • 1
    You can also do it like this: ... --unchanged-group-format="@@ %dn,%df%c'\012'%<" ... (Note the double quotes.)
    – Dennis Williamson
    Aug 24, 2010 at 3:12
  • Great stuff! I did not know these options, because I just looked at the diff man page ...
    – maxschlepzig
    Aug 24, 2010 at 18:17
  • I'm using diff --version diff (GNU diffutils) 2.8.1 And I get the following error: diff: conflicting output style options diff: Try `diff --help' for more information.
    – Sujay
    Apr 21, 2014 at 12:09
  • 1
    I was getting "error: diff: conflicting output style options diff" because I had a diff alias defined. Use which diff to see if this is your problem.
    – justinjhendrick
    Feb 28, 2017 at 3:08
12

comm can be used. man comm for all the options but you'll want to use comm -12 ... to show only lines that exist in both inputs.

As people have pointed out, you need to pass your input through sort first.

Improve this answer
4
  • 1
    Hm, that works only for common lines that are at the same line number in both files.
    – maxschlepzig
    Aug 23, 2010 at 15:02
  • 2
    comm seems to be for sorted files only and not to give that useful output for the usecase of the OP. His example: $ comm -12 a b Hello World comm: file 1 is not in sorted order comm: file 2 is not in sorted order
    – Marcel Stimberg
    Aug 23, 2010 at 15:04
  • @maxschlepzig: you should sort your files before passing them to comm.
    – Hemant
    Aug 23, 2010 at 15:05
  • 3
    By sorting you get rid of all information on the position of the common lines, though. You wouldn't sort files before comparing them with diff, either.
    – Marcel Stimberg
    Aug 23, 2010 at 15:17
8

I don't think there is a single command that does what you want it to do. You can try to combine the output of diff with grep, though. If your text files contain none of the characters |,<,>, the following gives you somewhat useful output:

$ diff --side-by-side a b | grep -n -v "[|<>]"
3:Hello                             Hello
4:World                             World
Improve this answer
3
  • Try this: diff --width=155 --left-column --side-by-side a b | grep -n -v '|' | sed 's/ *($//'
    – Dennis Williamson
    Aug 24, 2010 at 2:48
  • that looks better -- but you have to include < and > in the grep to get also rid of added lines in either file.
    – Marcel Stimberg
    Aug 24, 2010 at 7:46
  • +1 For a simple, visual, and self-explanatory solution.
    – sema
    Apr 9, 2020 at 9:50
4

Dick Grune wrote a family of tools for this kind of thing:

http://dickgrune.com/Programs/similarity_tester/

There are versions that parse the syntax of various languages, so that things like renamed variables can be seen as unchanged.

It is packaged as similarity-tester in Debian and Ubuntu.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.