Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Diff is a great tool to display the changes between two files. But how to display the similarities of two text files (while ignoring the differences)?

I.e. sample input:

a:
Foo Bar
X
Hello
World
42

b:
Foo Baz
Hello
World
23

Pseudo output (something like this):

@@ 2,3
=Hello World

Just sorting both files and using comm is not enough, because in that case the line information is lost.

share|improve this question

5 Answers 5

up vote 12 down vote accepted

How about using diff, even though you don't want a diff? Try this:

diff --unchanged-group-format='@@ %dn,%df 
  %<' --old-group-format='' --new-group-format='' \
  --changed-group-format='' a.txt b.txt

Here is what I get with your sample data:

$ cat a.txt 
Foo Bar
X
Hello
World
42
$ cat b.txt 
Foo Baz
Hello
World
23
$ diff --unchanged-group-format='@@ %dn,%df
%<' --old-group-format='' --new-group-format='' \
  --changed-group-format='' a.txt b.txt
@@ 2,3
Hello
World
share|improve this answer
    
wow, nice one :) –  Marcel Stimberg Aug 23 '10 at 19:43
    
You can avoid embedding a literal newline like this: ...%df'$'\n''%<'... –  Dennis Williamson Aug 24 '10 at 2:54
1  
You can also do it like this: ... --unchanged-group-format="@@ %dn,%df%c'\012'%<" ... (Note the double quotes.) –  Dennis Williamson Aug 24 '10 at 3:12
    
Great stuff! I did not know these options, because I just looked at the diff man page ... –  maxschlepzig Aug 24 '10 at 18:17
    
I'm using diff --version diff (GNU diffutils) 2.8.1 And I get the following error: diff: conflicting output style options diff: Try `diff --help' for more information. –  Sujay Apr 21 at 12:09

I don't think there is a single command that does what you want it to do. You can try to combine the output of diff with grep, though. If your text files contain none of the characters |,<,>, the following gives you somewhat useful output:

$ diff --side-by-side a b | grep -n -v "[|<>]"
3:Hello                             Hello
4:World                             World
share|improve this answer
    
Try this: diff --width=155 --left-column --side-by-side a b | grep -n -v '|' | sed 's/ *($//' –  Dennis Williamson Aug 24 '10 at 2:48
    
that looks better -- but you have to include < and > in the grep to get also rid of added lines in either file. –  Marcel Stimberg Aug 24 '10 at 7:46
cat file1 | grep -fxF file2

-f means matche plain strings (not regexps, -x means only whole-line matches, -F means take 'patterns' (i.e. lines with -f and x) from the file named as its argument

share|improve this answer
    
Aren't -f and -F exchanged?. At least in my grep version is like that. I need to provide file2 input to -f argument, like cat file1 | grep -Fxf file2, and then works. –  Birei Dec 15 '12 at 20:01

Dick Grune wrote a family of tools for this kind of thing:

http://dickgrune.com/Programs/similarity_tester/

There are versions that parse the syntax of various languages, so that things like renamed variables can be seen as unchanged.

It is packaged as similarity-tester in Debian and Ubuntu.

share|improve this answer

comm can be used. man comm for all the options but you'll want to use comm -12 ... to show only lines that exist in both inputs.

As people have pointed out, you need to pass your input through sort first.

share|improve this answer
1  
Hm, that works only for common lines that are at the same line number in both files. –  maxschlepzig Aug 23 '10 at 15:02
2  
comm seems to be for sorted files only and not to give that useful output for the usecase of the OP. His example: $ comm -12 a b Hello World comm: file 1 is not in sorted order comm: file 2 is not in sorted order –  Marcel Stimberg Aug 23 '10 at 15:04
    
@maxschlepzig: you should sort your files before passing them to comm. –  Hemant Aug 23 '10 at 15:05
1  
By sorting you get rid of all information on the position of the common lines, though. You wouldn't sort files before comparing them with diff, either. –  Marcel Stimberg Aug 23 '10 at 15:17

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.