15

How do I diff the output of multiple commands? vimdiff can support up to four files, but diff itself seems to support exactly two files.

Is it directly possible with some variant of diff, or do I have to save the output of all commands to temporary files, pick one and diff the remainder with it?


Context:

I have to check the output of a certain command on multiple servers and see if they all agree. For the moment, just reporting if any differences are found seems good, but if possible, I'd like to be able to say: X% servers agree with each other, Y% with each other; or that server Z is the odd one.

I have a four-way multi-master LDAP setup, and I want to verify that the ContextCSN values for all four agree with each other.

So now I do:

#! /bin/bash

for i in {1..4}.ldap 
do 
    ldapsearch -x -LLL -H ldap://$i -s base -b dc=example,dc=com contextCSN > $i.csn; 
done
set -e 
for i in {2..4}
do
    diff -q 1.csn $i.csn
done

And check the error code of the script. Are there better tools for this?

Any tools that can be used on Ubuntu 14.04 welcome.

12

The tool to do this is Diffuse. It is also generally available from repos (at least in Debian and Arch, where I checked). It works as you would expect it to:

  diffuse file1 file2 file3 file4

and so on.

3
  • Can it be used for scripting? – muru Dec 24 '14 at 14:19
  • I get a bunch of Python GTK errors, and the manpage on Ubuntu 14.04 says it's a graphical tool, as does Debian wheezy's manpage: manpages.debian.org/cgi-bin/… – muru Dec 24 '14 at 14:23
  • This looks like a really good diff tool, will probably start using it myself. Thanks for the tip :) – Graeme Dec 24 '14 at 14:52
8

The fdupes tool may be of some use to you here (should be in the repositories). If you have a large number of files to compare, you can use it to reduce the work you have to do by identifying which ones are already identical. As noted below, it only works with directory arguments. If you have all the files to diff in one directory you can do something like:

fdupes .

To identify files which are the same. Or:

comm -13 <(fdupes . |sort -u) <(find . -maxdepth 1 | sort)

To identify files which are unique.

The diff3 command may also be useful.

2
  • fdupes seems to need a directory as argument (easily worked around), but yes, a count of files returned by fdupes is very useful. – muru Dec 24 '14 at 14:37
  • @muru, sorry, missed that. Updated. – Graeme Dec 24 '14 at 14:48
5

If you only want to see if the files agree, then you don't really need diff. Just use sha1sum or something like that.

for instance:

#!/bin/sh
command="foo;bar|baz"
for server in server1 server2 server3 server4 server5 server6: do
    echo $server $(ssh $server "$command" |sha1sum)
done | sort -k2 

This will give you a space-separated list of server,sha1sum pairs. Servers with the same sha1sum have the same output:

server1 55ca6286e3e4f4fba5d0448333fa99fc5a404a73 *-
server3 55ca6286e3e4f4fba5d0448333fa99fc5a404a73 *-
server6 55ca6286e3e4f4fba5d0448333fa99fc5a404a73 *-
server2 d961c3de6d6c99429806ae3d6d03f316a1168ac6 *-
server4 d961c3de6d6c99429806ae3d6d03f316a1168ac6 *-
server5 dcdc24e139db869eb059c9355c89c382de15b987 *-

You can do further processing to get a list of matching servers, for example:

#!/bin/sh
command="foo;bar|baz"
for server in server1 server2 server3 server4 server5: do
    echo $server $(ssh $server "$command" |sha1sum)
done | sort -k2 | while read srv sum; do
    if [ "$prevsum" == "$sum" ]; then 
        echo -n " "
    else
        echo
    fi
    echo -n $srv
    prevsum=$sum
done

which gives output such as:

server1 server3 server6
server2 server4
server5

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.