16

How do I diff the output of multiple commands? vimdiff can support up to four files, but diff itself seems to support exactly two files.

Is it directly possible with some variant of diff, or do I have to save the output of all commands to temporary files, pick one and diff the remainder with it?


Context:

I have to check the output of a certain command on multiple servers and see if they all agree. For the moment, just reporting if any differences are found seems good, but if possible, I'd like to be able to say: X% servers agree with each other, Y% with each other; or that server Z is the odd one.

I have a four-way multi-master LDAP setup, and I want to verify that the ContextCSN values for all four agree with each other.

So now I do:

#! /bin/bash

for i in {1..4}.ldap 
do 
    ldapsearch -x -LLL -H ldap://$i -s base -b dc=example,dc=com contextCSN > $i.csn; 
done
set -e 
for i in {2..4}
do
    diff -q 1.csn $i.csn
done

And check the error code of the script. Are there better tools for this?

Any tools that can be used on Ubuntu 14.04 welcome.

3 Answers 3

13

The tool to do this is Diffuse. It is also generally available from repos (at least in Debian and Arch, where I checked). It works as you would expect it to:

  diffuse file1 file2 file3 file4

and so on.

4
  • Can it be used for scripting?
    – muru
    Dec 24, 2014 at 14:19
  • I get a bunch of Python GTK errors, and the manpage on Ubuntu 14.04 says it's a graphical tool, as does Debian wheezy's manpage: manpages.debian.org/cgi-bin/…
    – muru
    Dec 24, 2014 at 14:23
  • This looks like a really good diff tool, will probably start using it myself. Thanks for the tip :)
    – Graeme
    Dec 24, 2014 at 14:52
  • Worth to mention it has Windows GUI (not only Linux, came here searching for that)
    – sarh
    Jan 21, 2022 at 16:29
8

The fdupes tool may be of some use to you here (should be in the repositories). If you have a large number of files to compare, you can use it to reduce the work you have to do by identifying which ones are already identical. As noted below, it only works with directory arguments. If you have all the files to diff in one directory you can do something like:

fdupes .

To identify files which are the same. Or:

comm -13 <(fdupes . |sort -u) <(find . -maxdepth 1 | sort)

To identify files which are unique.

The diff3 command may also be useful.

0
6

If you only want to see if the files agree, then you don't really need diff. Just use sha1sum or something like that.

for instance:

#!/bin/sh
command="foo;bar|baz"
for server in server1 server2 server3 server4 server5 server6: do
    echo $server $(ssh $server "$command" |sha1sum)
done | sort -k2 

This will give you a space-separated list of server,sha1sum pairs. Servers with the same sha1sum have the same output:

server1 55ca6286e3e4f4fba5d0448333fa99fc5a404a73 *-
server3 55ca6286e3e4f4fba5d0448333fa99fc5a404a73 *-
server6 55ca6286e3e4f4fba5d0448333fa99fc5a404a73 *-
server2 d961c3de6d6c99429806ae3d6d03f316a1168ac6 *-
server4 d961c3de6d6c99429806ae3d6d03f316a1168ac6 *-
server5 dcdc24e139db869eb059c9355c89c382de15b987 *-

You can do further processing to get a list of matching servers, for example:

#!/bin/sh
command="foo;bar|baz"
for server in server1 server2 server3 server4 server5: do
    echo $server $(ssh $server "$command" |sha1sum)
done | sort -k2 | while read srv sum; do
    if [ "$prevsum" == "$sum" ]; then 
        echo -n " "
    else
        echo
    fi
    echo -n $srv
    prevsum=$sum
done

which gives output such as:

server1 server3 server6
server2 server4
server5
1
  • While I don't think this is the answer most people stumbling upon this question will be looking for, this a really clever and elegant way of performing and presenting this check. EDIT: Ah, I suppose it's not so clever in that if s1 and s3 are the same but s2 is not, the output would imply that s1, s2 and s3 are all unique. Jul 3, 2023 at 14:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .