6

In a recent Ask Ubuntu question on checking whether some files have differing content, I saw a comment stating that, if the differing sections didn't matter, cmp would be faster than diff. This Stack Overflow answer concurs, giving the reason that cmp would stop at the first differing byte. However, GNU diff has the -q (or --brief) flag, which is supposed to make it report only when files differ. It would seem logical that GNU diff would also stop comparing once any difference is found (like grep would stop searching after the first match when -l or -q is specified).

Is cmp truly faster than diff -q, in the context of Linux systems, which are likely to have the GNU version?

  • @josten I would, but I am not sure how to go about generating test data (and if I did, I'd like to test it out on a nice range to see whether cmp is faster for some file sizes). – muru Sep 2 '14 at 11:33
10

Prompted by @josten, I ran a comparison on the two. The code is on GitHub. In short1:

user-sys real

The User+Sys time taken by cmp -s seemed to be a tad more than that of diff in most cases. However, the Real time take was pretty much arbitrary - cmp ahead on some, diff ahead on some.

Summary:

Any difference in performance is pure coincidence. Use whatever you wish.

1The images are 1920x450, so do open them in a tab to see them in their full glory.

4

Using similar, but larger files from Anthon (100M lines, with a difference only on the last one):

yes | head -n 100000000 >aa
sed '$ s/d/e/' >ab

I get indistinguishable timings for diff -q and cmp -s:

/tmp% time diff -q aa ab
Files aa and ab differ
diff -q aa ab  0.04s user 0.33s system 99% cpu 0.370 total
/tmp% time cmp -s aa ab
cmp -s aa ab  0.04s user 0.36s system 99% cpu 0.403 total

cmp is slower than cmp -s. Presumably counting the line numbers is a significant burden.

/tmp% time cmp aa ab
aa ab differ: char 499999999, line 100000000
cmp aa ab  0.84s user 0.36s system 97% cpu 1.225 total

This is on Debian wheezy amd64, all running from RAM (on tmpfs).

cmp -s has the advantage of being supported by all POSIX platforms and by BusyBox.

2

No, diff -q seems to be faster and you can easily test that:

$ wc x1 x2
 10000000  10000000  50000000 x1
 10000000  10000000  50000000 x2
 20000000  20000000 100000000 total

Two files with 10 million lines of 4 chars each.

$ cat x1 x2 > /dev/null
$ diff x1 x2
9999999c9999999
< abcd
---
> abce

Differing only in the one before last line.

$ time diff -q x1 x2
Files x1 and x2 differ

real    0m0.043s
user    0m0.012s
sys     0m0.031s

$ time cmp x1 x2
x1 x2 differ: byte 49999994, line 9999999

real    0m0.085s
user    0m0.048s
sys     0m0.036s

diff -q is almost twice as fast in real time, and stays faster that way when using repeated execution.

  • 3
    You should compare with cmp -s. On my machine (Debian wheezy x86_64), cmp -s and diff -q have indistinguishable timings, cmp is slower. – Gilles 'SO- stop being evil' Sep 2 '14 at 21:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.