I have two ~1 GB files that I want to do a side by side diff of. Solutions I have tried:

  • diff - Works quickly but doesn't output side-by-side.
  • diff -y or sdiff - This outputs side-by-side but it outputs the entire file - not just the changes, so they are impossible to find.
  • icdiff - Just too slow (it's written in Python so no surprise there).
  • ydiff - Not actually tried but it is also written in Python so I assume it will also be very slow.
  • KDiff3 - Crashed.
  • Xcode FileMerge - Too slow (gave it a few minutes of non-responsiveness).
  • Beyond Compare - Max 500MB limit.
  • Meld - Made a little progress but far too slow to be useful.
  • vimdiff - Loads the files successfully and actually displays a side-by-side diff, but the colour scheme makes it unreadable, and normal things that you'd expect from a sane program do not work, e.g. pressing up to scroll up.
  • Generally I use vimdiff, but I've never tried it on such large files, so I can't say whether or not it will work or if so, what the performance will be like. That said, it might be worth a shot. – Andy Dalton Jul 27 '18 at 13:13
  • So vimdiff actually sort of worked! The only problem is it is vim and therefore impossible to use. Up and down arrows do nothing. You have to do some special command so the windows scroll together. Also the colour scheme is white text on a very light background so I can't actually read it anyway. – Timmmm Jul 27 '18 at 13:20
  • Give this Python script a try github.com/unhammer/diff-large-files – steve Jul 27 '18 at 13:52

The best I have so far is:

diff -y --suppress-common-lines --speed-large-files file1 file2

However unfortunately that doesn't show you any context lines.

I found another solution using grep which seemed ok but it uses a regex and is just too slow.


If you want context, you can pipe the (unsuppressed) output of sdiff (or diff -y) through egrep with before (-B) and after (-A) context:

sdiff -H -t file1 file2 | egrep -B1 -A1 '^.{64}[^ ]' 

The -t and the .{64} are there to get to the midway point of the default 130 character width (expanding tabs in the output so they don't mess up the count). You can specify a shorter width, but you'd have to adjust that offset accordingly:

sdiff -H -w 80 -t file1 file2 | egrep -B1 -A1 '^.{39}[^ ]' 
  • I mentioned the grep solution in my answer but a regex is too slow. – Timmmm Jul 28 '18 at 17:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.