There are a few binary editors that can show differences between files, but the ones I've tried (
vbindiff) don't seem to be able to detect and show insertions or deletions.
There are a number of binary diff utilities like
bsdiff that compute the difference between two files. But the purpose is to generate
patch files to be later applied to an original. You can't read those patch files to see what the differences are.
Something you can do, to visually see the difference, is to hexdump the files with one byte per line and
diff the results. The line numbers will tell you the offset in the file (though beware it starts at 1, not 0):
diff -u <(od -w1 -vAn -tx1 file1) <(od -w1 -vAn -tx1 file2)
od supports the non-standard