15

Is there a utility (or some shell magic) that allows me to compare two block devices?

Details: I have one large (0.5 TB) RAID device that I've backed up to a slightly larger SATA device using dd. The device has several partitions on it, however I copy from the 'parent' device and not the partitioned device nodes (e.g., I copy /dev/hda and not /dev/hda1 for example).

I would like to verify that backup is good/correct.

I have thought about running md5sum across each device, however that won't give me an accurate result as the second device is slightly larger than the first and thus the extra bytes will change the hash.

Improve this question
0

6 Answers 6

Reset to default
15

I would use the command

cmp /dev/hda /dev/hdb

it will stop at the first differing byte, writing out its offset, or on EOF on the shorter one.

Improve this answer
5
  • I don't see in the manpage anything about stopping at EOF, is that documented somewhere?
    – Kevin
    Jan 24, 2012 at 18:54
  • In fact, the exit status says: "1: The files are different; this includes the case where one file is identical to the first part of the other." This seems to imply that it does not report them as identical if one is shorter.
    – Kevin
    Jan 24, 2012 at 18:56
  • @Kevin: from info cmp: "The 'cmp' command compares two files, and if they differ, tells the first byte and line number where they differ."
    – enzotib
    Jan 24, 2012 at 20:47
  • 1
    You can use the --bytes=LIMIT option, setting the LIMIT value to the bytesize of the smaller device to stop at the end of the smaller drive (or partition boundary) without having cmp exit with non-zero status.
    – rozcietrzewiacz
    Oct 30, 2012 at 9:26
  • if I have a copy /dev/loop0 of /dev/sda and I do changes to /dev/loop0, how could I add the new changes to /dev/sda ?? Thank you
    – Milor123
    Dec 30, 2016 at 14:43
3

You are looking for binary diff software

rdiff
is designed for backup and network transmition, look at rdiff-backup and duplicity for backup propuse

xdelta3
is an open-source binary diff, differential compression tools, VCDIFF (RFC 3284) delta compression.

bsdiff
is too!! memory hungry, is not suitable for block devices.

Improve this answer
2

Depending on how full the drives actually are, it may be fastest to mount the file systems and do 

diff -qr /mnt/root1 /mnt/root2

Otherwise, I would say something like this: 

diff -q <(dd bs=1M count=500K if=/dev/sda) <(dd bs=1M count=500K if=/dev/sdb)

Obviously figure out just how big you need to make bs and count to stop at (or just before if it's much more convenient) the end of the smaller drive.

Or, thanks to enzotib's comment, it looks like you could use cmp, but you'll need the -n flag to limit the number of bytes to the shorter of the two drives.

cmp /dev/sda /dev/sdb -n 500GB
Improve this answer
3
  • Unfortunately I'm unable to mount the device as its an NTFS volume, and I have a LiveCD without the right drivers on it. For the second command the subshells you use don't try to suck the whole device into memory before feeding it into diff?
    – alanxz
    Jan 24, 2012 at 18:42
  • 1
    @alanxz No, the diff subshells will not read the entire device before writing it. dd writes bs bytes immediately after reading it (or processing if specified, not here), and they write to fifos, which block until they're read.
    – Kevin
    Jan 24, 2012 at 18:52
  • I was thinking more at the shell level (or whatever implements the FIFO). But that answers my question!
    – alanxz
    Jan 24, 2012 at 18:55
1

There is an utility called bsdiff, which does exactly this kind of thing. It does it really fast, and it's able to generate a binary patch, if you need.

See this post for some explanation around it.

Improve this answer
2
  • bsdiff's job is more complex than simply compare two files, so I doubt it can be faster than the simpler cmp.
    – enzotib
    Jan 24, 2012 at 20:54
  • The link is broken.
    – Owen Pauling
    Jun 11, 2018 at 14:39
1

You can compare two differently-sized block devices using md5sum like this:

(assuming the smaller device is /dev/sda)

# md5sum /dev/sda && head -c $(blockdev --getsize64 /dev/sda) /dev/sdb | md5sum
Improve this answer
0

Consider running fsck on the backup. You may need to play some tricks with losetup to get loopback devices set on partitions embedded in the backup, but if the file check says the file structure is consistent the data itself is likely good.

If you want a full checksum, you can use loopback devices with offsets to look at the internal 'partitions' of the backup and md5sum each separately, but a forced fsck may give you a basic idea much faster than byte for byte compares or checksum computations.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.