13

I just made a backup of an entire hard drive (50GB) over ssh via: 

dd if=/dev/hda | buffer -s 64k -S 10m | ssh myuser@myhost "cat > ~/image.img"

What's now the best way to check the integrity of the file image.img, i.e. how to verify that everything is copied correctly?

Improve this question
2
  • Usually when you dd, it is always perfect. However, you can always run sha1sum to check the sha1 hashes of the original and backup.
    – darnir
    Commented Aug 19, 2012 at 7:06
  • 2
    "Usually… it is always perfect." That reminds me of that quote from The Naked Gun: "Doctors say that Nordberg has a 50/50 chance of living, though there's only a 10 percent chance of that".
    – Sparhawk
    Commented May 20, 2017 at 9:38

3 Answers 3

Reset to default
14

If the command terminated successfully, then the backup is correct, barring a hardware fault (which could equally affect any verification you might perform). It may later become incorrect if the hardware is faulty, but most storage hardware detects corruption.

There is one caveat here: in a pipeline, the shell doesn't report errors from the left-hand side. (This is because of a fairly common scenario where the right-hand side doesn't need to read all the data, e.g. some_command | head, and the left-hand side dies because its output is no longer wanted.) So here a read error from dd would be ignored. In bash, set the pipefail option to report errors from all parts of the pipeline.

Also, beware that dd bs=… ignores some errors and dd is often slower than alternatives. I recommend not using dd at all: it has no benefits to just copy a whole file. Contrary to what you might have read somewhere, dd is not a low-level disk access command with special property, there is absolutely no magic in dd, the magic is in /dev/hda.

shopt -s pipefail
set -e
</dev/hda buffer -s 64k -S 10m | ssh myuser@myhost "cat > ~/image.img"

Nonetheless, if you wish to check the backup, the best way is to take a cryptographic checksum on each side and compare them. For example:

ssh myuser@myhost "sha1sum image.img" &
sudo sha1sum /dev/hda

Check that the two checksums are identical.

Note that this tests whether the backup and the original are identical at the time of the check. Anything you change on /dev/hda, including mounting and unmounting a filesystem even without making any change (which will update a last mount date on many filesystems), will change the checksum. If you want to verify the integrity later, note down the checksum of the disk at the time of the backup somewhere.

Improve this answer
5
  • 4
    Note that if anything on /dev/hda has changed since the back up was made the hashes won't match.
    – bahamat
    Commented Aug 20, 2012 at 4:39
  • 1
    The benefit of a hash check is that false-negatives (error made in recording image is, by chance, repeated during the hash run, so that a fault goes unnoticed) is astronomically miniscule. If any hardware fault is present, it should overwhelmingly result in a failed hash check. Will get a false-positive failed check if the disk has been changed, as has been noted.
    – Steven Lu
    Commented May 11, 2014 at 17:07
  • @StevenLu No, the chance of correlation is fairly high, because hardware faults aren't uniformly random. In particular, RAM errors (which are the most common hardware faults that are observable in typical PCs) occur at specific bits.
    – Gilles 'SO- stop being evil'
    Commented May 11, 2014 at 18:14
  • Okay, probably... hash like this with a big grain of salt, sure, but OP was transferring over a network. It's just inherently unreliable, TCP notwithstanding. Besides, chances are probably that we will not be seeing the same exact access pattern (in physical memory).
    – Steven Lu
    Commented May 11, 2014 at 18:57
  • New related question seeking clarification about @Gilles first sentence.
    – Sparhawk
    Commented May 20, 2017 at 11:54
5

As darnir & Giles mentioned, the best thing to do is run cryptographic hashes immediately after the back up before anything has been altered on your source disk. If, however, you've used the disk since then the hashes will most likely not match. Even changing one byte on the disk will result in a completely different hash.

Although it's far less than ideal you can spot check the image by mounting it. On the system where the disk image is, run the following (create /mnt/disk if it doesn't exist or us an alternate location):

mount -o loop image.img /mnt/disk

You can then browse around in /mnt/disk and see all of the files. Check the sha1 hashes of critical files inside the image against the originals to verify their integrity.

Improve this answer
-1

Use command qemu-img

qemu-img check image.img
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .