Imagine that I have a disk that is 128GB in size. On this disk, only 12GB is used. 116GB of the disk is empty space containing all zeroes (0x00).
I want to take an exact snapshot of the disk such that it can be exactly reconstructed in its current state in the future. To save space in the compressed image I'll pass it through a fast compression algorithm like
I can do this with
pv or similar tools, like this:
pv /dev/sdb | lz4 > disk.image.lz4
Now I have a disk image file that is around, say, 10GB in size, but actually contains a full 128GB image complete with zeros - the zeros were just compressed out.
Now later I want to write this image back to the disk. Naturally I can do this:
lz4 -d -c disk.image.lz4 > /dev/sdb
However the problem here is that writing the image back to disk can take a long time, since it is writing everything back to the disk - even the zeros.
Suppose one of two things: either that I don't care whether the blocks that used to be zeros still are zeros on the copy, or if it is an SSD I might just use
blkdiscard to discard all blocks on the SSD prior to writing the image, in effect zeroing out the disk in a matter of seconds.
Question: Is there a tool that can read the source image block-by-block, detect zeros, and simply skip writing those blocks on the output device?
For example, if we were working with 1MB blocks, my ideal tool would read 1MB of data, check to see if it is all 0x00, and if not, write it to the same position on the destination. If the block is indeed all 0x00, then just skip writing it altogether.
Here's why this would be an advantage:
- Writing all blocks on the destination disk can take a very long time. Especially if we are working with a spinning hard drive >2TB in size but which only contains a relatively small amount of actual data.
- It's quite a waste to use up SSD write cycles writing 0x00 to the entire drive when only a relatively small portion of the drive might contain data we care about.
- Since the image is being decompressed while it is written, this does not impose any extra read I/O to the source device.
I'm thinking of writing a simple tool to accomplish this if it doesn't exist already, but if there is a way to do this already, what is it?
EDIT: To give a little more detail, one example use case for this would be backing up a hard disk partition that contains an activated software license. A simple file copy or even a filesystem-aware partition image is unlikely to restore properly depending on the activation scheme. (For example, if an authorization scheme stores data in unallocated space, in sectors that are deliberately marked bad in the file table even though they're not, within the MFT itself on NTFS, etc). Thus, a bit-for-bit copy of everything that's not all zeros would be necessary to ensure that restoring the partition would still yield a valid license.