I've created an image backup for HDDA (partition) from HostA and saved on HostB

ssh -c blowfish user@ip-or-hostname "dd if=/dev/sda1 | gzip -1 -" | dd of=sda1.gz conv=sync,noerror

The HDD size is 200GB the image created is about 45GB.

Is the image size correct compared to the drive size? Does it compress at this amount about 77% compression?

My IDEA is:

I want to format the drive which is LVM2_member now and create it again as ext3 then I'll restore the image created to it so I can reach the data I'm looking for.

Thank you!

  • 1
    Note that the dd of=sda1.gz conv=sync,noerror makes no sense. Those conv=sync,noerror should have gone to the dd if=/dev/sda1 one. See gzip -l to get the uncompressed size. – Stéphane Chazelas May 22 '15 at 16:03
  • @StéphaneChazelas thanks for your many forums asked to use conv=sync,noerror in case the drive has errors I tried gzip -l compressed uncompressed ratio uncompressed_name 45575929856 1913412099 -2281.9% sde.img Something wrong you think? – Zaher May 22 '15 at 16:15
  • Yes, but you apply it to the dd that reads from the pipe, not the dd that reads from the disk. gzip stores the file size in a 32bit integer so gzip -l output is not helpful for files bigger than 4GB. That was a bad advise of mine sorry. – Stéphane Chazelas May 22 '15 at 16:27
  • @StéphaneChazelas no worries what is your best advise and command to use. I figure out a solution now I'm trying empty a partition and clone the damaged disk to this partition however after google some asked to use cat others rsync what you think? – Zaher May 22 '15 at 16:31
  • 2
    get rid of the sync,noerror on the of= side. best case it does nothing, worst case it will corrupt your data due to syncing incomplete pipe reads. – frostschutz May 22 '15 at 17:41

Compressibility of a disk image depends a lot on what kind of data is stored in there, how much of it is used or has ever been used (without being explicitely erased during the whole life of that drive).

In short, it's impossible to tell. 77% is completely plausible as are 0% (a disk full of videos/oggs) and 99% (an empty, recently erased with zeros disk).

Now, a few comments on your command:

ssh -c blowfish user@ip-or-hostname "dd if=/dev/sda1 | gzip -1 -" |
  dd of=sda1.gz conv=sync,noerror

The conv=sync,noerror are meant for reading from the disk. dd is a dumb applications, it just does fixed size reads from a source file and fixed-size write to a destination.

If the disk is failing, and a sector is failing, a fixed-size read of 512 bytes may fail and return nothing. If dd were to write 0 bytes on output for that sectory, that would mean a corrupted destination as the data would be shifted. What you want instead is to output 512 bytes worth of zeros to the destination to replace that failing sector. That's the conv=sync part.

Also, you don't want it to stop at the first error => conv=noerror.

Now, that's only useful for the dd that's reading the block device, your left side dd

Your right-side dd is reading from a pipe fed by the ssh process. Again, dd reads 512 bytes at a time from that pipe. What if ssh doesn't write its output in chunks of size that is not multiple of 512? Then, dd could end up reading incomplete 512 blocks from that pipe and because of conv=sync, pad them with zeros, corrupting the compressed file.

What you want here is:

ssh -c blowfish user@ip-or-hostname "dd if=/dev/sda1 conv=sync,noerror |
  gzip -1" > sda1.gz

Or use lzop instead of gzip which is a lot faster even if the compression is not as good and at least is able to record in its footer uncompressed sizes greater than 4GB.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.