I am trying to backup a disk image of a local system to a remote system, in a way I can automate by bash script at a later point. Backing up and restoring was going well until I introduced encryption, now I think I'm getting a backup but having difficulty restoring.

I'm hoping someone can spot what I'm doing wrong or whether what I'm trying to do isn't possible using these tools. To be more specific I am using:

  • dd - to image the disk
  • gzip - to compress the image
  • gpg - to encrypt the compressed image
  • ssh - to transfer the image to the:
  • Backup Controller ( which will be used to store and retrieve backups on. I will refer to it as remote because it is seperate to the local system I am aiming to back up.

Taking a step back to simplify it (without encryption) and to to show you what I did have working:

Backing Up:

dd if=/dev/sda | gzip --best - | ssh user@ dd of=/home/user/sys1-backup.gz.img

This backups up the local drive (/dev/sda) to the remote Backup Controller whilst compressing it with a high level of compression.

Restoring Backup:

When restoring I live boot to something like gparted or Turnkey Linux Core and run:

ssh user@ dd if=/home/user/sys1-backup.gz.img | gunzip -d - | sudo dd of=/dev/sda

This connects to a remote Backup Controller, pipes dd through gunzip to decompress the image and then restores it to the local disk successfully. That works.

But when I try to introduce encryption (pgp) things fail when restoring.

Backing up with Encryption:

dd if=/dev/sda | gzip --best - | gpg -q --symmetric --cipher-algo AES256 --yes --batch --passphrase MySuperSecurePassword | ssh user@ dd of=/home/user/servername-disk.gz.gpg

To the best of my knowledge I think this part (backing up) is working with encryption.

Restoring with Encryption - (which I think is failing):

ssh user@ dd if=/home/user/servername-disk.gz.gpg | gpg -q --symmetric --cipher-algo AES256 --yes --batch --passphrase MySuperSecurePassword | gunzip -d - | dd of=/dev/sda

Gives the output:

gzip: stdin: not in gzip format
0+0 records in
0+0 records out
0 bytes copied, 0.325412 s, 0.0 kB/s
gpg: [stdout]: write error: Broken pipe
gpg: DBG: deflate: iobuf_write failed
gpg: [stdout]: write error: Broken pipe
gpg: DBG: deflate: iobuf_write failed
gpg: [stdout]: write error: Broken pipe
gpg: filter_flush failed on close: Broken pipe

I'm new to using gpg and dd on the shell so if anyone can spot my mistake or inform me of limitations of the tools I'm using I would greatly appreciate it!

  • You don't seem to tell gpg to decrypt (-d or --decrypt) when restoring, which means it would instead encrypt it again. If this is the actual issue, I'd treat it as a typo.
    – Kusalananda
    Aug 8 '19 at 17:48
  • If /dev/sda is a live disk then you are highly likely to be creating faulty backups. In practical terms this means you are not creating any backups.
    – roaima
    Aug 8 '19 at 18:19
  • Simplify the problem. 1. What happens if you use echo this is a test as your source instead of dd ... - does that work? 2. What if you skip the ssh part and write to a local file (/tmp/disk.img for example)?
    – roaima
    Aug 8 '19 at 18:22
  • @Kusalananda I think you may be right! How daft of me to miss that, checking my notes I think that may be my mistake - back to more testing. Thank you.
    – user281618
    Aug 8 '19 at 18:48
  • 1
    Personally I wouldn't touch BTRFS since AFAIK it still has no ability to recover after unexpected power loss (for example). ZFS seems more stable in that regard but this is now way off scope for this question. I prefer ZFS or LVM, on top of LUKS on RAID, but I'm aware that this is no longer fashionable on the (b)leading edge
    – roaima
    Aug 8 '19 at 19:41

This seems like a perfect use-case for Rclone. Rclone can handle

At this moment, compression is still work-in-progress. In most cases, using gzip first shouldn't be a problem.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.