What I did so far
I wanted to create a disk image of my server HDD partition and copy it to my desktop (client). I used the following command on the server:
dd if=/dev/md3 status=progress bs=500k | bzip2 --best > /mnt/client/image.bz2
/mnt/client/ is an sshfs mount to my client.
Unfortunately, my client doesn't have a static IP address and the disk image took way longer than expected. At 511 GB (of 1 TB) my ISP forced a new IP address upon me and both ssh connections broke. That is the ssh connection on which
dd ... | bzip2 > ... was called and the sshfs mount.
Please note that I need to do a disk image. Copying the files is not enough. And please also note that there is not enough space on my server to store the disk image so it needs to be directly saved to my client.
How do I resume the disk image?
How do you resume the dd image such that previously copied data doesn't get transferred again? Especially since the already transferred data is inside a compressed archive.
Keeping the compression is preferable since the bandwidth of my network connection seems to be the bottleneck. But it's not strictly necessary if that makes it easier.
For files, I would think of something like
rsync but that doesn't apply here.
I thought of decompressing my already copied (partial) archive, measuring its size in bytes, and using something like
dd if=/dev/md3 skip=[size of uncompressed partial image] iflag=skip_bytes
But I'm not sure how I would append the data stream to the existing archive. I figured just using
>> to append to the archive probably doesn't work. And also decompressing the archive might take some time which may or may not be necessary. But I don't know how else I would get the uncompressed size of the partial image.
Let's say I ditch the whole compression thing. At least, would
dd if=... skip=... >> /mnt/client/uncompressed_image work as intended?
How do I ensure "connection safety" when using ssh?
How do I ensure that my ssh connection doesn't break with a non-static IP address? Or at least how do I ensure that I could easily resume the dd command when it breaks?
In hindsight it might have been smarter to just do something like the following from my client:
ssh root@server "dd if=/dev/md3 | bzip2 --best -c" | dd of=image.bz2
That way I don't need the IP of my client for the sshfs mount. But it will still break the ssh pipe when my IP address changes. So that alone solves nothing.