from a sparse file to a block device over the network

Question

If I have a sparse file representing a block device, then how can I efficiently restore the file over a network? Ideally, I'd use rsync -S host:file /dev/sdb, but rsync won't write to block devices. I am using ssh host dd if=file | dd of=/dev/sdb, but that's not efficient as the zero bytes will come over the network and be written to the device. Can I use tar -S or cp --sparse=always somehow? Can I do this without storing the file locally (even temporarily)? Can sshfs recognise sparse files?

Answer 1

I haven't tested it, but there is a write-devices patch to rsync, which would solve your problem. You can find the patch in the rsync-patches repository.

Answer 2

You can install the iSCSI Enterprise Target software and setup an iSCSI LUN from the sparse-file like so:

In /etc/iet/ietd.conf:

Target iqn.my.iscsi.test:disk1
        Lun 0 Path=/path/to/my/sparse_file,Type=fileio

Then initiate the target from the host you need to restore on. Since the target will show as a local device (eg. /dev/sdd), you can dd from that device to your local device.

Answer 3

Have you tried compressing/decompressing the data in flight? Batches of zeroes should compress well.

Actually, having thought about it again, it's very simple. On the remote host, create a pipe with mkfifo. scp/dd to that pipe as usual and cp --sparse=always from it to the actual target file.

Tested on my installation and it does produce a sparse file on output. Thanks for a nice question!