5

If I have a sparse file representing a block device, then how can I efficiently restore the file over a network? Ideally, I'd use rsync -S host:file /dev/sdb, but rsync won't write to block devices. I am using ssh host dd if=file | dd of=/dev/sdb, but that's not efficient as the zero bytes will come over the network and be written to the device. Can I use tar -S or cp --sparse=always somehow? Can I do this without storing the file locally (even temporarily)? Can sshfs recognise sparse files?

migrated from serverfault.com Aug 3 '12 at 12:57

This question came from our site for system and network administrators.

3

I haven't tested it, but there is a write-devices patch to rsync, which would solve your problem. You can find the patch in the rsync-patches repository.

2

You can install the iSCSI Enterprise Target software and setup an iSCSI LUN from the sparse-file like so:

In /etc/iet/ietd.conf:

Target iqn.my.iscsi.test:disk1
        Lun 0 Path=/path/to/my/sparse_file,Type=fileio

Then initiate the target from the host you need to restore on. Since the target will show as a local device (eg. /dev/sdd), you can dd from that device to your local device.

  • How is this more efficient then what I have now? Once I use dd, won't it read and write the zero bytes? – Jayen Aug 3 '12 at 22:50
  • No more hassle from "middle man" programs. – Tim Aug 7 '12 at 16:01
0

Have you tried compressing/decompressing the data in flight? Batches of zeroes should compress well.

Actually, having thought about it again, it's very simple. On the remote host, create a pipe with mkfifo. scp/dd to that pipe as usual and cp --sparse=always from it to the actual target file.

Tested on my installation and it does produce a sparse file on output. Thanks for a nice question!

  • ssh will compress for me. that's not the issue. – Jayen Aug 3 '12 at 13:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.