14

I'd like to use btrfs' send/receive feature for transmitting backup snapshots over a rather slow (initial seed of about 50-100GB, upstream bandwith ~1-2MBit/s) and unreliable (daily forced interruption on both ends) connection.

I see following requirements:

  • encrypted transfer (usually achieved by using an SSH tunnel)
  • robustness to interrupted connections

It seems ZFS is able to resume interrupted transfers automatically, similar to how rsync does. Does this also apply to BTRFS? The send/receive wiki page is not useful with respect to interrupted transfers. If btrfs would resume interrupted transfers, all I would have to do is using an SSH tunnel and resume if interrupted.

If not, I'd have to use some buffer in-between that make sure the btrfs-connection survives interrupts, or get both servers close to each other for seeding (which will be a problem with respect to added files that excel the daily transmission capacities and sending snapshots).

What will I have to consider for transmitting the seed and snapshots?

3 Answers 3

10

I've been using "btrfs send" for backups for a while. It does not handle resuming broken transfers. I've written a utility that does resume transfers, and automatically syncs snapshots in the local btrfs file system to Amazon S3. You're welcome to try it out to see if it helps, and I'd love any feedback! It's both on GitHub (https://github.com/AmesCornish/buttersink) and PyPi (https://pypi.python.org/pypi/buttersink/).

0
4

"send" to a file or over a pipe. "receive" with the file or pipe on the other end. With a questionable connection, you would likely do best with files and a transport like rsync.

1
  • 1
    right, it's even possible to split the file in multiple chunks. It may be better for rsync to resume an interrupted transfer (or use the --partial with rsync)
    – ascobol
    Jul 1, 2014 at 16:29
2

In addition to buttersink's S3 resume, btrbk supports:

Resume of backups (if backup target was not reachable for a while)

Edit: I've since learned that the transfer of individual snapshots isn't resumed with btrbk, however it will deal with the backup target going offline (restarting the transfer of the interrupted snapshot).

1
  • 1
    I'll definitely have a look at btrbk and how it implements resuming transfers, although I somewhat dislike the programming language choice -- but it seems to implement pretty much the feature set I require.
    – Jens Erat
    Feb 25, 2017 at 10:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .