1

I currently have two servers both have the exact same hardware, disks, etc.

One server (server1) is going to be the "main" server. It's basically a server with raidz2 that has SMB shares on it that people connect to.

The other server (server2) is configured the same as server1 (raidz2) but is only to be for backing up server1. It's meant to be an offsite backup in the event we lose server1 from disk failure, fire, water damage, etc.

I'm trying to figure out the best way to do the backups to server2.

At first, I was thinking something like rsync. This is trivial to set up in a cron, and I can have it go once a week.

Alternatively, I was thinking of something with zfs send/recv. My understanding is that ZFS can do "snapshots" so I thought it would be great if I can create snapshots/incremental backups without losing a lot of space. I feel like this could be more difficult to implement/prone to errors.

Are there any other alternatives?

As I said before, both servers are configured the same in terms of hardware and raidz2 layout. What would you all recommend for my current situation? Thanks in advance.

1
  • I think jlliagre gave you a great and detailed answer down below. I would also suggest you consider using mirrors (or even a triple mirror) vs raidz2. For 2 reasons: Rebuild times, and with drive costs dropping while the size keeps increasing, you'll find yourself in a better position to grow you're pool(s) down the road.
    – sleepyweasel
    Jan 23, 2017 at 22:22

1 Answer 1

Reset to default
4

I would use incremental ZFS send/receive. It should be more efficient than rsync as ZFS knows what has been changed since the previous snapshot without needing to explore the whole file system.

Assuming you want to fully backup a file system namen datapool/fs.

You first create a pool to store your backup on the destination server and a recursive snapshot on the source pool:

dest # zpool create datapool ...
source # zfs snapshot -r datapool/fs@snap1

then you send the whole data as an initial backup:

source # zfs send -R datapool/fs@snap1 | ssh dest zfs receive datapool/fs

Next week (or whatever period you like), you create a second snapshot on the source pool and send it incrementally on the destination on. That time, ZFS is smart enough to only send what has changed during the week (deleted, created and modified files). When a file is modified, it is not sent as a whole but only the modified blocks are transmitted and updated.

source # zfs snapshot -r datapool/fs@snap2
source # zfs send -ri snap1 datapool/fs@snap2 | 
            ssh dest zfs receive -F datapool/fs

Repeat the operation with incrementing the snapshot numbers each time you backup.

Remove the unused old snapshots on either servers when you no more need them.

If you have bandwidth constraints, you can compress/decompress data on the fly, for example with inserting gzip/zip commands in the pipeline or by enabling ssh compression. 

source # zfs send -ri snap1 datapool/fs@snap2 | gzip | 
            ssh dest "gunzip | zfs receive -F datapool/fs"

You might also leverage mbuffer get a steadier bandwidth usage, as described in this page:

dest # mbuffer -s 128k -m 1G -I 9090 | zfs receive datapool/fs

source # zfs send -i snap2 datapool/fs@snap3 | 
            mbuffer -s 128k -m 1G -O w.x.y.z:9090

Note: The zfs -r flag is not available with non Solaris ZFS implementations, see http://lists.freebsd.org/pipermail/freebsd-fs/2012-September/015074.html . In such case, don't use the -F flag on the target but instead explicitly rollback datasets. If new datasets have been created on the source, send them first independently before doing the snapshot + incremental send/receive.

Of course, if you have only one file system to backup without an underlying dataset hierarchy, or if you want to perform independent backups, the incremental backup is simpler to implement and should work identically whatever the ZFS implementation:

T0:

zfs snapshot datapool/fs@snap1
zfs send datapool/fs@snap1 | ssh dest zfs receive datapool/fs

T1:

zfs snapshot datapool/fs@snap2
zfs send -i snap1 datapool/fs@snap2 | 
            ssh dest zfs receive -F datapool/fs
2
  • Does that work well over WAN? The backup server is accessible over a VPN but will be limited in terms of speed. I thought with ZFS send/receive you send snapshots. The question is whether its smart enough to know how much to really send when only a couple files changed.
    – Robert Goulet
    Dec 23, 2014 at 22:17
  • 1
    The first backup will require the whole data to be copied but subsequent ones will only consist of the delta between snapshots, with a filesystem block granularity. I.e. only the part of a file that has changed will be transmitted.
    – jlliagre
    Dec 23, 2014 at 22:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .