3

I have a Linux server (Ubuntu 16.04) where I would like to mount an external "storage box" where to periodically backup several terabyte of server's data (both high and low-frequency changing), possibly keeping track of user's attributes (owner/group/permissions).

The provider of these storage box offers the following protocols: FTP, FTPS, SFTP, SCP, Samba/CIFS, HTTPS, WebDAV

Which backup tool should I consider? If you suggest rsync, as I would have first to mount the data locally, which protocol should I prefer for the mounting ? Would it be possible to backup user's attributes (i.e. mapping users between server and host ) ?

2
  • You mean those storage boxes don't have rsync support natively (over ssh or directly)? Have they got at least ssh support? What's the storage technology on them? If ZFS/btrfs, you could possibly use zfs send/btrfs send. – Stéphane Chazelas Sep 8 '17 at 13:00
  • 1
    scp would suggest they have ssh. I'd be surprised if you couldn't use rsync without mounting the FS. Network file system will always be slower than a syncing method over a TCP connection. – Stéphane Chazelas Sep 8 '17 at 13:03
2

That really depends on the specifics of the network, and how the storage appliance is designed.

If you absolutely have to mount the storage directory as a filesystem, you're pretty obviously stuck using either SMB/CIFS or SSHFS (which just needs working SFTP support on the storage device). Of those two, SMB is almost always going to get better performance for anything except bulk transfers (and for bulk transfers, either one may win out). If you can get working NFSv4 support (and you absolutely should be able to if it's an Ubuntu box, regardless of what the OEM says), that will usually outperform both SMB/CIFS and SSHFS most of the time.

If you just want to throw data across the network, then your options aren't much better to be honest. SFTP and SCP are what I would probably use in your situation because regular FTP (and FTPS, which is FTP over SSL) is crap, and WebDAV (regardless of if it's over HTTP or HTTPS) was not designed for bulk storage scenarios (DAV stands for Distributed Authoring and Versioning, which pretty well describes the fact that it was designed as an alternative to FTP for pushing content to web servers). In an ideal situation for that, I would either use rsync (if I had lots of files to transfer), or possibly just netcat (if I needed to just copy one big file).

0

I recommend using duplicity (but i did not test it at the scale of several terabyte), which has a lot of backends and does incremental backups split up into chunks (by default 25 MB) and only needs to download the chuncks, which are actually required for a restore. I would suggest using it with scp/sftp as it will work fine.

If you need to backup as fast as your network is able to transfer files, none of these protocols are ideal. But for one big backup and then incremental daily (or even hourly) backups of only a few gb duplicity should work fine.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.