1

I need to back up ~0.5 TB of files (about 600,000 of them) from a Windows Server 2003 machine to a CentOS 6.4 machine across a 100MB network, and update them once a night.

The backup script would be hosted on CentOS. Will SlimFTPd server+standard CentOS ftp client, or Windows AD fileshare+Samba 4 client be faster for synchronizing the backup via rsync? I.e. once it's going, say there are about 15 GB of changes, which one will be faster at comparing the directory structure to mirror them?

Improve this question
3
  • Is this the only backup of those files? If it is, I'd strongly recommend doing incremental backups instead of just rsync, otherwise you may not be able to recover all the information you need.
    – Paulo Almeida
    Jul 14, 2013 at 22:49
  • @Paulo Almeida, why is that, if it creates a mirror? Is this not roughly the Linux equivalent of robocopy?
    – Kev
    Jul 15, 2013 at 11:35
  • If a file gets corrupted, or you accidentally overwrite it, and you only notice it after the rsync runs, now both the original and the backup are corrupted. Keeping history is one of the principles of good backup practice. Of course, as in everything related to backups, there's a balance depending on how important the data are to you. But for instance, slm recommended Backuppc, which keeps history without using too much space, by using hard links.
    – Paulo Almeida
    Jul 15, 2013 at 12:11

3 Answers 3

Reset to default
1

I would use rsync, because that only copies what has changed.

I would install cygwin with openssh and rsync on the windows server, and use rsync over ssh to make the backup, with a command such as this:

rsync -e ssh -var --progress --partial server:/cygdrive/c/myfiles $HOME/mybackup

The advantage over using either of your ftp or samba options is that in this case rsync runs locally on both the centos box and the windows server collecting file names+sizes+timestamps and (if necessary) checksums, and only (those parts of) files and folders that are changed (or new) will be transferred.

Cygwin can be downloaded from here http://www.cygwin.com/ Make sure you install openssh (server) and rsync.

And this probably works to enable the ssh server after installing windows: http://www.noah.org/ssh/cygwin-sshd.html

Improve this answer
1

Rsync is a completely appropriate solution here. @jellefoks' answer covers most of how to do it, I'd like to add that I use rsync to backup a 20+ TB NAS over 1GB to another NAS and it takes roughly 1.5 hours to do this nightly. So using 100MB it will obviously be slower but given your data set sizes it should be completely acceptable.

The only caveat with rsync is if the nature of the data is such that it's a lot of files, then it will obviously take longer, than if they're fewer larger files. There are some optimizations you can make with rsync to tell it to compare files based on timestamp and size vs. a checksum to improve it's performance.

Lastly I'd like to suggest using BackupPC. It gives you the option of using a variety of methods to backup, (Samba, rsync, etc.) but it's true value is the web based reports. These are extremely useful in seeing if a backup ran and how long it took.

   ss of backuppc report

Improve this answer
0
0

For this use case (daily mirroring), you should install the rsync daemon rsyncd from Cygwin, and not worry about ssh/ftp/samba. Here is a good how-to explaining setup on Windows and CentOS.

http://www.gaztronics.net/howtos/rsync-windows.php

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.