What is the best solution for daily backup of Linux folder to NAS server over WAN? Here is the situation: My customer has expressed a wish that source code for his project must be backed up every day on my local server and then exported to his NAS. He gave me access to NAS via FTP protocol, and now I want to use cron scheduler for ftp upload to his server. Is this OK, and how is this done?
2 Answers
As per comments rsync
is a good tool to use. Basic rsync usage simply mirrors a directory. For example:
rsync -a --delete /source/dir /backup/dir
Will make the backup directory match the source; if there is stuff in the backup that isn't in the source, it will be deleted (--delete
), and if there is stuff that is in both, it will be updated in the backup if the timestamp in the source is more recent (i.e., the file has changed).
Note you can also use rsync
via SSH if you don't have the remote directory locally mounted (and the NAS machine also runs an ssh server).
rsync -a --delete user@ip:/source/dir /backup/dir
This requires that you keep the mirror directory on your backup machine. If you want rolling backups, you could then archive and compress this:
tar -cjf backup.tb2 /source/dir
This can then be extracted with tar -xjf backup.tb2
. To prevent each backup from overwriting the last, you could use a timestamp:
tar -cjf backup.`date +%m%d%y`.tb2 /source/dir
This will produce a filename with a MMDDYY timestamp in it such as backup.030814.tb2
.
So, that's a two line script you can execute daily via cron.
The simplest approach, assuming you want to back up folder /foo
would be to create a simple little crontab
to run rsync
daily.
Create your
crontab
by runningcrontab -e
. In the editor window that will appear, add this line (assuming your NAS is mounted locally, you can usessh
if not):@daily rsync -glprtu /foo /path/to/NAS/mount
Close the editor window. That's it, your folder will now be backed up to your NAS daily.
The rsync
options used in this example are:
-g, --group preserve group
-l, --links copy symlinks as symlinks
-p, --perms preserve permissions
-r, --recursive recurse into directories
-t, --times preserve modification times
-u, --update skip files that are newer on the receiver
I suggest you read through man rsync
for more options, it is a very powerful tool. You might, for example, be interested in
--delete delete extraneous files from dest dirs
-z, --compress compress file data during the transfer
-C, --cvs-exclude auto-ignore files in the same way CVS does
UPDATE: If you need to do this via ftp, I recommend you mount his NAS locally first:
Install
curlftps
sudo aptitude install curlftpfs
Edit
/etc/fstab
and add a line for the NAS:curlftpfs#ftpUsername:ftpPassword@ftp://ftpUrl /localDirectory fuse rw,uid=1000,umask=0777,user,suid,allow_other,exec,auto,utf8 0 1
Then, point your rsync
command to the local mountpoint.
-
-
1@MarkoFrelih you'll have to mount the FTP server locally. See updated answer for a way to do that.– terdon ♦Mar 8, 2014 at 17:04
-
Wow, did not know that mounting FTP server locally is possible. Thanks for hint, now I am digging into docs and work! Thanks for very useful guideline! Mar 8, 2014 at 17:10
-
I've succesfully solved the problem, but the operation of copying itself is so slow that I think it is unusable! 20 megabytes were copied in 4 hours! Mar 10, 2014 at 11:25
-
rsync
command in a script and script insidecron.daily