I'm sure this has been been answered in bits and pieces on many sites, but somehow I can't seem to get this (as a whole) to work reliably.
Scenario: I have a server that I would like to do incremental backups on (I'm thinking daily).
The files and directories I want to backup are located in the directories:
The backup should be .tar.gz to
/mnt/backup/ directory, and the filename should be something like
After the .tar.gz is successfully created, the file should be sent to an off-site server using ssh.
Right now the server creates a FULL backup once a week, and I manually send it off-site at random intervals (the .tar.gz size is about 60 GB, so it takes a while...).
Ideally I would like to .tar.gz ONLY regular files and folders. Files starting with . and ~ should be excluded, as well as directories starting with .
There is one exception:
/srv/samba/[someshare]/ folders I have ".recycle" folders, that should be included in the backup, but the file exclusions should be applied.
/home/user/.hiddendirectory/ (excluded) /home/user/.hiddendirectory/myfile.txt (excluded) /home/user/myfile.txt (included) /home/user/.myfile.txt (excluded) /home/user/~myfile.txt (excluded) /home/user/mydirectory/myfile.txt (included) /home/user/.recycle/ (included) /home/user/.recycle/~tmpfile (excluded) /home/user/.recycle/.myfile (excluded) /home/user/.recycle/myfile.txt (included)
I get started by using find:
Please note that the following commands include non-standard operators (-not, -or), so if used, please ensure your system supports them
find /home -not -path '*/\.*' ! \( -name '.*' -or -name '~*' \) -mtime -1
This seems to find the files and folders I need. But immediately that I pipeline this to tar, it seems to ignore all the exclusions?
find /home -not -path '*/\.*' ! \( -name '.*' -or -name '~*' \) -mtime -1 -print0 | tar -zcvf /tmp/testbackup1.tar.gz --null -T -
I seem to get the correct result using the following:
find /home -not -path '*/\.*' ! \( -name '.*' -or -name '~*' \) -mtime -1 -print0 | tar -Avf /tmp/testbackup1.tar | gzip -c /tmp/testbackup1.tar > testbackup1.tar.gz
...but this seems overly complicated, and results in an empty tarball, although the tar has significant size? And none of these take into account the .recycle inclusion.
A bonus would be an encrypted tar.gz for added security.
Sending the file using SSH is the easy part.
My thoughts are to put this into a script file, that is executed by Cron.
The script would:
- Create the backup file (handling the naming procedure)
- Send the backup file over SSH to an off-site server
- Delete old files (e.g. 2 weeks) from the .recycle directories
The (in place) full backup would be executed once a month, and will also handle cleanup of old backup files (2 months).
So, in the end I would have 2 full backups and approx 60 incremental backups. Once a full backup is dropped, the incremental backups (refereing to that) would also be deleted. And all the backup files will be located on-site and off site.
Any advice (even totally new approaches) is really appreciated!
Thanks in advance!