Scheduled folder backup

Question

I'm looking for how to automatically backup a user's home directory in CentOs 7 to a remote host or NAS or just to ~/.snapshot. In some Linux setups, I have seen a .snapshot folder in the user's home directory (~/.snapshot/) that holds hourly, nightly, and weekly backups of their home directory (ie ~/.snapshot/weekly1 for a copy of what was in the user's home directory 1 week ago).

The /home/username/.snapshot/ directory would be read-only by the user. It's not a backup for the purpose of guarding against hardware failure. It's just nice to have the ability to recover a file from yesterday or this morning if you don't like the changes that have been made.

I have seen several related posts on stack overflow, but so far, I haven't seen a guide that explains the complete workflow.

This is what I know so far:

1. Use rsync to copy the contents of a given folder to the remote host, NAS, or (~/.snapshot/hourly0)
2. Create a shell script to execute the rsync command

#!/bin/bash sudo rsync -av --progress --delete --log-file=/home/username/$(date +%Y%m%d)_rsync.log --exclude "/home/username/.snapshot" /home/username/ /home/username/.snapshot/hourly1

3. Change the permissions on the script to make it executable

sudo chmod +x /home/username/myscript.sh

4. Use crontab to schedule the rsync command at the desired backup interval

5. Somehow move hourly0 to hourly1 before running the scheduled hourly rsync

6. Delete the oldest backup once rsync completes successfully

Are there any guides that cover how to do this? I don't understand how to automatically rename the folders as time goes on (ie weekly1 to weekly2), or how to delete "week10" if I decide to only keep weeks up to 9. Is this another cron job?

Update: After some more Googling, I've discovered that NetApp creates the snapshot folders. I just don't currently have a NetApp NAS. https://library.netapp.com/ecmdocs/ECMP1635994/html/GUID-FB79BB68-B88D-4212-A401-9694296BECCA.html

Answer 1

How about this guide:

1. create your script: create new file and call it myrsync.sh, copy/paste the lines below:

   #!/bin/bash sudo rsync -av --progress --delete --log-file=/home/your-username/Desktop/$(date +%Y%m%d)rsync.log --exclude "/home/your-username/.folder" /home/data /media/dataBackup$(date +%Y%m%d_%T)

Meaning of the flags:

 -av bit: 'a' means archive, or copy everything recursively, preserving things like permissions, ownership and time stamps. 
  -'v' is verbose, so it tells you what its doing, either in the terminal, in this case, in the log file.
  --progress gives you more specific info about progress.
  --delete checks for changes between source and destination, and deletes any files at the destination that you've deleted at the source.
  --log-file saves a copy of the rsync result to a date-stamped file on my desktop.
  --exclude leaves out any files or directories you don't want copied. In the command above, the .folder directory

  /home/data is the directory I want copied. /home/data copies the directory and its contents, /home/data would just copy the contents. 

  /media/dataBackup_$(date +%Y%m%d_%T) is the separate drive. Change this to whatever your backup location is. Note that `rsync` will name every sync differently based on day/time of sync

2. Save myrsync.sh in your ~$HOME and make it executable by typing:

   sudo chmod +x /home/your-username/Desktop/rsync-shell.sh

You can now double click that .sh file, choose Run in Terminal, it will ask you for your password and run, then leave a log file on your desktop. Or, you can make a cron job to do it for you!

3. The cron job

Copy your myrsync.sh file to /root by typing:

sudo cp /home/your-username/Desktop/myrsync.sh /root

Then type:

sudo crontab -e

You'll see a line which reads: minute hour day month year command

Under that, type: 0 22 * * * /root/myrsync.sh > $HOME/readme.log 2>&1

This means:

The minute
The hour in military time (24 hour) format (0 to 23)
The day of the month (1 to 31)
The month (1 to 12)
The day of the week(0 or 7 is Sun, or use name)
The command to run
So at 22:00 (10pm) every day root will run the shell script, without prompting you for sudo password (because its running as root already).

Now press Control-X, then type "Y", then press Enter

In order to delete older back ups, one way of doing this is to create a file with the timestamp of every sync in it. For example add the following command after the command rsync in myrsync.sh

date +%Y%m%d_%T >> time.txt

Use the command find to delete backups that matches the timestamp e.g: add this command after the date +%Y%m%d_%T >> time.txt in myrsync.sh

find . -type f ! -newer /tmp/timestamp -delete

Or

find . ! -newermt $date ! -type d -delete

This will delete back ups before specific date/time.

More details and sample codes for hourly/daily/monthly backups can be found here