1

I'm looking for how to automatically backup a user's home directory in CentOs 7 to a remote host or NAS or just to ~/.snapshot. In some Linux setups, I have seen a .snapshot folder in the user's home directory (~/.snapshot/) that holds hourly, nightly, and weekly backups of their home directory (ie ~/.snapshot/weekly1 for a copy of what was in the user's home directory 1 week ago).

The /home/username/.snapshot/ directory would be read-only by the user. It's not a backup for the purpose of guarding against hardware failure. It's just nice to have the ability to recover a file from yesterday or this morning if you don't like the changes that have been made.

I have seen several related posts on stack overflow, but so far, I haven't seen a guide that explains the complete workflow.

This is what I know so far:

  1. Use rsync to copy the contents of a given folder to the remote host, NAS, or (~/.snapshot/hourly0)
  2. Create a shell script to execute the rsync command

#!/bin/bash sudo rsync -av --progress --delete --log-file=/home/username/$(date +%Y%m%d)_rsync.log --exclude "/home/username/.snapshot" /home/username/ /home/username/.snapshot/hourly1

  1. Change the permissions on the script to make it executable

sudo chmod +x /home/username/myscript.sh

  1. Use crontab to schedule the rsync command at the desired backup interval

  2. Somehow move hourly0 to hourly1 before running the scheduled hourly rsync

  3. Delete the oldest backup once rsync completes successfully

Are there any guides that cover how to do this? I don't understand how to automatically rename the folders as time goes on (ie weekly1 to weekly2), or how to delete "week10" if I decide to only keep weeks up to 9. Is this another cron job?

Update: After some more Googling, I've discovered that NetApp creates the snapshot folders. I just don't currently have a NetApp NAS. https://library.netapp.com/ecmdocs/ECMP1635994/html/GUID-FB79BB68-B88D-4212-A401-9694296BECCA.html

  • Welcome to U&L! We are happy to assist, but we aren't a scriptwriting service. Please show what you've tried, and explain how it did not work as you expected or intended, and we'll be happy to help guide you. – DopeGhoti Sep 18 '18 at 18:18
  • Would you please clarify what you would like to delete. An arbitrary deletion of the backups or just backups older than specific time? – user88036 Sep 18 '18 at 18:42
  • Er, backing up a user's /home into that directory doesn't seem like an actual backup... – jasonwryan Sep 18 '18 at 18:58
  • @jasonwryan: Why not? Taking a snapshot of the contents of a user's home directory at scheduled times seems like the definition of a backup to me. – Seth Sep 18 '18 at 19:22
  • 1
    @Seth. for the snapshots. Look here mikerubel.org/computers/rsync_snapshots/#Extensions they have complete codes to do hourly, daily, monthly back ups – user88036 Sep 18 '18 at 20:38
3

How about this guide:

1) create your script: create new file and call it myrsync.sh, copy/paste the lines below:

  #!/bin/bash
    sudo rsync -av --progress --delete --log-file=/home/your-username/Desktop/$(date +%Y%m%d)_rsync.log --exclude "/home/your-username/.folder" /home/data /media/dataBackup_$(date +%Y%m%d_%T)

Meaning of the flags:

 -av bit: 'a' means archive, or copy everything recursively, preserving things like permissions, ownership and time stamps. 
  -'v' is verbose, so it tells you what its doing, either in the terminal, in this case, in the log file.
  --progress gives you more specific info about progress.
  --delete checks for changes between source and destination, and deletes any files at the destination that you've deleted at the source.
  --log-file saves a copy of the rsync result to a date-stamped file on my desktop.
  --exclude leaves out any files or directories you don't want copied. In the command above, the .folder directory

  /home/data is the directory I want copied. /home/data copies the directory and its contents, /home/data would just copy the contents. 

  /media/dataBackup_$(date +%Y%m%d_%T) is the separate drive. Change this to whatever your backup location is. Note that `rsync` will name every sync differently based on day/time of sync

2) Save myrsync.sh in your ~$HOME and make it executable by typing:

sudo chmod +x /home/your-username/Desktop/rsync-shell.sh

You can now double click that .sh file, choose Run in Terminal, it will ask you for your password and run, then leave a log file on your desktop. Or, you can make a cron job to do it for you!

3) The cron job

Copy your myrsync.sh file to /root by typing:

sudo cp /home/your-username/Desktop/myrsync.sh /root

Then type:

sudo crontab -e

You'll see a line which reads: minute hour day month year command

Under that, type: 0 22 * * * /root/myrsync.sh > $HOME/readme.log 2>&1

This means:

The hour in military time (24 hour) format (0 to 23)
The day of the month (1 to 31)
The month (1 to 12)
The day of the week(0 or 7 is Sun, or use name)
The command to run
So at 22:00 (10pm) every day root will run the shell script, without prompting you for sudo password (because its running as root already).

Now press Control-X, then type "Y", then press Enter

In order to delete older back ups, one way of doing this is to create a file with the timestamp of every sync in it. For example add the following command after the command rsync in myrsync.sh

date +%Y%m%d_%T >> time.txt

Use the command find to delete backups that matches the timestamp e.g: add this command after the date +%Y%m%d_%T >> time.txt in myrsync.sh

find . -type f ! -newer /tmp/timestamp -delete

Or

find . ! -newermt $date ! -type d -delete

This will delete back ups before specific date/time.

More details and sample codes for hourly/daily/monthly backups can be found here

  • How do you age-off old backups? Like weekly1 to weekly2 or hourly1 to hourly2? – Seth Sep 18 '18 at 18:34
  • You don't have. It will create new folder at each sync and it will capture day/month/year and time in the newly created backup folder. See $(date +%Y%m%d) in the code above. the command will sync from /home to /media/HomeBackup_$(date +%Y%m%d) and $(date +%Y%m%d) will give new folder name with every sync. – user88036 Sep 18 '18 at 18:36
  • Then you need a separate cron job to delete backups older than a set amount of time? Or modify the shell script to delete the oldest backup once the rsync command completes successfully? – Seth Sep 18 '18 at 18:37
  • For the hourly backups, I'd like to keep the last three hours (hourly1, hourly2, hourly3) and delete any hourly older than 3. If the user logs out, then pause the hourly backups (so that I really have a backup of the last three hours that the user was active). For weekly, keep the last 2 weeks (weekly1, weekly2) and delete any weekly backups older than 2. Run weekly backups once a day meaning that the weekly1 backup is what was in the users's home folder 7 days ago. If no files have changed since the last backup, then I don't want to create a new backup or delete any of the old ones. – Seth Sep 18 '18 at 18:50
  • This needs a lot of coding. but one way of doing this is to create a file with the time-stamp in it. e.g date +%Y%m%d_%T >> /home/your-username/Desktop/time.txt then run the command find find . -type f ! -newer /home/your-username/Desktop/time.txt -delete – user88036 Sep 18 '18 at 19:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.