I need to create a script, maybe bash, maybe just a .sh that checks for files changes on a root directory and sub-directories of the root, and adds to a new file. The tar.gz file name should have the creation date at the end (e.g. monitor-17.06.2013.tar.gz). I made this:

tar -zcvf /home/monitor.tar.gz /var/www/html/monitor --exclude /var/www/html/monitor/cache /var/www/html/monitor/log

and it works, but the directory /var/www/html/monitor/log isn't omitted, the file name doesn't contain the date, and this is missing the part that adds only new or changed files because I don't know how to do it.

EDIT: clarify a bit my needs Ok, due to some comments by users here I'll try to leave exactly what I'm looking for.

Day1: 2013-06-15
Directory: /var/www/html/monitor
Backup File Created: monitor-2013-06-15.tar.gz (contains all the content of /monitor folder)

Day2: 2013-06-16
Directory: /var/www/html/monitor
Files Changed: 

Files Added: 
Backup File Created: monitor-2013-06-16.tar.gz (contains **only** the files index.php, file1.php, file2.php and file3.php)

Day3: 2013-06-17
Directory: /var/www/html/monitor
Files Changed: 
Backup File Created: monitor-2013-06-17.tar.gz (contains **only** the file index.php)

The first backup always will have all the content but the rest should only have the modified files and the newest files, nothing else.

migrated from serverfault.com Jun 17 '13 at 22:01

This question came from our site for system and network administrators.


To insert the current date in the file name, call the date command in a command substitution.

tar -czf monitor-$(date -d %Y-%m-%d).tar.gz …

I recommend using a year-month-day format for the date for several reasons: the lexicographic order matches the chronological order; there's no risk of confusion between US month-day-year and rest-of-the-world day-month-year; it's an ISO standard.

To exclude several directories, you need to repeat --exclude for each of them: if you write tar -czf monitor.tar.gz --exclude dir1 dir2, then dir2 is a path to include in the archive, only dir1 is an argument to --exclude.

tar -czf monitor-$(date -d %Y-%m-%d).tar.gz --exclude /var/www/html/monitor/cache /var/www/html/monitor/log /var/www/html/monitor

To store files that have been modified after a certain date, pass the --newer option. You can specify either a reference file or a date. It is a lot more reliable to work with a timestamp file: create that file before each backup, and use the timestamp file from the previous backup as a reference for the next one.

touch monitor-backup.timestamp.new
tar … --newer=./monitor-backup.timestamp
mv -f monitor-backup.timestamp.new monitor-backup.timestamp

You seem to be rolling your own incremental backup system. This is surprisingly difficult to do reliably. I recommend using an existing backup application such as Duplicity.

  • +1 for mentioning Duplicity. Much easier then rolling your own! – slm Jun 18 '13 at 1:15
  • @Guilles I'm not trying to get my own incremental backup system I just want a new tar.gz file each time but only with files changed or added from a start point. See my edition at main post – Reynier Jun 18 '13 at 1:20
  • @Reynier What you are doing is an incremental backup (at the granularity of files). The tar options I gave you do everything you ask, but I recommend using a dedicated tool nonetheless. – Gilles Jun 18 '13 at 7:18
  • @Gilles let me see if I understood the process: 1 Create the first file by using tar -czvf monitor-$(date -d %Y-%m-%d).tar.gz --exclude /var/www/html/monitor/cache --exclude /var/www/html/monitor/log, 2 Create the timestamp file by using touch monitor-backup.timestamp.new, 3 Create the rest of files by using tar -czvf monitor-$(date -d %Y-%m-%d).tar.gz --exclude /var/www/html/monitor/cache --exclude /var/www/html/monitor/log --newer=./monitor-backup.timestamp, 4 Move the files by using mv -f monitor-backup.timestamp.new monitor-backup.timestamp is that right? – Reynier Jun 18 '13 at 13:42

As for the ommitted directory, you will have to use the --exclude option for each directory, it doesn't mean that it excludes everything after it. Some commands will take comma separated values, but not all of them.

tar -zcvf /home/monitor.tar.gz /var/www/html/monitor --exclude /var/www/html/monitor/cache --exclude /var/www/html/monitor/log

As for the part of appending the date, with your explanation of what you're looking for, I'm thinking that you can use:

-d, --diff, --compare
      find differences between archive and file system

edit: Here is a quick little script to do this. There is no error checking or anything, but it's a start.

NEW_FILE="monitor.`date +%d.%m.%Y`.tar.gz"
PREV_FILE={However you want to find this}

tar cf $NEW_FILE `tar --diff -f $PREV_FILE  | awk --field-separator=: '{print $1}' | uniq`
  • see my edition and for the --update I want to create some kind of difference to reduce the file size. Less said for example first file will have 20MB but second will have only 20kb because only two files was edited, this is what I want – Reynier Jun 17 '13 at 22:32
  • I added a part about using --diff which may be useful to you here. – pferate Jun 17 '13 at 22:53

Not the answer you're looking for? Browse other questions tagged or ask your own question.