What I want:

I'm trying to back up many files to S3 each day. I want to compress these into archives daily so that each individual backup doesn't cost as much. Therefore, I want a backup system whereby I can create differential archives from the previous day to be backed up to S3.

My current reasoning is to create an initial backup, and then base backups per day that are differential wrt that initial backup.


Say I have a directory named dir containing a file, a.txt:


Then let's say I run rsync -r dir backup.1, or some other backup program, such as, but not limited to, rdiff-backup. Great stuff, I have a backup located at backup.1.

Now, I add one file, b.txt, so that folder's contents is as follows:


Is there an rsync, or another backup utility, command I can run that will give me a separate directory, backup.2, that contains only the file b.txt?

If I modify the permissions of a.txt, will that file also be included?

What I've tried:

I've tried rsync and rdiff-backup, with a range of flags, but I cannot get anywhere. I've been doing it for hours now, and I cannot figure it out.


Halfway there!

rsync -n -rpgov backup/ backup-copy/

Please note the -n means this does a dry-run,, so you can happily test this without wrecking your own local directories with file changes.

This outputs the difference between two local folders. However, I cannot for the life of me figure out how to get this output, through rsync's options, to a new directory.

I've tried using rsync's --compare-dest flag:

rsync -n -rpgov --compare-dest=backup/ backup-copy/

but this seems to output different files to the above command.How to push this over the edge?

  • Are the leading dashes part of the filenames, e.g. - a.txt ? Commented Sep 4, 2018 at 21:07
  • Is your rationale for having backup trees of changed files so that you save disk space? (If so, what's wrong with rsnapshot and its hard linked files?) How would your partial backup tree represent a deleted file or directory? Commented Sep 4, 2018 at 21:09
  • 1
    @roaima It isn't for saving file space. I create a lot of individual files (in the hundreds of thousands) that need to be backed up. We get charged in S3 per GET/PUT request, and it's becoming prohibitively expensive. Having one archive makes the process significantly cheaper
    – Nick Bull
    Commented Sep 4, 2018 at 22:03
  • @roaima The leading dashes are not part of the filename
    – Nick Bull
    Commented Sep 4, 2018 at 22:04
  • 1
    Thanks. Your comments about S3 costs would be really useful in your question as background information. It's the sort of information that can help direct someone to a most useful suggestion. Commented Sep 4, 2018 at 22:05

1 Answer 1


Here's the answer!

rsync -rpgov --compare-dest=/home/nick/docs/test/backup-copy backup/ backup-output/

Note the complete path, and the lack of trailing slash at the end of the --compare-dest flag. This made the difference!

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .