Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I'm using rsync --link-dir to obtain daily snapshots with little space usage due to the hardlinking. It would however be nice to also have a directory that, while mirroring the original structure, only contains (hardlinks to the full backup of) the files that have actually changed. As an example, here's the directory structure before:

source/        <- contains the original data
backup/current <- symlink to the most recent snapshot
backup/12102?  <- recent daily snapshots (irrelevant)

and here the desired structure after:

source/
backup/current <- symlink to the _updated_ snapshot
backup/previous<- symlink to the snapshot that was was "current" before
backup/12102?  <- ...
backup/current.changes  <- (symlink to) directory containing _only_ the files that
                           have changed between "current" and "previous"
share|improve this question
add comment

1 Answer 1

up vote 1 down vote accepted

Here's the script I ended up using:

#!/bin/bash
set +x
SRC=$1
BCK=$SRC/.snapshots
CUR=$BCK/current
DAT=$(date +%Y-%m/%y%m%d)
DST=$BCK/$DAT
PAR="-aP --chmod=a-w,o-rwx --no-owner"

LNK="--link-dest=$CUR --link-dest=$DST.changeset"
CMP="--compare-dest=$CUR --prune-empty-dirs"
XCL="--exclude-from $BCK/.rsyncignore"
LOG="$BCK/log/$DAT"
LGP="--log-file $LOG"

mkdir -pm750 $DST.incomplete
mkdir -pm750 $DST.changeset.incomplete
mkdir -pm750 $(dirname $LOG)

rsync $PAR $CMP $XCL $LGP.change.log $SRC/ $DST.changeset.incomplete | tee $LOG.change.out
mv $DST.changeset.incomplete $DST.changeset
find $DST.changeset -type d -empty -delete
rsync $PAR $LNK $XCL $LGP.log $SRC/ $DST.incomplete | tee $LOG.out
mv $DST.incomplete $DST
rm -f $CUR
ln -s $DAT $CUR

It will create snapshots in a DST=$1/.snapshots/YYYY-MM/yymmdd fashion, and $DST.changeset will contain only the non-empty directories containing files that have been created or modified. Deletion (and indirectly mv) is not recognized, although a third rsync swapping $DST and $CUR would create the reverse-modlog, which then could be merged with the incomplete changeset maybe using diff syntax for some intuitive renaming... Or, you just parse rsync's log output or directly use git since you're basically versioning...


old answer:

My first thought was using a hardlink copy of the previous backup together with the --backup-dir parameter, but that would obtain the previous version of the files that have changed, which is the oposite of what I am looking for. The trick is inverting this:

  • First, create the usual snapshot using hardlinks:
    rsync -a --link-dest=/backup/current /source/ /backup/$TODAY.incomplete
    (You may have to use --no-owner --chmod=... etc. to make rsync definitely use hardlinks)
  • Now overwrite the new snapshot with the previous one, but let --backup move the changed files:
    rsync -ab --backup-dir=/backup/$TODAY.changelog.incomplete --delete /backup/current/ /backup/$TODAY.incomplete
    mv /backup/$TODAY.changelog.incomplete /backup/$TODAY.changelog
  • Congratulations, you screwed up the new snapshot at the expense of creating a changelog!
    Fix this (and cleanup) with
    cp -alf /backup/$TODAY.changelog/* /backup/$TODAY.incomplete
    mv /backup/$TODAY.incomplete /backup/$TODAY
    mv -f /backup/current /backup/previous
    ln -sf /backup/$TODAY /backup/current

Note that this "changelog" actually only contains changed or new files (the latter only if you didn't forget the --delete); neither deletions nor mvs are tracked. Some modifications may fix this...

share|improve this answer
    
In hindsight, parsing rsync --log-file=... might be both more efficient and (when applied to a cp -al $PREVIOUS $TODAY with --delete) actually track everything (mv would require something like git's rename detection) –  Tobias Kienzler Oct 25 '12 at 5:42
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.