I am using Rsync to do the incremental backups of the data, but for some reason after 3-4 proper incremental backups, rsync performs Full backup, occupying twice the space, then I quickly run out of space on the backup machine. The scenario is that cron performs the incremental rsync everyday, and then additional script removes certain backups , leaving:
- 7 daily backups
- 4 (approx.) weekly backups
- 12 monthly backups
- 5 annual backups
Here is the script I am using:
#!/bin/bash
TODAY=$(date +%Y-%m-%d)
DATADIR=/mnt/remote_ananda
BACKUPDIR=/media/backup/data
SCRIPTDIR=/media/scripts
LASTDAYPATH=${BACKUPDIR}/$(ls ${BACKUPDIR} | tail -n 1)
TODAYPATH=${BACKUPDIR}/${TODAY}
if [[ ! -e ${TODAYPATH} ]]; then
mkdir -p ${TODAYPATH}
fi
rsync -a --link-dest ${LASTDAYPATH} ${DATADIR} ${TODAYPATH} $@
${SCRIPTDIR}/deleteOldBackups.sh
And then the deleteOldBackups.sh looks like:
#!/bin/bash
BACKUPDIR=/media/backup/data
function listYearlyBackups() {
for i in 0 1 2 3 4 5
do ls ${BACKUPDIR} | egrep "$(date +%Y -d "${i} year ago")-[0-9]{2}-[0-9]{2}" | sort -u | head -n 1
done
}
function listMonthlyBackups() {
for i in 0 1 2 3 4 5 6 7 8 9 10 11 12
do ls ${BACKUPDIR} | egrep "$(date +%Y-%m -d "${i} month ago")-[0-9]{2}" | sort -u | head -n 1
done
}
function listWeeklyBackups() {
for i in 0 1 2 3 4
do ls ${BACKUPDIR} | grep "$(date +%Y-%m-%d -d "last monday -${i} weeks")"
done
}
function listDailyBackups() {
for i in 0 1 2 3 4 5 6
do ls ${BACKUPDIR} | grep "$(date +%Y-%m-%d -d "-${i} day")"
done
}
function getAllBackups() {
listYearlyBackups
listMonthlyBackups
listWeeklyBackups
listDailyBackups
}
function listUniqueBackups() {
getAllBackups | sort -u
}
function listBackupsToDelete() {
ls ${BACKUPDIR} | grep -v -e "$(echo -n $(listUniqueBackups) |sed "s/ /\\\|/g")"
}
cd ${BACKUPDIR}
listBackupsToDelete | while read file_to_delete; do
rm -rf ${file_to_delete}
done
What am I doing wrong that after 3-4 incremental backups rsync does not link the previous, but make the new FULL one ? Thank you