I'm looking for some way to troubleshoot incremental backups with tar on linux.
I've got huge datasets (about 3TB) that need backing up to tape. For this I use the linux tar command and mt / mtx on LTO4 device.
As the backup is very very very very slow (I suppose I'll have to put that in another question) there is no choice for me but to backup with incrementals to avoid it running during production time.
basically, it goes like this:
A full tar is made: (with level 0)
tar --new-volume-script=changetape.sh \ --exclude=.zfs \ --listed-incremental=file.index \ --label=full_DS01 \ -cvf /dev/nst0 \ /mnt/storage/sharename >>filelisting.log 2>>errlisting.errlog
Incremental tar per day:
mtx -f /dev/sg1 load X #(load correct tape) mt -f /dev/nst0 eod #(forward to end of data to write a new incremental tar) tar --exclude=.zfs \ --listed-incremental=file.index \ --label=incremental_DS01 \ -czvf /dev/nst0 \ /mnt/storage/sharename >>filelisting.log 2>>errlisting.errlog
Repeat incremental tars. using the same process as in 2 so I always use the level 0 listed-incremental wich is adapted every time. This should mean that I always backup the files that have changed since my previous incrmental update. (as opposite to differential where you always compare with the full update)
Problem:
After a few iterations, the incrementals start to fail, meaning: the backup runs but it seems to think that ALL the files have been changed and it does a full backup in fact.
This happened in different datasets.
How can I troubleshoot this? And how is this possible? All seems to run well and then it thinks that all files have changed?
for clarity:
- the dataset is a readonly mountpoint
- the folder structure is the same as in the full backup
- the upper folder name has not changed
How can I solve this issue?
--no-check-device
? I did never use the feature for incremental dumps in tar. But I read this site few weeks ago. There is a information/warning that you may get "spurious redumping of unmodified files" in some cases and that you can avoid it by using this option.