2

To restore an incremental backup we need to first restore the full backup then the incrementals in the same order then deleted files are removed. It is useless to restore files then delete them and that is a waste of time with big files. Is there a way to avoid that using tar?

6
  • tar does not support incremental backups, could you be a bit more specific? – schily Jul 13 '20 at 20:00
  • It does. I use it. Check the docs. – Eduardo Jul 13 '20 at 20:26
  • You must be mistaken, the official tar man page schillix.sourceforge.net/man/man1/tar.1.html does not mention incremental backups at all. You must be referring to something else. Which program do you have in mind? – schily Jul 13 '20 at 20:38
  • Check GNU tar docs section 5.2. I'm using tar (GNU tar) 1.30 and the option is there when issuing tar --help – Eduardo Jul 13 '20 at 20:45
  • This is a different program. GNU tar claims to support incremental backups since 1992, but fails to restore the incrementals in case that the differences are not trivial. I cannot recommend making backups with this program as it is too unreliable. I guess you like the guarantee that any backup will always restore correctly. You may like to search this portal for gnu tar incremental backup for detailled problem discussions. – schily Jul 13 '20 at 20:55
1

If you like to skip some of the files from being extracted, you would need to know their names before they would be extracted.

This would require you to completely read all tapes in advance to analyze the content. In other words: the time to restore would double. Given that today, the restore of the backups for a typical filesystem takes more than a day, this does not seem to be a useful alternative.

If you know on Monday, which files will be deleted on Tuesday, backups could be more efficient, but a backup from Monday only has the knowledge from Monday.

A backup is only as efficient as it's meta data set and GNU tar stores very few meta data. GNU tar only knows the names of all files from archived directories.

A typical backup utility stores a lot more meta data than GNU tar. An important information that is missing from the GNU tar archives is the list of inode numbers for all files. If you have inode numbers in the backup, you can track file renames. GNU tar cannot track renames due to the lack of meta data.

If you have a backup program that stores sufficient meta data, the state of the art method to restore incremental backups is the following:

  • start with an empty target filesystem

  • extract the initial full dump to the target filesystem and keep a data base that remembers which files are directories and which files are non-directories as well as the original inode number from the backed up filesystem and the new inode number used on the filesystem where the restore takes place.

  • extract the incremental follow up backups in the order they have been created.

  • with every file and directory in the current incremental that is restored, check it's inode number in the data base for the related new inode number. If there is no new inode number in the data base, this is a new file, if there is already a new inode number in the data base, and the current file name differs, rename the file to the current name.

  • if a directory and it's content is extracted from an incremental archive and a file/directory is no longer in the new state of that directory rename all these files/directories that curently are on the target filesystem but not on the directory just processed to a temporary directory and remember the temporary name.

  • If the current incremenal has been completely restored and there are remaining files in the directory with the temporarily renamed files, these files are files that have been removed in the original filesystem and thus need to be removed on the new target filesystem.

You always may need to be able to store more data in the new target filesystem than in the source for that reason. Because of the few meta data in the GNU tar backups, there is a tendency that GNU tar needs more space for the incremental backups and more additional space in the target filesystem than solutions that store more meta data.

2
  • It is obvious that the downvoting trolls have no clue. Otherwise, they would be able to write a useful cmment... – schily Jul 16 '20 at 10:35
  • You are mistaken. I am not the author of the original tar implementation and I do not have te arrogance to call my tar implementation tar. – schily Jul 30 '20 at 12:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.