Easy incremental backups to an external hard drive

Question

For a while I used Dirvish to do incremental backups of my machines, but it is slightly cumbersome to configure, and if you do not carry a copy of your configuration it can be hard to reproduce elsewhere.

I am looking for backup programs for Unix, Linux that could:

• Incrementally update my backup
• Create "mirror" trees like dirvish did using hardlinks (to save space)
• Ideally with a decent UI

Answer 1

Try rsnapshot. It uses rsync and hardlinks and is incremental.

Answer 2

This crude -but functional- script will backup everything under the sun to your external hard drive under a hard link farm. The directory name is a timestamp, and it maintains a symlink to the latest sucessful backup. Think of it as a Time Machine sans the fancy GUI.

#!/bin/sh
DATE=`/bin/date +%Y%m%d%H%M%S`
RSYNC=/usr/bin/rsync
BASE=/mnt/externalhd
TARGET=$BASE/daily
$RSYNC -av --exclude $TARGET --exclude-from=/etc/backup/rsync.exclude --link-dest=$TARGET/latest/ / $TARGET/$DATE/
touch $TARGET/$DATE/
rm $TARGET/latest
ln -s $TARGET/$DATE $TARGET/latest

Set it up creating an empty $TARGET and symlink a dummy $TARGET/latest to it. Populate /etc/backup/rsync.exclude with lost+found, tmp, var/run and everything else you need to skip during backup, or go for --include-from if it fits you better; man rsync is your friend.

Proper sanity checks, error control, remote backup and pretty GNOME GUI are left as an exercise to the reader ;-)

Answer 3

The Backup-Comparison of backup tools at the Ubuntu-Stackexchange is not really Ubuntu-specific. Perhaps you get some suggestions there.

I recommend Restic. It does incremental backups by default, has a decent UI but doesn't use hardlinks.

---

Previously, I recommended DAR - the Disk ARchive program. It does not come with a GUI, but its config is easy to reproduce. It has great incremental backup support. It does not use hardlink mirror trees, but it has a convenient shell for navigating the filesystem view of different snapshots.

I don't recommend it anymore because Restic is faster, implements modern crypto and supports various targets (such as S3 object storage).

Answer 4

I use backintime, which is primarily targeted towards Gnome/KDE desktops. However, it can work from the commandline as well.

I describe backintime as a backup system with "poor man's deduplication".

If you were to write your own backup script to use rsync and hardlinks, you would end up with something similar to backintime.

• I use cron to kick off the backintime job once per night.
• As the documentation says: The real magic is done by rsync (take snapshots and restore), diff (check if somethind changed) and cp (make hardlinks).
• backintime can be configured with different schedules. I keep monthly backups for 1 year, weeklies for 1 month, and dailies for 1 week.
• backintime uses hardlinks. I have 130GB worth of data, and I back this up nightly. It only uses 160GB worth of space on the second drive because of the magic of hardlinks.
• Restoring data from the backup location is as simple as running cp /u1/backintime/20100818-000002/backup/etc/rsyslog.conf /etc/rsyslog.conf. You don't need to use the GUI.
• On the second drive, the initial copy was expensive (since you can't do hardlinks between two different filesystems), but subsequent copies are fast.
• I copy data from my primary filesystems to a second filesystem onto a second hot-swappable drive, and periodically rotate the secondary drive.

Answer 5

Rdiff Backup is really good http://rdiff-backup.nongnu.org/

Note that it is abandoned, with latest stable and unstable releases from 2009.

Answer 6

I've had some success with RIBS (Rsync Incremental Backup System)

It uses rsync so hardlinks are supported and can do incremental backups hourly, daily, weekly and monthly.

However, it is a PHP script only. To set up you need to edit the settings and then set up related cronjobs. It works, but it's not the most user friendly and requires PHP.

Answer 7

I've been using epitome for about a year now for deduplicated backups of my personal data . It has a tar like interface so it's quite comfortable for a unix user and setup is a breeze, at least, on OpenBSD. You can easily cron it to backup your directories on a daily basis, and it takes care of the deduplication of your data. You basically are left with a meta-file that you can use to restore your snapshot at a later date. As I said the interface is tar-like so doing a backup is as easy as:

# epitomize -cvRf 2010-08-16-home.md /home

Note that epitome is abandoned, only partial copy of website at https://web.archive.org/web/20140908075740/https://www.peereboom.us/epitome/ remains.

Answer 8

BackupPC sounds like it fits the bill. It manages a tree of hard links for dedupe and can backup many machines, or just the local machine.

Answer 9

NB: Obnam's author retired it in 2017.

---

Lars Wirzenius's obnam:

• Does deduplication when it backs up things, which means that backups are likely to take little space, potentially a lot more than simply hardlinking files.
• As the backups are with deduplication, every backup is "full", with no need of having incremental backups. It simply detects that not many things have changed and only does what is needed.
• Each backup is, effectively, a snapshot of your system, without the need to recover the last full backup and each incremental backup in turn to get the system to be restored.
• Contrary to bup (which is another strong contender with deduplication), obnam is able to delete previous backups to save space of unnecessary backups.
• Besides using the regular recovery methods of a backup program, there is a fuse filesystem that provides a view of obnam's backups as a plain filesystem and that can choose which snapshot/backup/generation to mount, which is super handy, as far as "user" interfaces go (given that we are in a Unix-related site, a flexible command line interface is highly valued).
• It supports encryption as an integral part of the backups (and not as an afterthought).
• It was written with support for remote backups in mind.

In my opinion, one serious contender for the Backup World Day (and not only that day).