5

I'm looking for an incremental backup tool to use on a running, disk based linux system, such as ubuntu or the like, and a tool which is freeware.


Clonezilla can do accurate backups but only when the system is idle.

Acronis for linux can do accurate backups on a running system, but it's not freeware. There are methods that Acronis and BTRFS use to capture file system changes as it runs, so as to emulate the effect of stopping the OS while it is working. I'm looking for something that does this.

rsync, dump and many other ill-suited tools are suggested and even used for this purpose, but they can not be trusted to accurately capture a running OS. rsync, is fine when used on a static file-system, but not on a multi-threaded running file-system.

It is surprising to me how many are under the belief that one can do an accurate backup of a running file system just by copying it in some fashion or another. Having built a small multi-tasking engine some years ago I am well aware of the danger of one task polluting another's work. Only if a backup runs as an atomic task, with all other tasks stopped while it works, can it be assured to capture a 100% accurate restorable backup. There is nothing worse than having a backup that you rely upon, and believe will save you, only to have it corrupt when you try to use it.

I need this for a plain old desktop linux, and not for a virtual setup.

6

As you mentioned, BTRFS can do this. This is how I regularly backup my laptop (which has an uptime of 9 weeks, 5 days as I type this).

Within my BTRFS filesystem, I have subvolumes. The way that you split your data into subvolumes and how you nest them is unimportant here, so long as you aren't using the root of the filesystem to store data that you want to back up.

The following commands are to illustrate syntax and possibilities, I recommend wrapping them up in a script that runs as a cronjob or systemd.timer.

To snapshot a subvolume:

btrfs subvolume snapshot -r <source> <dest>

To serialise a snapshot:

btrfs send <snapshot>

To serialise a snapshot relative to an older one (i.e. differential):

btrfs send -p <start> <end>

To generate a diff, compressing on the fly, and sending to backup server, with "progress" monitoring:

btrfs send -p <start> <end> | \
    pv -bart | \
    pbzip2 --best | \
    ssh me@backup.server "cat > /backups/name.bz2"

To do similar, but re-create the BTRFS subvolumes on the backup server rather than just compressed BTRFS streams:

btrfs send -p <start> <end> | \
    pv -bart | \
    pbzip2 --best | \
    ssh me@backup.server "pbzip2 -d | \
    btrfs receive <target>"

To restore, apply your snapshots in order to a new BTRFS filesystem, via btrfs receive.


Here is more information about BTRFS Incremental Backups

  • You may also find this useful: events.linuxfoundation.org/sites/events/files/slides/… – Mark K Cowan Jul 28 '16 at 10:05
  • Regarding stability, I've been using this for over a year now with only one issue. That issue turned out to be due to a failing disk — BTRFS detected data corruption (checksumming FTW!) and I lost no data due to the early warning and regular differential snapshots :) In theory, you could probably use BTRFS snapshots to create a "live" DR-site, with near-instant failover, although there are much better ways to accomplish that. – Mark K Cowan Jul 30 '16 at 13:05
3

As @mark-k-cowan said you can do this with btrfs snapshots. btrfs is too immature for me. Using zfs and its snapshotting would be better. As @brady-dean said LVM snapshots are a filesystem-agnostic method. LVM snapshots can have a huge negative performance impact.

Acronis has a kernel module that works at the block layer to create snapshots. The Datto Block Driver is a GPLv2 (ahem "freeware") kernel module that does the same thing. (Disclaimer: I have never used it.)

-1

It's not really possible to clone a running machine. You could write a script to add specific files to a tarball. You could also use LVM snapshots to backup the system drive.

  • 1
    Actually it is possible as Acronis does it. – Elliptical view Jul 27 '16 at 15:12
-1

Use rsync (or a variant which name I shall remember shortly and edit in ...)

Do a full backup with it and then do incrementals (or differentials)

It is available on numerous platforms as well, allowing you to backup to and from several different OS

See also : https://serverfault.com/q/138287/146493

It should be fine for most of the files on a running syste, if not all (some being written to files will be truncated, but that is normal if you backup while running) (you should use dd also to save the extra sectors informations, such as the boot sectors, if you intend a full recovery) ... or as you said, use a fs with that capacity built in.

If you intend to backup a running database, use that db dumping facility first and backup this.

  • rsnapshot perhaps – roaima Jul 27 '16 at 12:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.