10

I'm planning a backup strategy based on rsnapshot.

I want to do a full system backup excluding files and directories that would be useless for the restore to have a working system again. I already excluded:

# System:
exclude /dev/*
exclude /proc/*
exclude /sys/*
exclude /tmp/*
exclude /run/*
exclude /mnt/*
exclude /media/*
exclude /lost+found

# Application:
exclude /*.pyc
exclude /*.pyo

I wonder which other entries I can add to the exclude list without compromising the restored system. Talking about a "generic" Linux system, can you suggest further glob extensions, temporary directories, caches, etc. I can exclude safely?

11

First off, you should read up a little on rsync's include/exclude syntax. I get the feeling that what you want to do is better done using ** globs than * globs. (** expands to any number of entries, whereas * expands only to a single entry possibly matching multiple directory entries. The details are in man rsync under Include/Exclude Pattern Rules.)

That said, if you want to be able to restore the system to a known working state from the backup with a minimum of hassle, you should be careful with excluding files or directories. I use rsnapshot myself and have actually taken the opposite approach: include everything except for a few carefully selected directories.

So my rsnapshot.conf actually states (with tabs to make rsnapshot's configuration file parser happy):

interval backup NNN # pick your poison
one_fs 0
exclude /backup/**
exclude /dev/**
exclude /proc/**
exclude /run/**
exclude /sys/**
exclude /tmp/**
backup / ./

and very little else. Yes, it means I might copy a bit more than what is strictly needed, but it ensures that anything not intended as ephermal is copied. Because of rsnapshot using rsync's hardlink-to-deduplicate behavior, the only real cost to this is during the first run; after that, assuming you have a reasonably sized (compared to your total data set size) backup target location, it takes very little extra in either time or disk space. I exclude the contents of /backup because that's where I mount the backup target file system; not excluding it would lead to the situation of copying the backup into itself. However, for simplicity if I ever need to restore onto bare metal, I want to keep the mount point!

In my case I also cannot reasonably use one_fs 1; I run ZFS with currently ~40 file systems. Listing all of those explicitly would be a maintenance nightmare and make working with ZFS file systems a lot more involved than it needs to be.

Pretty much anything you want to exclude above and beyond the above is going to depend on the distribution, anyway, so it's virtually impossible to give a generic answer. That said, you're likely to find some candidates under /var.

  • 1
    exclude /somepath/* is perfectly fine in this case; it excludes everything in /somepath/, just as expected. You don't need ** because there's no need to look deeper when everything in /somepath/ is already excluded. – Martin von Wittich Nov 3 '14 at 11:15
  • Or just use exclude /somepath and ignore these directories altogether - not just their contents. – Frank Kusters Feb 10 '15 at 19:15
  • 4
    @spaceknarf That breaks mounting when you restore onto bare metal, because then the mount point doesn't exist. – a CVn Feb 10 '15 at 20:46
4

Most of what you are trying to do can probably be accomplished simply by using the one_fs setting. Set the filesystems you want to include in your backups, then use that setting to ignore the rest (proc, sys, dev, etc.). I'd include /lost+found because that directory should always be empty unless you've backed-up a corrupted filesystem, in which case you probably want a backup of anything that fsck recovered. Also, .pyc and .pyo should not really be in the root directory in the first place, so I'd remove those lines too. /tmp and /var/tmp are about the only remaining paths on a "generic" system which contain data that can be reliably excluded from backups. So maybe try something like:

one_fs 1

exclude /tmp/
exclude /var/tmp/
  • I didn't really mean /*.pyc and /*.pyc but system wide *.pyc and *.pyo, I fixed that. I'm not sure if one_fs set to 1 might exclude anything I want, though. – Paolo Dec 11 '13 at 20:41
  • 1
    What if a system package uses such files? – depquid Dec 11 '13 at 20:42
  • you are right, but I'm almost sure that every file .py will be recompiled automatically sooner or later. – Paolo Dec 11 '13 at 20:45
  • 3
    Perhaps, but on my system such files are installed by vendor packages. Which means that if the system is restored from backup, files that the package manager thinks are there will be missing. You asked about a solution for a "generic" Linux system, and I don't think it's safe to always assume that such files can be lost without causing problems. – depquid Dec 11 '13 at 20:50
  • a thing worth noting I forgot to say in the Q. is that bind mounts should be excluded as well to avoid data duplication. – Paolo Dec 11 '13 at 20:50
1

I find it is better to have a package list, the contents of /etc, /home, and any user/system data from /var and elsewhere. It is usually faster to reinstall the packages and copy back the working config.

  • Why would installing packages, which includes writing all system files as well as processing configuration and meta-data be faster than simply copying files? – depquid Dec 12 '13 at 17:15
  • It has been my experience that when a real backup is needed you also find out that you had not properly stored and documented all of the bits about a system. Focusing instead on recreation rather than restoration makes it easier, faster, and more often done. Obviously YMMV. – Sean Perry Dec 12 '13 at 20:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.