3

I am setting up a complete online backup for my (Ubuntu 20.04) computer using Duplicati (to restore to a new machine in case of theft, fire etc). I am currently deciding which parts of my home directory /home/drubbels/ to include - trying to get all irreplaceable personal files, while excluding as much as possible application data (which would just waste space, and is probably easier to re-download and reinstall from source than to restore from a backup).

Originally, I thought I should exclude all home folder files starting with . and include all the rest, but that turned out not to work - the Steam folder includes game data for all Steam games (shouldn't be in the backup), while the .minecraft folder includes such as things as single-player savegames (should be in the backup). In addition, I don't know whether important personal settings for some applications end up outside the home folder entirely (in /etc/, in /usr/...). Ideally, I would like for a restore to include my custom Nautilus bookmarks, my choice of terminal bell sound...

Obviously, the Steam and .minecraft game directories mentioned above are a rather specific example, and it may not be possible to give a 100% correct answer for the general case, but, broadly speaking, which directories should (and which shouldn't) be in a complete backup of a Linux install?

4
  • 5
    Pretty much everything in your home will be user-specific stuff and not "application data" you can re-download. There will be some default settings yes, but they would take up a tiny amount of space. I don't think this is worth bothering with: just back up the entire home dir.
    – terdon
    Commented Nov 14, 2020 at 18:46
  • Duplicate of these: askubuntu.com/questions/545655/… & askubuntu.com/questions/40992/…
    – oldfred
    Commented Nov 14, 2020 at 20:54
  • This is not an answer, it's a warning. Your /home folder contains a lot of your personal data, often including things like passwords for different applications. I hope you're encrypting the data before you upload it.
    – KGIII
    Commented Nov 14, 2020 at 23:00
  • 1
    @KGIII Not to worry - I'm using Duplicati's AES setting with a 64-bit passphrase.
    – Drubbels
    Commented Nov 15, 2020 at 10:21

2 Answers 2

3

I don't think it's possible to answer this. Application can store their data anywhere they want in the home directory so it's hard to tell what is important and what not. I'd say backup everything. The only obvious folders to omit are IMHO .cache and .local/share/Trash. Applications should save their data to .local, GNOME settings should be in .gnome (and KDE in .kde) but Firefox uses .mozilla, if you are using Wine it saves all data in .wine, flatpak applications store data in .var etc.

I don't know whether important personal settings for some applications end up outside the home folder entirely (in /etc/, in /usr/...)

Applications you run as "normal" user can't write outside of your home so they can't store data in /etc, /etc contains system configuration.

0

My recommendation for backups it to default to including everything and then trim down the fat as needed with an exclude list that carves out pieces that are not needed to be backed up. There are many little files that are not critical, but are often so small that it isn't worth the effort to exclude it. You can use programs like du from the command-line or GNOME's Disk Usage Analyzer if you prefer a GUI. My recommended version of du would be this:

du -xhad1 | sort -h

Which shows all the child folders of your current folder sorted by human-readable sizes. You'll probably find ~/.cache/ that can be safely removed and I also tend to include ~/Downloads/ in the exclusion list because it can get large, but everything can be re-downloaded later. And, of course, you can dig down deeper by finding the biggest folders and running the command inside of them. For example, I found that ~/.local/share/baloo stuck out as a 22 GB folder carrying a search index. Baloo is the KDE File indexing software for searching and locating content on your desktop. That should be safe to exclude and just let it re-index if you have to do a complete restore of your home folder later. It's also likely to change a lot possibly duplicating that index file for each change and quickly eating up disk space for your backups.

On that note, you might also want to compare backups between iterations and see if there are any large files/folders that keep getting backed up on every iteration because they were changed. Baloo stores all of it's contents in a single file and there are probably many other examples as well.

My favorite backup tools are ones like Dirvish and rsnapshot that keep files in a transparent directory tree, one per image, and use hardlinks to share files that have not changed between them. With these images, I can access them like normal files because that's what they are, but each image, which is kept in it's own folder, will share files efficiently between other image with a hardlink if the file hasn't changed. With that, I can use commands like du across them and it will only count a files size the first time it sees it. For example:

du -shc 2*
127G    20230706
3.7G    20231005
3.6G    20240105

This shows that the base image I backed up in 2023-07-06 took 127 GB of storage, but a few months later, there was only 3.7 GB of new content even though I know the whole backup should be over a hundred gigabytes. This is because du will avoid over-counting hardlinks. If I run du directly on that second image, I see it's total size with it's previously hard-linked files:

# du -shc 20231005
127G    20231005
127G    total

Effectively, du allows me to see the incremental disk growth of the backups. If I see a particular backup grow by a huge amount, I can compare the two and see where it came from.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .