Btrfs: To enable snapshotting of subvolumes, do you need to create seperate subvolumes dedicated for snapshots, for each data subvolume?

Question

As an initial disclaimer, I'm in the camp of not wanting snapshots to include snapshots, Inception-style.

So for example if I have @ mounted at /, and @_snapshots mounted at /.snapshots, then since /.snapshots is a subvolume, it won't be included in snapshots of /, but will still show up under /. Which is nice.

But what about, say, @home_fsinatra, mounted at /home/fsinatra? Where "should" snapshots of that subvolume go? In a dedicated snapshot subvolume called, say, @home_fsinatra_snapshots, showing up at /home/fsinatra/.snapshots? Or instead somewhere in /.snapshots/... (though I'm not sure how that would be accomplished).

I'm very familiar with ZFS's inherently nested datasets and snapshots, and have also used Btrfs since the beginning but have avoided messing with snapshots until now. This is for a new system drive to be installed on Btrfs, that, like my current system on ZFS, will make practical use of logically nested subvolumes and snapshots (though possibly in a literal flat configuration for simplicity and "nested" via fstab). Snapshots will be manual as well as automatic via probably Snapper.

I'm not wedded to any particular idea or requirement, except Btrfs - and snapshotting some stuff (eg system and individual user homes) but not others (eg various tmp and cache dirs, /opt, /var, docker, etc.) My subvolumes also don't need to be flat, I'm just assuming based only on the zeitgeist that that's easiest.

(This is for Debian Bookworm BTW, amd64, on an SSD.)

Answer 1

This doesn't just seem to be an unpopular topic or question to answer, there's also just a lot of misinformation or at least poor understanding around this topic across Earth and the 12 people who care enough to write on the Internet about it.

More relevantly, the general concept seems to be needlessly complex as filesystem user-tooling design decisions, IMO.

I had initially answered my own question with an explanation of why the proposition made in my original question, might indeed be the "best" - if not only - way to do it. Specifically:

But what about, say, @home_fsinatra, mounted at /home/fsinatra? Where "should" snapshots of that subvolume go? In a dedicated snapshot subvolume called, say, @home_fsinatra_snapshots, [manually mounted] at /home/fsinatra/.snapshots?

But that isn't the only way. In fact the drawback of this approach became evident quickly: If you do frequent or automatic (or really any) snapshots - say of your home directory - then a bunch of cruft builds up very quickly under your home folder (specifically in the ~/.snapshots folder), that bog various utilities down - some in ways that are difficult or impossible to really avoid. Sure, you can exclude it with many utilities like rsync and backup programs, and with certain flags with other utilities - but not others. It doesn't present as a mere symlink, for example.

ZFS solves this conundrum in a brilliant way, with a virtual folder that is hidden by default. (Not just dot-hidden, I mean it will never show up in a directory list, so is not reachable unless specifically poked by name.)

ZFS presents snapshots of "subvolumes" (aka datasets or filesystems) under the subvolume, very much like the idea presented above. However by default, it is not discoverable unless you traverse to it manually by literal string. No utility or program will stumble into it by walking/scanning the filesystem, unless it was programmed to explicitly test for that exact named folder some reason.

But since Btrfs doesn't do it that way (and to be fair it's not very "standard"), I'd expect this "snapshot-under-the-volume" to have too many downsides for typical users. Even though, by being its own subvolume, it still avoids the "snapshot inception" problem. However, this could also still be accomplished with more lightweight symlinks, that most tools are already perfectly adept at detecting and avoiding unless the user asks not to.

So my new approach is to have a single snapshot volume, sibling to @ and @home (and others), that all snapshots go under. My bare Btrfs volume looks something like this:

@/                          -> /
@home~fsinatra/             -> /home/fsinatra/
@snapshots/                 -> /.snapshots/
@snapshots/@/               -> /.snapshots/@/
@snapshots/@home~fsinatra/  -> /.snapshots/@home~fsinatra/

Note that the first-level objects under @snapshots, are just regular filesystem folders, not nested subvolumes. In other words, for @snapshots/@/, @snapshots is a subvolume, but @ under it is just a regular folder. (I didn't know this was possible when I posted this question, but it made things easier.) When snapshots are created for @ or /, they are just directed to go under /.snapshots/@/

Each subvol gets it's own dedicated folder, with the same name, under @snapshots/. (All manually set up of course.)

There's no reason you have to mirror the subvol names and same flat layout under snapshots. You could instead mimic the mounted hierarchical structure. But that would look weird - and be hard to manage - to have snapshot folders mixed together with hierarchy folders, the latter which could grow into the hundreds or thousands. I think it's cleaner to explicitly define, for example, snapshots that are only under root (@), and each other subvol with it's own dedicated snapshot folder at the same level.

And again, if you really want snapshot folders nested in with the hierarchical filesystem, just symlink folders named .snapshots back to the appropriate place under /.snapshots. By being symlinks, it avoids most of the mentioned problems that heavyweight mounted folders presents.

It also makes sense (to me) to just mirror the subvol names 1:1 under /.snapshot. Then snapshots can just be named, say, YYYMMDD-HHMMSS and make perfect sense in context.

Huge caveat that you need to know about regardless of what strategy you adopt

If you want a truly flat subvol layout (whether or not you nest them via manually via fstab):

You shouldn't create the subvolumes on the mounted filesystem.

If you do, all the new ones will actually be nested under @. (Or whatever subvolume name you have mounted at /.)

What you need to do instead, is mount the underlying Btrfs partition itself - with no subvolume specified - to somewhere safe and out of the way. (Say, /mnt/btrfs-root.) I just do that in fstab for now, maybe permanently. That gives you easy access to the top level, where you can see nothing but @ and @home, and any other top-level subvols. So when you create your other flat subvolumes for snapshots and/or, say, user homes, make them relative to /mnt/btrfs-root/ (or whatever), rather than /. That way they'll be siblings to - not children of - the others including "root" subvol.

(Otherwise you'd have to specify, say, -o subvol=@/@snapshots in fstab, or it won't mount. It won't exist at @snapshots. This is the gotcha that sent me down this godforsaken rabbit hole.)

That concept may seem obvious once understood, and maybe it was always obvious to you, reader - but there's maybe only one other person on the entire internet who has ever written about it, to my knowledge - which saved me so much [further] headache, may Odin favor his or her soul in the afterlife.

May this help pay it forward for like three more people over the next ten years.

One more tip that deeply perplexed me and possibly others out there: On Debian Testing, when Calamares installs on Btrfs, it puts subvol entries in fstab named /@ and /@home. Whereas Ubuntu has always named them @ and @home, respectively (without the leading slash.) I tested it, booting with and without the leading slash, and in my case it worked either way. I speculate that the leading slash is just more precise, otherwise mount just assumes it's relative to the root of the specified volume when mounting.