1

As an initial disclaimer, I'm in the camp of not wanting snapshots to include snapshots, Inception-style.

So for example if I have @ mounted at /, and @_snapshots mounted at /.snapshots, then since /.snapshots is a subvolume, it won't be included in snapshots of /, but will still show up under /. Which is nice.

But what about, say, @home_fsinatra, mounted at /home/fsinatra? Where "should" snapshots of that subvolume go? In a dedicated snapshot subvolume called, say, @home_fsinatra_snapshots, showing up at /home/fsinatra/.snapshots? Or instead somewhere in /.snapshots/... (though I'm not sure how that would be accomplished).

I'm very familiar with ZFS's inherently nested datasets and snapshots, and have also used Btrfs since the beginning but have avoided messing with snapshots until now. This is for a new system drive to be installed on Btrfs, that, like my current system on ZFS, will make practical use of logically nested subvolumes and snapshots (though possibly in a literal flat configuration for simplicity and "nested" via fstab). Snapshots will be manual as well as automatic via probably Snapper.

I'm not wedded to any particular idea or requirement, except Btrfs - and snapshotting some stuff (eg system and individual user homes) but not others (eg various tmp and cache dirs, /opt, /var, docker, etc.) My subvolumes also don't need to be flat, I'm just assuming based only on the zeitgeist that that's easiest.

(This is for Debian Bookworm BTW, amd64, on an SSD.)

1 Answer 1

1

This doesn't just seem to be an unpopular topic or question to answer, there's also just a lot of misinformation or at least poor understanding around this topic across Earth and the 12 people who care enough to write on the Internet about it.

More relevantly, the general concept seems to be needlessly complex as filesystem user-tooling design decisions, IMO.

I had initially answered my own question with an explanation of why the proposition made in my original question, might indeed be the "best" - if not only - way to do it. Specifically:

But what about, say, @home_fsinatra, mounted at /home/fsinatra? Where "should" snapshots of that subvolume go? In a dedicated snapshot subvolume called, say, @home_fsinatra_snapshots, [manually mounted] at /home/fsinatra/.snapshots?

But that isn't the only way. In fact the drawback of this approach became evident quickly: If you do frequent or automatic (or really any) snapshots - say of your home directory - then a bunch of cruft builds up very quickly under your home folder (specifically in the ~/.snapshots folder), that bog various utilities down - some in ways that are difficult or impossible to really avoid. Sure, you can exclude it with many utilities like rsync and backup programs, and with certain flags with other utilities - but not others. It doesn't present as a mere symlink, for example.

ZFS solves this conundrum in a brilliant way, with a virtual folder that is hidden by default. (Not just dot-hidden, I mean it will never show up in a directory list, so is not reachable unless specifically poked by name.)

ZFS presents snapshots of "subvolumes" (aka datasets or filesystems) under the subvolume, very much like the idea presented above. However by default, it is not discoverable unless you traverse to it manually by literal string. No utility or program will stumble into it by walking/scanning the filesystem, unless it was programmed to explicitly test for that exact named folder some reason.

But since Btrfs doesn't do it that way (and to be fair it's not very "standard"), I'd expect this "snapshot-under-the-volume" to have too many downsides for typical users. Even though, by being its own subvolume, it still avoids the "snapshot inception" problem. However, this could also still be accomplished with more lightweight symlinks, that most tools are already perfectly adept at detecting and avoiding unless the user asks not to.

So my new approach is to have a single snapshot volume, sibling to @ and @home (and others), that all snapshots go under. My bare Btrfs volume looks something like this:

@/                          -> /
@home~fsinatra/             -> /home/fsinatra/
@snapshots/                 -> /.snapshots/
@snapshots/@/               -> /.snapshots/@/
@snapshots/@home~fsinatra/  -> /.snapshots/@home~fsinatra/

Note that the first-level objects under @snapshots, are just regular filesystem folders, not nested subvolumes. In other words, for @snapshots/@/, @snapshots is a subvolume, but @ under it is just a regular folder. (I didn't know this was possible when I posted this question, but it made things easier.) When snapshots are created for @ or /, they are just directed to go under /.snapshots/@/

Each subvol gets it's own dedicated folder, with the same name, under @snapshots/. (All manually set up of course.)

There's no reason you have to mirror the subvol names and same flat layout under snapshots. You could instead mimic the mounted hierarchical structure. But that would look weird - and be hard to manage - to have snapshot folders mixed together with hierarchy folders, the latter which could grow into the hundreds or thousands. I think it's cleaner to explicitly define, for example, snapshots that are only under root (@), and each other subvol with it's own dedicated snapshot folder at the same level.

And again, if you really want snapshot folders nested in with the hierarchical filesystem, just symlink folders named .snapshots back to the appropriate place under /.snapshots. By being symlinks, it avoids most of the mentioned problems that heavyweight mounted folders presents.

It also makes sense (to me) to just mirror the subvol names 1:1 under /.snapshot. Then snapshots can just be named, say, YYYMMDD-HHMMSS and make perfect sense in context.

Huge caveat that you need to know about regardless of what strategy you adopt

If you want a truly flat subvol layout (whether or not you nest them via manually via fstab):

You shouldn't create the subvolumes on the mounted filesystem.

If you do, all the new ones will actually be nested under @. (Or whatever subvolume name you have mounted at /.)

What you need to do instead, is mount the underlying Btrfs partition itself - with no subvolume specified - to somewhere safe and out of the way. (Say, /mnt/btrfs-root.) I just do that in fstab for now, maybe permanently. That gives you easy access to the top level, where you can see nothing but @ and @home, and any other top-level subvols. So when you create your other flat subvolumes for snapshots and/or, say, user homes, make them relative to /mnt/btrfs-root/ (or whatever), rather than /. That way they'll be siblings to - not children of - the others including "root" subvol.

(Otherwise you'd have to specify, say, -o subvol=@/@snapshots in fstab, or it won't mount. It won't exist at @snapshots. This is the gotcha that sent me down this godforsaken rabbit hole.)

That concept may seem obvious once understood, and maybe it was always obvious to you, reader - but there's maybe only one other person on the entire internet who has ever written about it, to my knowledge - which saved me so much [further] headache, may Odin favor his or her soul in the afterlife.

May this help pay it forward for like three more people over the next ten years.

One more tip that deeply perplexed me and possibly others out there: On Debian Testing, when Calamares installs on Btrfs, it puts subvol entries in fstab named /@ and /@home. Whereas Ubuntu has always named them @ and @home, respectively (without the leading slash.) I tested it, booting with and without the leading slash, and in my case it worked either way. I speculate that the leading slash is just more precise, otherwise mount just assumes it's relative to the root of the specified volume when mounting.

5
  • Thanks for sharing, @jim! Would you mind clarifying the following? For @snapshots/@home~fsinatra/ -> /.snapshots/@home~fsinatra/, does the -> represent the symlink? If yes, what is your command to do that symlink? I imagine one would need to mount the btrfs root filesystem first to access @snapshot to create the regular folder @snapshots/@home~fsinatra/ to be able to symlink? Also, would that mean the btrfs root needs to be mounted all the time for the symlink to work?
    – Thomas
    Dec 8, 2023 at 22:36
  • @Thomas Reply 2/2: For this solution, symlinks only come into play as a convenience factor: To place virtual snapshot folders directly under the "real" folders getting snapshotted. IOW, since the snapshots for /home/kermit live so far away in the directory tree, it would be nice if instead they were also accessible there. Hence, a symlink named /home/kermit/.snapshots, that points to /.snapshots/@home~kermit. The reason for a symlink and not just putting the snapshots there, is because backup tools etc. are universally good at ignoring symlinks by default. Btrfs subvols, not so much.
    – Jim
    Dec 10, 2023 at 15:51
  • @Thomas Reply 1/2: No, in this particular solution, /.snapshot is a mounted btrfs subvolume, @snapshots. So it would need to already exist and be mounted in fstab, or via mount, etc. But everything underneath it is either a regular directory, or a snapshot sent to one of those subdirs, of a mounted subvolume. (eg sudo btrfs sub snap -r /home/kermit /.snapshots/@home~kermit/20231210-074053. @home~kermit folder could just as easily be @home/kermit - or arbitrarily anything - but in this case my subvols are flat, so I try to mimic that with the snapshot destination dir structure.)
    – Jim
    Dec 10, 2023 at 15:56
  • thanks for clarifying about mounting @snapshots first. I use a similar technique with symlinks after mounting @snapshots, but wasn't clear what you meant from your original post.
    – Thomas
    Dec 11, 2023 at 3:17
  • @Jim awesome summary of this topic! I use btrfs for a few years now, mostly auto configured from my OS but never really understood the practical implications of the different setups. Now I do, thanks to you :)
    – JAR
    Feb 17 at 12:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .