Initial problem
So, I mounted a subvolume of a new BTRFS pool as /home
. I thought I had copied everything to it before reformatting the old drive to add to the pool, but on later examination some files (all my multimedia) were missing. I resigned myself to rebuilding the collection.
Later on, I ran the built-in "Baobob" disk-usage utility. It showed more contents on /mnt/wd
than expected. I looked more closely. /mnt/wd/home/$USER/multimedia
still had all my "missing" files (and not what I'd put in ~/multimedia
since the move) (root of the BTRFS system has label wd
).
Looking back at ~/multimedia
, the old files were not there and the new files still were.
Commonality between missing files
The files that were initially missing had one thing in common. Before being copied to the new drive, the files that didn't go missing were on an ext4
that was mounted as /home
. The ones that did were all on an ntfs
partition of the old drive, mounted as /media/$USER/data
, with symlinks pointing to them from /home/$USER
. That was to accomodate a dual-boot that I have since stopped using. When copying everything to the new volume (with rsync
, in case it's relevant), I put all the files together in one subvolume instead of splitting them and symlinking (mv
of the contents of one directory into the other, after both rsync
s had completed). I did fiddle about with subdirectories a bit before settling on that result, but I ended up with just the one subvolume as best as I can tell. Before mounting the new subvolume as /home
, this has appeared to work. After, the view from /home
seemed to match the original /home
partition, and the view from /mnt/wd/home
seemed to match the merged version.
Initial configuration/status
My fstab
at this point was:
# <file system> <mount point> <type> <options> <dump> <pass>
# / was on /dev/nvme0n1p5 during installation
UUID=699ad207-3b40-4caa-9926-503830121327 / ext4 errors=remount-ro 0 1
# BTRFS pool
UUID=24f6bbb6-cccb-4f12-808c-195f18b19e05 /home btrfs defaults,autodefrag,subvol=home 0 2
UUID=24f6bbb6-cccb-4f12-808c-195f18b19e05 /mnt/app_drive btrfs defaults,autodefrag,subvol=applications 0 2
UUID=24f6bbb6-cccb-4f12-808c-195f18b19e05 /mnt/snapshots btrfs defaults,autodefrag,subvol=snapshots 0 2
# Swap file
/swapfile1 none swap sw 0 0
I have also mounted the root of the BTRFS volume as /mnt/wd
via sudo mount UUID=24f6bbb6-cccb-4f12-808c-195f18b19e05 /mnt/wd
.
Running sudo btrfs subvolume list .
from any of the mounted locations of any subvolume (or the root):
ID 638 gen 4492 top level 5 path home
ID 639 gen 4433 top level 5 path applications
ID 640 gen 4404 top level 5 path snapshots
ID 800 gen 4402 top level 640 path snapshots/home_2017-07-06
But ls ./$USER/multimedia
(and any of the other missing directories) shows entirely different contents when run from /home
or from /mnt/wd/home
.
qwertystop@dt /home $ ls $USER/multimedia
pics unprepared
qwertystop@dt /home $ cd /mnt/wd/home
qwertystop@dt /mnt/wd/home $ ls $USER/multimedia
3d ml_resource music osu! phone-ready pics Unprepared vids
Does anyone have the faintest idea what's going on here, or how I might reconcile these two? I had the idea of just copying everything from one to the other - but that would still mean that I'd have to make sure to do that for every new file, best-case.
Further experimentation
Snapshots
I tried making snapshots. No matter which mount point I use for creating the snapshot, and no matter where I put it, it always matches the version found in /mnt/wd/home
, not the version in /home
. I've tried:
sudo btrfs subvolume snapshot /home /home/snapshot
sudo btrfs subvolume snapshot /mnt/wd/home /mnt/snapshots/snapshot
sudo btrfs subvolume snapshot /home /mnt/snapshots/snapshot
And all of them are identical. Which is a plus for internal consistency, except it doesn't tell me why the inconsistency still exists when I try to access any files.
fstab
/mounting experimentation:
I have tried mounting the BTRFS root at /mnt/wd
in my fstab (line is , placed before the /home
line) instead of post-boot with mount
. My fstab
at this point was:
# <file system> <mount point> <type> <options> <dump> <pass>
# / was on /dev/nvme0n1p5 during installation
UUID=699ad207-3b40-4caa-9926-503830121327 / ext4 errors=remount-ro 0 1
# BTRFS pool
UUID=24f6bbb6-cccb-4f12-808c-195f18b19e05 /mnt/wd btrfs defaults,autodefrag 0 2
UUID=24f6bbb6-cccb-4f12-808c-195f18b19e05 /home btrfs defaults,autodefrag,subvol=home 0 2
UUID=24f6bbb6-cccb-4f12-808c-195f18b19e05 /mnt/app_drive btrfs defaults,autodefrag,subvol=applications 0 2
UUID=24f6bbb6-cccb-4f12-808c-195f18b19e05 /mnt/snapshots btrfs defaults,autodefrag,subvol=snapshots 0 2
# Swap file
/swapfile1 none swap sw 0 0
This produced no change.
I have tried not mounting the home subvolume at /home
in fstab
at all (commenting the line out). When I then uncommented the line and mounted /home
, the resulting mounted drive matched the one on /mnt/wd
, with the old files and not the new ones. So I think this might be something going wonky at boot time?
"Unchanged" files still diverge
The files that didn't go missing between the two versions are still being tracked separately by the filesystem. I looked at tail .bash_history
in each, and they were different.
Problem is with the mount point and the subvolume in combination?
I mounted the home
subvolume at /mnt/foo
(and retried, later, at /foo
) instead of /home
. It matches the old version. I made a snapshot of the home
subvolume, called it newhome
, and mounted that at /home
. It matches the new version (missing old files, present new files).
I added a new file to one of the directories that was only in the new version (still in newhome
). I changed back to mounting home
at /home
and rebooted. The file was gone. Changed back to using newhome
, the file was still there. Did the same in the other order (putting it in the home
version and checking newhome
), and with putting the file in one of the directories that had not differed. The results were the same - the file only existed in the subvolume that it had been put in, and only in the new version (which only appeared when mounted as /home
, but was different for each one). That rules out my newest idea, which was that it saw "the subvolume mounted at the /home
mount-point" as a distinct thing from whatever subvolume I actually mounted, for whatever reason.
I took an unrelated subvolume, added a subdirectory for my username, and mounted it at /home
. The result was that the OS still recognized my username/password, but regenerated all the config files and directories in it as though I were a new user - it did not use the "new version" of the /home
partition.
I think at this point I can summarize the situation. This does not help me solve the problem or determine a cause.
tl;dr
The subvolume named "home", and snapshots descended from it, are actually two subvolumes, one of which only appears when it is mounted as /home
, the other when it is mounted elsewhere or visible as a subdirectory of the root volume of the BTRFS device-pool. For unknown reason, the two diverged at some point no later than the first time I mounted it as /home
, such that the version mounted at /home
contained the contents of the old /home
partition which I had rsync
-ed over, and the version mounted anywhere else contained the filesystem as I had modified it after that initial rsync
(the replacement of several symlinks-to-another-partition with the files previously linked).
I don't know why this happened. I can confirm that it does not happen when a subvolume not descended from "home" is mounted as /home
, with no files from "home" copied in. I can confirm that it does happen with a small subset of files copied to the new subvolume. I will now test copying all files, with cp
instead of making a snapshot.
That worked, with one tricky bit that may point to the problem. When I attempted to cp -a --reflink=always ./*
from /home/qwertystop
to my new home subvolume, I got an error message per-file: "Invalid cross-device link." This did not appear when running the same command relative to /mnt/wd/home/qwertystop
. It would seem that the problem causes, is caused by, or otherwise relates to the computer reading one mount location as being on a different device than the other. This is, I suppose, technically possible - it's a BTRFS pool of three distinct devices.
When I switched to copying out of the /mnt/wd/home
version, it seemed initially to work, but light use quickly revealed that the duplication appeared in the new subvolume the same as in the old.
/
. Could you indicate what part of my post led you to think I was doing so? I don't see it, but if I've said so by mistake I want to fix that.