How do I fix btrfs? [closed]

Question

I've trawled mailing lists and finally finished up on Ubuntu's btrfs page, and am left feeling that btrfs still does not have a full fixing utility (as indicated on their home page). Even though months ago it was slated for being the default for Oracle's Linux and is included in many distros.

So, in lieu of that, is there a troubleshooting guide somewhere about how to fix btrfs?

Failing that, will copying my backups over the top of my FS fix things up? (While deleting snapshots if necessary for space? Or to delete corruption?) Should I instead try to revert to a previous snapshot and then restore missing files from backup? Or restore missing files from my @ and @home snapshots?

Note: This is a general question. I have deliberately omitted my exact FS issues (for the moment); I want to find a general/canonical workflow and troubleshooting guide.

(Ok, ok - here's some more details ;)):

I powered off during a hung shutdown, and have been presented with system instability hence. The system will boot and run for some amount of time until it writes enough data and freezes. Last time I just had Thunderbird open. These require more hard resets and presumably more corruption. sudo btrfsck /dev/sda1 oscillates between a few errors - often first time of the form

root 338 inode 7861227 errors 1000
root 338 inode 7904568 errors 1000
root 338 inode 7955174 errors 400
found 46242054144 bytes used err is 1
total csum bytes: 43112400
total tree bytes: 2074640384
total fs tree bytes: 1889853440
btree space waste bytes: 547680627
file data blocks allocated: 110756974592
 referenced 68393684992
Btrfs Btrfs v0.19

oooo, now it's getty really fruity (I only expected to see parent transid verify failed here...)

parent transid verify failed on 14266105856 wanted 464223 found 464221
parent transid verify failed on 14266105856 wanted 464223 found 464221
Extent back ref already exists for 14261530624 parent 0 root 256 
leaf parent key incorrect 14261751808
bad block 14261751808
Extent back ref already exists for 66455355392 parent 0 root 2 
Extent back ref already exists for 66455257088 parent 0 root 2 
Extent back ref already exists for 14257274880 parent 0 root 2 
block 14262571008 rec extent_item_refs 2, passed 2
block 14262575104 rec extent_item_refs 1, passed 1
block 14262579200 rec extent_item_refs 1, passed 1
Extent back ref already exists for 14262579200 parent 0 root 257 
leaf 14263906304 items 50 free space 132 generation 464224 owner 2
fs uuid 7d049403-cf6e-4b52-a624-32051e1f5b2a
chunk uuid be6f8f93-320c-4465-85d6-f53907698c32
item 0 key (14263341056 EXTENT_ITEM 4096) itemoff 3944 itemsize 51
    extent refs 1 gen 464168 flags 2
    tree block key (8332576 1 0) level 0
    tree block backref root 257
item 1 key (14263345152 EXTENT_ITEM 4096) itemoff 3893 itemsize 51
    extent refs 1 gen 464168 flags 2
    tree block key (8332586 c 8332543) level 0
    tree block backref root 257
failed to find block number 14263525376

(All heavily summarised of course; I never wanted to overwhelm you with these details :))

And now my final execution leaves me with the familiar:

parent transid verify failed on 14265458688 wanted 464230 found 464221
parent transid verify failed on 14265458688 wanted 464230 found 464221
parent transid verify failed on 14265458688 wanted 464230 found 464223
btrfsck: root-tree.c:46: btrfs_find_last_root: Assertion `!(path->slots[0] == 0)' failed.

, including the optional random error at the end. Oh happy joy. Note that these verify failed's change as data is written to the drive.

Another random error:

btrfsck: disk-io.c:412: find_and_setup_root: Assertion `!(!root->node)' failed.

Answer 1

To help with answer:

parent transid verify failed on 14265458688 wanted 464230 found 464221

Can be fixed with:

$ btrfs-zero-log DEVICE

NOTE: data can be lost! First try and mount with:

$ mount -t btrfs -o recovery,nospace_cache,clear_cache DEVICE MOUNTPOINT

If cant mount data like it says "bad fs":

$ btrfs restore DEVICE DIRECTORY_TO_DUMP_DATA_TO

Here is a real, though difficult to follow, email I sent to clarify his solution. Hopefully you can make out this cryptic explanation:

excerpt of email

Re: Question: How can I recover this partition? (unable to find logical $hugenum len 4096)

Hugo Mills carfax.org.uk> writes:

On Mon, Aug 26, 2013 at 01:10:54PM -0600, Chris Murphy wrote:

On Aug 26, 2013, at 11:41 AM, Nick Lee nickle.es> wrote:

There was a discussion on IRC a few days ago that the problem with the tree root's bloco was likely the result of either an issue with the disk itself, or the chunk tree/logical mappings. I ran the chunk recover, looked over the errors it found, and hit write. (If it failed, I was going to run something photorec, loss of organization as a side effect.)

I can write something more clear after my flight lands tomorrow if you want.

I'm just curious about when to use various techniques: -o recovery, btrfsck, chunk-recover, zero log.

Let's assume that you don't have a physical device failure (which is a different set of tools -- mount -odegraded, btrfs dev del missing).

First thing to do is to take a btrfs-image -c9 -t4 of the filesystem, and keep a copy of the output to show josef. :)

Then start with -orecovery and -oro,recovery for pretty much anything.

If those fail, then look in dmesg for errors relating to the log tree -- if that's corrupt and can't be read (or causes a crash), use btrfs-zero-log.

If there's problems with the chunk tree -- the only one I've seen recently was reporting something like "can't map address" -- then chunk-recover may be of use.

After that, btrfsck is probably the next thing to try. If options -s1, -s2, -s3 have any success, then btrfs-select-super will help by replacing the superblock with one that works. If that's not going to be useful, fall back to btrfsck --repair.

Finally, btrfsck --repair --init-extent-tree may be necessary if there's a damaged extent tree. Finally, if you've got corruption in the checksums, there's --init-csum-tree.

Hugo.