2

I have a btrfs disk that suffered some damage from hardware failure and spits i/o errors when copying certain files. I ran btrfs scrub, and when it reported csum errors I offlined it and did a btrfs check --check-data-csum and it returned several dozen of the following lines:

mirror 0 bytenr 549766098944 csum 1874004453 expected csum 2335064354

As far as I know, using --backup has a good chance to fix this problem, and would be the first step to take in repairing the file system. However, this was my virtual disk storage for qemu, and I'm worried that the internal congruity of the virtual disks (especially the windows one) will be harmed if I do this.

The btrfs manpage mentions an --init-csum-tree flag alongside other dangerous commands. Is this a good excuse to use this, or do I have other options?

CentOS Linux 7, kernel 3.10.0-514.26.2.el7.x86_64

btrfs-progs version 4.4.1 release 1.el7

Disk is a WD red 6TB (5.5TiB) WD60EFRX, one 5.5TiB partition

Virtual disks are in .qcow2 format

1

If the checksums are bad, the data is probably bad, and clearing out the checksum tree (which is what --init-csum-tree does) will not fix that, it will just expose the bad data directly to userspace and prevent detection of any other bit rot in old data on the FS. Essentially, you only had one copy of the data on the disk, and that copy is corrupted, so you're past the point of needing to worry about data potentially being bad in those disk images, since there almost certainly is some data corruption. If you only got a dozen or so of those error messages, then there won't be much corruption (each one should correspond to 4-16KiB of data, since BTRFS does checksums at the block level) at least, so that's a good thing.

In this case, I would actually suggest using btrfs restore to pull the files off of the disk to a different location, or alternatively restoring from a backup. If you've just got a single disk and therefore no data replication, there's not much you can do when you get checksum errors short of restoring known good data to a new location.

  • Last backup was a few months ago, I'll pull what I can with restore and get the rest from backup. I checked the logs and I have exactly 192 checksum errors. That's only 1.5/5767168MiB, about 0.000026% corruption. Crossing my fingers... – PSpacer Aug 31 '17 at 16:25
  • Hate to be pessimistic here, but it's worth pointing out that that's about 0.000026% corruption in used blocks. BTRFS doesn't checksum unused space (because that would be a waste of time for almost everyone), so those are 192 blocks that failed checksums. If you're lucky, they still may be stuff like empty blocks inside the disk images though. – Austin Hemmelgarn Aug 31 '17 at 17:27
  • I guess it's a crapshoot until I boot it up, but if the corruption was caused by hardware failure elsewhere in the system (looks like memory) then the corrupted blocks were probably in use. – PSpacer Sep 1 '17 at 3:31
  • This analysis is correct for me too. I have corrected errors: 0, uncorrectable errors: 88647, unverified errors: 0 on a btrfs partition I was trying to resize when the board went OOM. I have a hours old backup I'll have to restore that because i'm getting tons of I/O errors and I think --init-csum-tree will just make things worse. – Avio Apr 9 at 11:18
1

Please note that there is a known issue with images of virtual machines on btrfs. So your data could indeed be ok. You should expect be more of these warnings/errors popping up in the future. https://www.spinics.net/lists/linux-btrfs/msg25940.html

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.