0

I am running Debian Buster, and out of nowhere the file system of the root partition got corrupted. I wasn't doing anything out of the ordinary, just surfing the net basically, and at some point i find out that the root partition is remounted as read only.

I try rebooting, but grub starts into a minimal shell instead of its usual menu. So i boot into rescue mode from my Debian usb stick and fsck the root and efi partitions and a lot of errors are found but all seem to be successfully fixed. Badblocks doesn't find any issues, and the hard drive seems physically fine (no weird noises or anything like that).

Grub is working again, but when i try booting Debian in either normal or recovery mode, it kernel panics, saying it can't find libseccomp.so.2. And then I remember that fsck will put files in lost+found if it can't figure out where they were.

And there are a LOT of files there. Is there a way to recover all the system files from there automatically? Or should I just format and reinstall?

EDIT: Results of SMART check

# smartctl -HA -f brief -l xerror,error /dev/sda
smartctl 7.0 2018-12-30 r4883 [x86_64-linux-4.19.34-1-lts] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     PO-R--   100   100   050    -    0
  2 Throughput_Performance  P-S---   100   100   050    -    0
  3 Spin_Up_Time            POS--K   100   100   001    -    1712
  4 Start_Stop_Count        -O--CK   100   100   000    -    4003
  5 Reallocated_Sector_Ct   PO--CK   100   100   050    -    0
  7 Seek_Error_Rate         PO-R--   100   100   050    -    0
  8 Seek_Time_Performance   P-S---   100   100   050    -    0
  9 Power_On_Hours          -O--CK   042   042   000    -    23297
 10 Spin_Retry_Count        PO--CK   179   100   030    -    0
 12 Power_Cycle_Count       -O--CK   100   100   000    -    3471
191 G-Sense_Error_Rate      -O--CK   100   100   000    -    3116
192 Power-Off_Retract_Count -O--CK   100   100   000    -    73
193 Load_Cycle_Count        -O--CK   061   061   000    -    392741
194 Temperature_Celsius     -O---K   100   100   000    -    36 (Min/Max 8/48)
196 Reallocated_Event_Count -O--CK   100   100   000    -    0
197 Current_Pending_Sector  -O--CK   100   100   000    -    0
198 Offline_Uncorrectable   ----CK   100   100   000    -    0
199 UDMA_CRC_Error_Count    -O--CK   200   200   000    -    0
220 Disk_Shift              -O----   100   100   000    -    0
222 Loaded_Hours            -O--CK   052   052   000    -    19435
223 Load_Retry_Count        -O--CK   100   100   000    -    0
224 Load_Friction           -O---K   100   100   000    -    0
226 Load-in_Time            -OS--K   100   100   000    -    274
240 Head_Flying_Hours       P-----   100   100   001    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

SMART Extended Comprehensive Error Log Version: 1 (64 sectors)
No Errors Logged
1
  • tools like cruft (or cruft-ng) should be able to tell what's missing by comparing with installed packages, but you have to hope (or manage to) have the environment working for them (probably booting from rescue and doing chroot anyway)
    – A.B
    Commented Sep 25, 2019 at 8:49

1 Answer 1

0

Before any significant recovery efforts, you should check the SMART health status of your disk device, and backup any critical data. Your disk has already proven to be not completely reliable, so make sure your data is safe first. If the SMART data indicates the disk is less than healthy, it would be better to get a new one rather than attempt to prolong the suffering of the old one.

A good smartctl command for checking the health of e.g. disk /dev/sda would be:

smartctl -HA -f brief -l xerror,error /dev/sda

libseccomp.so.2 is supposed to be in /usr/lib/x86_64-linux-gnu/ (or equivalent for different hardware architectures). If that directory has been lost, that would be quite a big mess to fix as there are a lot if library files in it. Fortunately it might be as simple as telling the package management tools to verify any lib* packages and then to reinstall any that have files missing - at least in theory.

Since you have Debian, dpkg -V | grep -v ' c ' should check all files on the system that were installed from .dpkg packages and report on any that have been changed or are missing. The grep part excludes any configuration files from the listing. That's the most minimal useful tool for this situation I can think of; if you happen to have debsums installed, or can install it, debsums | grep -v 'OK$' could be used for the same purpose.

If you still happen to have enough undamaged libraries to allow this command to work,

apt-get install --reinstall $(dpkg -S $(debsums -c) | cut -d : -f 1 | sort -u)

will automatically reinstall any packages with changed non-configuration files. Using apt-get install --reinstall to re-install any packages that have some of their files damaged would be a much better option than trying to parse them out of lost+found.

3
  • Thank you for your advice, I ran the smartctl command from SystemRescueCD and I don't see anything that would suggest imminent failure. Though I have backed up my data jic, and will append the output to my post. As for the libraries, well, the /root, /srv, /tmp, /usr and /var directories are completely gone, so I'm starting to wonder if it's even worth the trouble trying to salvage this Commented Sep 26, 2019 at 3:57
  • Ouch. Even if all those files are just moved to lost+found rather than been lost completely, reinstalling will probably be both faster and easier than salvaging. The SMART info seems fine. Assuming your disk is connected with SATA, try running smartctl -l sataphy /dev/sda every now and then; if the error counters seem to increase, it might indicate a bad SATA cable.
    – telcoM
    Commented Sep 26, 2019 at 12:34
  • For now the only non-zero counters are: Transition from drive PhyRdy to drive PhyNRdy (7) and Device-to-host register FISes sent due to a COMRESET (6) Commented Sep 26, 2019 at 14:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .