Loss of disk access on external USB system/boot drive

Question

I'm running ubuntu 10.10 on a Dell D400 laptop. I'm booting and running the system from LVM volumes on a WD Scorpio Blue 2.5" drive mounted in a NexStar3 USB enclosure.

For the most part, this works great. However, after a few days or so the system will often seem to lose access to the system partition. I'm not sure what exactly happens, and it's difficult to diagnose because I can't access the pertinent system utilities:

$ mount
bash: /bin/mount: Input/output error
$ ls
Bus error
$ tail /var/log/messages
bash: /usr/bin/tail: Input/output error
$ cat /var/log/messages
Segmentation fault

Even shutting down cannot be done cleanly:

$ sudo shutdown -h now
bash: /usr/bin/sudo: Input/output error
$ sudo halt
bash: /usr/bin/sudo: Input/output error

The only recovery procedure I've found is a hard reboot by holding the power button down for a few seconds.

When this error state is active, the enclosure light is on steady. Normally it blinks pretty constantly.

I migrated the system files over from an internal drive. I was not having these problems prior to the migration. I seem to recall that they started soon afterwards.

update

Following Gilles' suggestion to do remote logging yielded this syslog output:

<6>Aug 12 01:00:25 box kernel: [229825.120148] usb 1-5: reset high speed USB device using ehci_hcd and address 2
<3>Aug 12 01:00:26 box kernel: [229825.700267] usb 1-5: device not accepting address 2, error -71
<3>Aug 12 01:00:26 box kernel: [229825.758145] ehci_hcd 0000:00:1d.7: port 5 reset error -110
<3>Aug 12 01:00:26 box kernel: [229825.758160] hub 1-0:1.0: hub_port_status failed (err = -32)
<6>Aug 12 01:00:26 box kernel: [229825.960116] usb 1-5: USB disconnect, address 2
<6>Aug 12 01:00:26 box kernel: [229825.980062] sd 2:0:0:0: Device offlined - not ready after error recovery
<6>Aug 12 01:00:26 box kernel: [229825.980108] sd 2:0:0:0: [sdb] Unhandled error code
<6>Aug 12 01:00:26 box kernel: [229825.980112] sd 2:0:0:0: [sdb] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
<6>Aug 12 01:00:26 box kernel: [229825.980118] sd 2:0:0:0: [sdb] CDB: Read(10): 28 00 31 b1 a0 35 00 00 20 00
<3>Aug 12 01:00:26 box kernel: [229825.980132] end_request: I/O error, dev sdb, sector 833724469
<3>Aug 12 01:00:26 box kernel: [229825.980167] Buffer I/O error on device dm-1, logical block 44564614
<4>Aug 12 01:00:26 box kernel: [229825.980171] lost page write due to I/O error on dm-1
<3>Aug 12 01:00:26 box kernel: [229825.984068] Aborting journal on device dm-4-8.
<3>Aug 12 01:00:26 box kernel: [229825.984100] Buffer I/O error on device dm-4, logical block 1081344
<4>Aug 12 01:00:26 box kernel: [229825.984104] lost page write due to I/O error on dm-4
<3>Aug 12 01:00:26 box kernel: [229825.984113] JBD2: I/O error detected when updating journal superblock for dm-4-8.
<3>Aug 12 01:00:26 box kernel: [229825.984181] Buffer I/O error on device dm-4, logical block 1675867
<4>Aug 12 01:00:26 box kernel: [229825.984185] lost page write due to I/O error on dm-4
<3>Aug 12 01:00:26 box kernel: [229825.984190] Buffer I/O error on device dm-4, logical block 1675868
<4>Aug 12 01:00:26 box kernel: [229825.984193] lost page write due to I/O error on dm-4
<3>Aug 12 01:00:26 box kernel: [229825.984200] Buffer I/O error on device dm-4, logical block 1684657
<4>Aug 12 01:00:26 box kernel: [229825.984204] lost page write due to I/O error on dm-4
<4>Aug 12 01:00:26 box kernel: [229825.984389] JBD2: Detected IO errors while flushing file data on dm-4-8
<43>Aug 12 01:00:26 box rsyslogd: /var/log/syslog
<43>Aug 12 01:00:26 box rsyslogd: /var/log/kern.log
<43>Aug 12 01:00:26 box rsyslogd: /var/log/messages
<2>Aug 12 01:00:26 box kernel: [229825.991754] EXT4-fs error (device dm-4): ext4_journal_start_sb: Detected aborted journal
<2>Aug 12 01:00:26 box kernel: [229825.991765] EXT4-fs (dm-4): Remounting filesystem read-only
<3>Aug 12 01:00:26 box kernel: [229826.136474] Buffer I/O error on device dm-1, logical block 44726233
<4>Aug 12 01:00:26 box kernel: [229826.136480] lost page write due to I/O error on dm-1
<3>Aug 12 01:00:26 box kernel: [229826.136487] Buffer I/O error on device dm-1, logical block 44726234
<4>Aug 12 01:00:26 box kernel: [229826.136491] lost page write due to I/O error on dm-1
<3>Aug 12 01:00:26 box kernel: [229826.144081] Aborting journal on device dm-1-8.
<3>Aug 12 01:00:26 box kernel: [229826.144099] Buffer I/O error on device dm-1, logical block 13139968
<4>Aug 12 01:00:26 box kernel: [229826.144103] lost page write due to I/O error on dm-1
<3>Aug 12 01:00:26 box kernel: [229826.144110] JBD2: I/O error detected when updating journal superblock for dm-1-8.
<0>Aug 12 01:00:26 box kernel: [229826.144124] journal commit I/O error

The 10th line from the end, at 229825.991765, is where the filesystem is remounted readonly. I scanned through the rest and didn't see anything indicating that it is fully dismounted, so it is a bit of a mystery why the system itself becomes inaccessible.

In any case, the USB errors at the beginning seem to be the reason for the crash.

Is this likely to be a problem with the laptop's USB controller/drivers? Or more likely with the USB enclosure? How can I tell?

Answer 1

This doesn't sound good. Check with dmesg if the kernel has some issue, but that command might fail too. Many Linux distributions show kernel messages on console 10 you can go there with Ctrl-Alt-F10.

Answer 2

Is it a sleep issue? You can try apm=off and acpi=off on the kernel line to see if it keeps the chipset awake. Also check your BIOS settings and make sure it's not trying to put things to sleep.

Also, you can check with hdparm if the drive is trying to put itself to sleep after a certain time. Don't remember the details off-hand, pinfo hdparm or man hdparm may help.

Answer 3

Input-output errors mean there is some data loss underway, which means pretty much no information concerning that pipeline will be reliable.

I would skip the diagnosis part and consider data rescue scenarios.