5

I have a backup drive that has been intermittently failing (but almost always succeeding) to run rsnapshot lately.

ls and e2label output are not encouraging:

$ ls -al /mnt/backup/
ls: cannot access /mnt/backup/daily.3: Input/output error
ls: cannot access /mnt/backup/daily.5: Input/output error
ls: cannot access /mnt/backup/weekly.3: Input/output error
ls: cannot access /mnt/backup/monthly.1: Input/output error
ls: cannot access /mnt/backup/weekly.1: Input/output error
ls: cannot access /mnt/backup/daily.1: Input/output error
ls: cannot access /mnt/backup/daily.0: Input/output error
ls: cannot access /mnt/backup/weekly.2: Input/output error
ls: cannot access /mnt/backup/daily.4: Input/output error
total 52
drwxr-xr-x 19 root root  4096 Aug 12 20:01 .
drwxr-xr-x  7 root root  4096 Jun 18 19:08 ..
d?????????  ? ?    ?        ?            ? daily.0
d?????????  ? ?    ?        ?            ? daily.1
d?????????  ? ?    ?        ?            ? daily.3
d?????????  ? ?    ?        ?            ? daily.4
d?????????  ? ?    ?        ?            ? daily.5
dr-xr-xr-x 19 root root  4096 Aug 13 06:00 daily.6
drwx------  2 root root 16384 Mar 26 13:13 lost+found
dr-xr-xr-x 23 root root  4096 Aug 12 20:01 minutes.0
dr-xr-xr-x 23 root root  4096 Aug 10 19:54 minutes.2
dr-xr-xr-x 23 root root  4096 Aug  8 22:24 minutes.3
dr-xr-xr-x 23 root root  4096 Aug  8 17:26 minutes.4
dr-xr-xr-x 23 root root  4096 May 18 19:39 monthly.0
d?????????  ? ?    ?        ?            ? monthly.1
dr-xr-xr-x 23 root root  4096 Jul 13 20:04 weekly.0
d?????????  ? ?    ?        ?            ? weekly.1
d?????????  ? ?    ?        ?            ? weekly.2
d?????????  ? ?    ?        ?            ? weekly.3

$ e2label /mnt/backup
e2label: Attempt to read block from filesystem resulted in short read while trying to open /mnt/backup
Couldn't find valid filesystem superblock.

But what is strange is that dmesg reports the drive is offline:

$ dmesg | tail
sd 2:0:0:0: rejecting I/O to offline device
sd 2:0:0:0: rejecting I/O to offline device
sd 2:0:0:0: rejecting I/O to offline device
EXT4-fs error (device sdc1): __ext4_get_inode_loc: unable to read inode block - inode=50729874, block=202899801
sd 2:0:0:0: rejecting I/O to offline device
sd 2:0:0:0: rejecting I/O to offline device
EXT4-fs error (device sdc1): __ext4_get_inode_loc: unable to read inode block - inode=50331649, block=201326624
sd 2:0:0:0: rejecting I/O to offline device
sd 2:0:0:0: rejecting I/O to offline device
EXT4-fs error (device sdc1): __ext4_get_inode_loc: unable to read inode block - inode=47185921, block=188743712

Sometimes we also get:

EXT4-fs error (device sdc1): ext4_put_super: Couldn't clean up the journal

And lately, unattaching and reattaching the device as a HyperV passthrough to the VM, we have gotten many of these is dmesg:

scsi scan: INQUIRY result too short (5), using 36

...and now can't mount it at all.

Now, if it's offline, how can I see any output from ls at all?

Is it more likely that this indicates corruption of the host VM doing the backup, or that the USB hard drive has hardware trouble, or both?

UPDATE: I also cannot format it:

$ sudo mkfs.ext4 -L 2015backup2new /dev/sdc1
mke2fs 1.41.12 (17-May-2010)
mkfs.ext4: No such device or address while trying to determine filesystem size

And I cannot see or modify its partitions:

$ sudo fdisk -l /dev/sdc
$ sudo fdisk /dev/sdc

Unable to open /dev/sdc

But it does exist:

$ sudo ls -al /dev/sdc*
brw-rw---- 1 root disk 8, 32 Aug 10 15:26 /dev/sdc
brw-rw---- 1 root disk 8, 33 Aug 10 15:26 /dev/sdc1

I also can't ignore the partition:

$ sudo mkfs.ext4 -L 2015backup2new /dev/sdc
mke2fs 1.41.12 (17-May-2010)
/dev/sdc is entire device, not just one partition!
Proceed anyway? (y,n) y
mkfs.ext4: No such device or address while trying to determine filesystem size

UPDATE: Interestingly, a two other USB drives now exhibit this problem. But if I attach any of them to a CentOS7 image (instead of the main 6.5 image we were working with), it formats and mounts OK. (CentOS 7's dmesg output also shows scsi 3:0:0:1: scsi scan: INQUIRY result too short (5), using 36 several times, but it functions anyway.) However, detaching and then attaching the formatted drive back to the CentOS 6.5 image still exhibits all the above errors (except for the initial partial showing from ls.)

0

I don't have an answer for your actual hardware trouble, but I can clear up the confusion about your diagnostics.

The actual official source for what partitions are available is /proc/partitions. Just because there is a device node in /dev doesn't really mean anything. In fact, on really old linux distros they would populate /dev with every possible device name regardless of which were available on the system. The newer methods of creating device nodes on demand is for your convenience.

As for the directories, your results are caused by the filesystem cache. A directory holds only a list of filenames and "inodes" (file id numbers). The rest of the details about a file like ownership, permissions, and timestamps come from the file itself. When you list a directory, it starts by consulting the list of files, and then runs "stat" on each file to load the rest of the data. If the directory is cached, and some of the files are cached, but it is unable to stat any files which aren't already cached, then you see output like what you pasted above.

dmesg is correctly explaining "we can't reach this device at all, anymore".

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.