Recovering an ext4 filesystem with corrupt journal

Question

I have a hard drive that seems to have developed some kind of hardware problem in the journal. This prevents the drive from being loaded. Running mount -r -t ext4 /dev/sda5 /mnt/root results in the following kernel spew:

[ 1420.671055] ata1.00: exception Emask 0x0 SAct 0x1000000 SErr 0x0 action 0x0
[ 1420.673862] ata1.00: irq_stat 0x40000008
[ 1420.676600] ata1.00: failed command: READ FPDMA QUEUED
[ 1420.679326] ata1.00: cmd 60/40:c0:a8:9d:0a/00:00:08:00:00/40 tag 24 ncq 32768 in
[ 1420.679326]          res 41/40:40:e1:9d:0a/00:00:08:00:00/00 Emask 0x409 (media error) <F>
[ 1420.685014] ata1.00: status: { DRDY ERR }
[ 1420.687858] ata1.00: error: { UNC }
[ 1420.761064] ata1.00: configured for UDMA/133
[ 1420.763863] sd 0:0:0:0: >[sda] Unhandled sense code
[ 1420.766653] sd 0:0:0:0: >[sda]  
[ 1420.769364] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 1420.772104] sd 0:0:0:0: >[sda]  
[ 1420.774780] Sense Key : Medium Error [current] [descriptor]
[ 1420.777491] Descriptor sense data with sense descriptors (in hex):
[ 1420.780249]         72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00 
[ 1420.783175]         08 0a 9d e1 
[ 1420.785972] sd 0:0:0:0: >[sda]  
[ 1420.788730] Add. Sense: Unrecovered read error - auto reallocate failed
[ 1420.791545] sd 0:0:0:0: >[sda] CDB: 
[ 1420.794331] Read(10): 28 00 08 0a 9d a8 00 00 40 00
[ 1420.797242] end_request: I/O error, dev sda, sector 134913505
[ 1420.800072] ata1: EH complete
[ 1420.800077] JBD2: Failed to read block at offset 2748
[ 1420.812529] JBD2: recovery failed
[ 1420.815275] EXT4-fs (sda5): error loading journal

I barely understand what this means, but it sure looks like a hardware problem preventing the journal from being loaded. Is there anyway to work around this? Can the journal, say, be relocated to a different offset while leaving the rest of the drive mostly intact? I have no intention of continuing to use the drive; I just want to mount it so I can copy a few crucial files.

Answer 1

To mount it read-only, without replaying the journal so you can access your files (you will probably lose some changes which haven't yet been committed to the regular fs):

mount -o ro,noload ...

Answer 2

Have a look at the output of smartctl -a /dev/sda. If it's not a cable problem then the disk may be dying. If you don't have a recent backup you should try to copy the block device content (without mounting) using dd_rescue or dd with respective options.

After that you may try hdparm --write-sector (always funny these "VERY DANGEROUS" options...).

Another option is to create a DM device which bypasses the defective sectors. That doesn't recover their contents, of course. You must convert the sda offset (134913505) into a sda5 offset (find out the number of the first sector of sda5 and subtract it). Let's assume the sda5 offset is 12345. You create a block device whose sector 12345 is not mapped to sda5 but somewhere else (e.g. loop device). The device definition (for dmsetup create) looks like this:

0 12345 linear /dev/sda5 0
12345 1 linear /dev/loop0 0
12346 99987653 /dev/sda5 12345

# format:
# logical_start_sector num_sectors linear destination_device start_sector

You will not be so lucky that there is only one defective sector. You can map out as many as you want (resulting in the DM device delivering zeros for those sectors but being writable) but calculation the offsets in loop0 may become fun. Another option would be to use a virtual device of the same size for remapping and use either thin provisioning or a snapshot (with small chunk size, 1 sector e.g.) of a zero target device.

Edit 1

The loop device should not point to a file on the same disk (sda), of course.

Answer 3

You can boot in recovery mode, in which the FS will be mounted read-only, and copy your stuff.