Some background
A friend of mine was using in his office a NAS Buffalo-LS-WVL with two disks of 1TB each. It seems that the two disks were mounted as raid 1, but, as you will read, probably they have been not. The NAS gave some problems of extremely slowness and then suddenly didn't work anymore. I've been called in to rescue his data. Both disk have exactly the same partitioning: one physical and 6 logical, and in the 6th data lives in (aprox 80GB out of 0,95TB).
Disk /dev/sdd
seems to give hardware problems (slowliness, sector reading error, etc.), whereas /dev/sde is a well-physically functioning disk.
The goal is to extract data that were contained in the NAS. If not all data, the most to be extracted, the better. These data are vital for the company of this friend of mine.
What I have tried already
1st attempt: Mounting disks alone
This is the very first try, hoping it works, I've tried to get each disk and mount it alone, and I got this message:
root@ubuntu:~# mount /dev/sdd6 /mnt/n -or- root@ubuntu:~# mount /dev/sde6 /mnt/n
both gave me the same message:
mount: unknown filesystem type 'linux_raid_member'
2nd attempt: Creating disk array RAID 1 and try to mount them
OK, if I cannot mount them alone, then I need to create an array of disks. Let's suppose (the most logical) the original configuration was raid 1, and use one disk at a time:
root@ubuntu:~# mdadm --create --run --level=1 --raid-devices=2 \ /dev/md/md-singolo-e6--create-missing /dev/sde6 missing
gives:
mdadm: /dev/sde6 appears to be part of a raid array: level=raid0 devices=2 ctime=Mon Sep 26 10:23:48 2011 mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90 mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md/md-singolo-e6--create-missing started.
So, it seems that the original raid was in 0 mode and not 1 mode. Bad new, as a disk is giving sector problems.
Anyway, I gave a try to mount the newly created RAID1 array (even if I know it's no-sense):
root@ubuntu:~# mkdir /mnt/md-singolo-e6--create-missing root@ubuntu:~# mount /dev/md/md-singolo-e6--create-missing \ /mnt/md-singolo-a6--create-missing/
gave:
mount: /dev/md127: can't read superblock
Exactly the same result has been given for the other disk.
3rd attempt: Creating disk array RAID 0 and try to mount them
OK, as it has been stated that it was Raid0, let's go for it:
root@ubuntu:~# mdadm --create --run --level=0 --raid-devices=2 \ /dev/md/md001hw /dev/sdd6 /dev/sde6
gives:
mdadm: /dev/sdd6 appears to be part of a raid array: level=raid1 devices=2 ctime=Mon Oct 14 16:38:33 2013 mdadm: /dev/sde6 appears to be part of a raid array: level=raid1 devices=2 ctime=Mon Oct 14 17:01:01 2013 mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md/md001hw started.
OK, once created I try to mount it:
root@ubuntu:~# mount /dev/md/md001hw /mnt/n mount: you must specify the filesystem type
At this point all ext2,3,4 specified with
-t
gave error.4th attempt: Creating disk images and work with them
OK, as a disk has problem it is much better to work on a copy (dd) of the data partition, padded with 0 (sync) in case of block read error (error). I therefore created the two images:
This one for the good disk (block of 4MB, to be faster):
root@ubuntu:~# dd bs=4M if=/dev/sde6 of=/media/pietro/4TBexthdd/sde6-bs4M-noerror-sync.img conv=noerror,sync
and this one for the disk with problems (minimum block size, to be safer)
root@ubuntu:~# dd if=/dev/sde6 of=/media/pietro/4TBexthdd/sdd6-noerror-sync.img conv=noerror,sync
Once I got the two images I've tried to use them as RAID 0, with the command specified above. Nothing to do, the answer that came is that the images "is not a block device" and it does not create the array.
5th attempt: going byte-a-byte to rescue some data
OK, if a proper mounting is not working, let's go to extract data trough byte-a-byte reading and header and footer info. I used *foremost*to do this job, both on each single disk: for disk 1:
root@ubuntu:~# foremost -i /dev/sde6 -o /media/pietro/4TBexthdd/foremost_da_sde6/
it creates sub-folders with file extensions, but no population at all in them. Whereas for disk 2 (the damaged one):
root@ubuntu:~# foremost -i /dev/sdd6 -o /media/pietro/4TBexthdd/foremost_da_sdd6_disco2/
neither the sub-folder structure is created by foremost.
Same result when I tried foremost on RAID 0 array:
root@ubuntu:~# foremost -i /dev/md/md001hw -o /media/pietro/4TBexthdd/foremost_da_raid_hw/
Neither sub-folder structure has been created.
Where I need some help / My Questions
- First and foremost question: how to rescue data? Does anyone has any hint I've not tried?
- Could anyone of you suggest anything different of what I've done?
Other questions:
- I'm new to
mdadm
, did I do everything correctly? - Was effectively the original array created on Sept 26th, 2011 in Raid 0 mode?
- Why I cannot use the partition images to create an array?
Appendix
This is the output of dmesg
in case of reading from the failing disk (/dev/sdd
):
[ 958.802966] sd 8:0:0:0: [sdd] Unhandled sense code
[ 958.802976] sd 8:0:0:0: [sdd]
[ 958.802980] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 958.802984] sd 8:0:0:0: [sdd]
[ 958.802987] Sense Key : Medium Error [current]
[ 958.802994] sd 8:0:0:0: [sdd]
[ 958.802999] Add. Sense: Unrecovered read error
[ 958.803003] sd 8:0:0:0: [sdd] CDB:
[ 958.803006] Read(10): 28 00 00 d5 c7 e0 00 00 f0 00
[ 958.803021] end_request: critical target error, dev sdd, sector 14010336
[ 958.803028] quiet_error: 36 callbacks suppressed
[ 958.803032] Buffer I/O error on device sdd, logical block 1751292
[ 958.803043] Buffer I/O error on device sdd, logical block 1751293
[ 958.803048] Buffer I/O error on device sdd, logical block 1751294
[ 958.803052] Buffer I/O error on device sdd, logical block 1751295
[ 958.803057] Buffer I/O error on device sdd, logical block 1751296
[ 958.803061] Buffer I/O error on device sdd, logical block 1751297
[ 958.803065] Buffer I/O error on device sdd, logical block 1751298
[ 958.803069] Buffer I/O error on device sdd, logical block 1751299
[ 958.803074] Buffer I/O error on device sdd, logical block 1751300
[ 958.803078] Buffer I/O error on device sdd, logical block 1751301
[ 961.621228] sd 8:0:0:0: [sdd] Unhandled sense code
[ 961.621236] sd 8:0:0:0: [sdd]
[ 961.621238] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 961.621241] sd 8:0:0:0: [sdd]
[ 961.621243] Sense Key : Medium Error [current]
[ 961.621248] sd 8:0:0:0: [sdd]
[ 961.621251] Add. Sense: Unrecovered read error
[ 961.621254] sd 8:0:0:0: [sdd] CDB:
[ 961.621255] Read(10): 28 00 00 d5 c8 d0 00 00 10 00
[ 961.621266] end_request: critical target error, dev sdd, sector 14010576
[ 964.791077] sd 8:0:0:0: [sdd] Unhandled sense code
[ 964.791084] sd 8:0:0:0: [sdd]
[ 964.791087] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 964.791090] sd 8:0:0:0: [sdd]
[ 964.791092] Sense Key : Medium Error [current]
[ 964.791096] sd 8:0:0:0: [sdd]
[ 964.791099] Add. Sense: Unrecovered read error
[ 964.791102] sd 8:0:0:0: [sdd] CDB:
[ 964.791104] Read(10): 28 00 00 d5 c8 00 00 00 08 00
[ 964.791114] end_request: critical target error, dev sdd, sector 14010368
[ 964.791119] quiet_error: 22 callbacks suppressed
[ 964.791122] Buffer I/O error on device sdd, logical block 1751296
mdadm --create
in a data recovery situation.mdadm
's defaults for metadata version and offsets change every so often, it usually ends up overwriting your filesystem superblock. You should've made the copy first, preferrably usingddrescue
, or if usingdd
withconv=noerror,sync
, you most likely want to pick a smaller blocksize. Withbs=4M
it might skip 4M for a single bad 512b block. Once you have a copy, you should not write to it either, but make another copy or use a snapshot mechanism to leave it unchanged when you run your experiments on it.