i have an debian squeeze with 6x3TB Disks. On the SWRaid5 are 4 partitions. First for gpart Bios, Raid1 for "/", Raid5 for swap and the biggest part raid5 for files. While the system has build status all disks lay on the open case. after all is running i saw that one is a bit apoart of the case and could possibly fall. thats why i move two slightly a bit. while i doing this both spins down, sorry the system runs ;(, and the raid crashes with: error cant start while 2 missing disks.
"mdadm: /dev/md2 assembled from 4 drives and 1 \
spare - not enough to start the array."
at this point i had no writeoperations. I safe some infos and figure out that disk sdd4 and sdf4 failed and in my opinion sdd fails before sdf:
mdadm -E /dev/sda4
/dev/sda4:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : e84f0346:3f5ff3f1:507b6f9c:0fa02c63
Name : mfsnode1:2 (local to host mfsnode1)
Creation Time : Tue Feb 5 17:44:45 2013
Raid Level : raid5
Raid Devices : 6
Avail Dev Size : 5842757597 (2786.04 GiB 2991.49 GB)
Array Size : 29213772800 (13930.21 GiB 14957.45 GB)
Used Dev Size : 5842754560 (2786.04 GiB 2991.49 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 4f8851b4:001bf0c0:3aab60e0:b2c5558f
Update Time : Tue Feb 5 17:44:45 2013
Checksum : c0376a50 - correct
Events : 0
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 0
Array State : AAAAA. ('A' == active, '.' == missing)
---------------
mdadm -E /dev/sdb4
/dev/sdb4:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : e84f0346:3f5ff3f1:507b6f9c:0fa02c63
Name : mfsnode1:2 (local to host mfsnode1)
Creation Time : Tue Feb 5 17:44:45 2013
Raid Level : raid5
Raid Devices : 6
Avail Dev Size : 5842757597 (2786.04 GiB 2991.49 GB)
Array Size : 29213772800 (13930.21 GiB 14957.45 GB)
Used Dev Size : 5842754560 (2786.04 GiB 2991.49 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : c2f63fa7:768e9945:64826929:6f1f68c2
Update Time : Tue Feb 5 17:44:45 2013
Checksum : b3ea7d20 - correct
Events : 0
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 1
Array State : AAAAA. ('A' == active, '.' == missing)
---------------------
mdadm -E /dev/sdc4
/dev/sdc4:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : e84f0346:3f5ff3f1:507b6f9c:0fa02c63
Name : mfsnode1:2 (local to host mfsnode1)
Creation Time : Tue Feb 5 17:44:45 2013
Raid Level : raid5
Raid Devices : 6
Avail Dev Size : 5842757597 (2786.04 GiB 2991.49 GB)
Array Size : 29213772800 (13930.21 GiB 14957.45 GB)
Used Dev Size : 5842754560 (2786.04 GiB 2991.49 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : e9861f3e:4de4d0ce:7d4b6dd7:e1215fc7
Update Time : Tue Feb 5 17:44:45 2013
Checksum : 86fc2eab - correct
Events : 0
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 2
Array State : AAAAA. ('A' == active, '.' == missing)
-----------------------
mdadm -E /dev/sdd4
/dev/sdd4:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 7b99380e:51d754cf:921c68e9:7b830d6a
Name : mfsnode1:2 (local to host mfsnode1)
Creation Time : Tue Feb 5 17:06:37 2013
Raid Level : raid5
Raid Devices : 6
Avail Dev Size : 5842757597 (2786.04 GiB 2991.49 GB)
Array Size : 29213772800 (13930.21 GiB 14957.45 GB)
Used Dev Size : 5842754560 (2786.04 GiB 2991.49 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 0da58625:14ed8675:6a7c4ba4:337d8c4b
Update Time : Tue Feb 5 17:06:37 2013
Checksum : 5f97164a - correct
Events : 0
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 5
Array State : AAA.AA ('A' == active, '.' == missing)
-------------------
mdadm -E /dev/sde4
/dev/sde4:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : e84f0346:3f5ff3f1:507b6f9c:0fa02c63
Name : mfsnode1:2 (local to host mfsnode1)
Creation Time : Tue Feb 5 17:44:45 2013
Raid Level : raid5
Raid Devices : 6
Avail Dev Size : 5842755584 (2786.04 GiB 2991.49 GB)
Array Size : 29213772800 (13930.21 GiB 14957.45 GB)
Used Dev Size : 5842754560 (2786.04 GiB 2991.49 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : b70cd4f6:1594cc29:b4346929:89a5ed34
Update Time : Tue Feb 5 17:44:45 2013
Checksum : 5a36c944 - correct
Events : 0
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 4
Array State : AAAAA. ('A' == active, '.' == missing)
-------------------
mdadm -E /dev/sdf4
/dev/sdf4:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : e84f0346:3f5ff3f1:507b6f9c:0fa02c63
Name : mfsnode1:2 (local to host mfsnode1)
Creation Time : Tue Feb 5 17:44:45 2013
Raid Level : raid5
Raid Devices : 6
Avail Dev Size : 5842755584 (2786.04 GiB 2991.49 GB)
Array Size : 29213772800 (13930.21 GiB 14957.45 GB)
Used Dev Size : 5842754560 (2786.04 GiB 2991.49 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 06202661:79792af2:6c8d02ae:769bdded
Update Time : Tue Feb 5 17:44:45 2013
Checksum : ca70109c - correct
Events : 0
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 3
Array State : AAAAA. ('A' == active, '.' == missing)
I think the superblock is ok, the chunksize the same and all other parameter looks good too. After that i started to test some variations:
Test1:
mdadm --create --level 5 -n 6 --chunk=512 --assume-clean /dev/md2 \
/dev/sd{a,b,c,d,e,f}4
-> filesize 708MB with 20603326 lines and canceling at the end by e2fsck
- bad superblock or partitiontable is damage
- bad checksum of group or descriptor
- lots of invalid inodes
- canceld with lots of illegal blocks in inodes
Test2:
mdadm --create --level 5 -n 6 --chunk=512 --assume-clean /dev/md2 /dev/sd{a,b,c,d,e}4 \
missing
-> filesize 1,3GB with 37614367 lines and canceling by e2fsck at the end
- back to original superblock
- bad superblock or damaged partitiontable at the beginning
- lots of invalid inodes
- canceld with iteration of inade
Test3:
mdadm --create --level 5 -n 6 --chunk=512 --assume-clean /dev/md2 /dev/sd{a,b,c}4 \
missing /dev/sd{e,f}4
-> filesize 1,4GB with 40745425 lines and canceling by e2fsck at the end
- errors see test2
- read error while reading next inode
Test4:
mdadm --create --level 5 -n 6 --chunk=512 --assume-clean /dev/md2 \
/dev/sd{a,b,c,f,e,d}4
->filesize 874MB with 25412000 lines and break by e2fsck at the end
- try original superblock
- bad superblock or damaged partitiontable
- than lots of checksumm invalid deskriptor of group
- at the end illegal block in inode to much invalid blocks in inode
Test5:
mdadm --create --level 5 -n 6 --chunk=512 --assume-clean /dev/md2 /dev/sd{a,b,c}4 \
missing /dev/sd{e,d}4
-> filesize 1,6GB with 45673505 lines and canceling at the end by e2fsck
Test6:
mdadm --create --level 5 -n 6 --chunk=512 --assume-clean /dev/md2 /dev/sd{a,b,c,f,e}4 \
missing
- try original superblock
- bad superblock or damage partitiontable
- lots of checksumm error in group deskriptor
- ends with conflict in inode table with another filesystem block
-> filesize 542MB with 15727702 lines and cancelingat the end by e2fsck
Teset6 looks like the best one, but what do you think and perhaps what could i do \ else? Any further help?
I think the best could be the last but iam absoluteliy unsure. Please help for further investigation and hopefully recovery of my data. thx
