0

I have a QNAP server with 8 disks. I have 2 backup/extra disks that are not in use. (I had the money an knew eventually the running ones would need replaced) The full array was across the 8 disks @6TB each... math... but I pretty sure less than 1/3 was currently filled.

I had a rep from QNAP work on it and said that with a double disk dive it most likely wont recover. I have faith in the rep in regards to what he is allowed/can do... but..

I have seen things where people pretty much recovered from this type of disaster. My hope is resting on the fact that according to the system all the disks are still good???

Getting a "Bad magic number" / "Couldn't find valid filesystem superblock"

This is the device diag output before rep and then after rep:

**** BEFORE ****:

RAID metadata found!
UUID:       cccd0319:89c30791:58322cfe:12ed5c64
Level:      raid5
Devices:    8
Name:       md1
Chunk Size: 64K
md Version: 1.0
Creation Time:  Mar 23 11:49:45 2017
Status:     OFFLINE
===============================================================================
 Disk | Device | # | Status |   Last Update Time   | Events | Array State
===============================================================================
   5  /dev/sdl3  0   Active   Apr 28 08:03:55 2019     3927   AAAAA.AA                 
   6  /dev/sdk3  1   Active   Apr 28 08:03:55 2019     3927   AAAAA.AA                 
   7  /dev/sdj3  2   Active   Apr 28 08:03:55 2019     3927   AAAAA.AA                 
 --------------  3  Missing   -------------------------------------------
   9  /dev/sdh3  4   Active   Apr 28 08:03:55 2019     3927   AAAAA.AA                 
  10  /dev/sdg3  5   Active   Apr 23 15:03:27 2019     3515   AAAAAAAA                 
  11  /dev/sdf3  6   Active   Apr 28 08:03:55 2019     3927   AAAAA.AA                 
  12  /dev/sde3  7   Active   Apr 28 08:03:55 2019     3927   AAAAA.AA                 
===============================================================================

**** AFTER ****:

RAID metadata found!
UUID:       a6860c7d:0b020f8d:1a61ec72:4684aeb7
Level:      raid5
Devices:    8
Name:       md1
Chunk Size: 64K
md Version: 1.0
Creation Time:  Apr 29 14:08:04 2019
Status:         ONLINE (md1) [UU_UUUUU]
===============================================================================
 Disk | Device | # | Status |   Last Update Time   | Events | Array State
===============================================================================
  12  /dev/sde3  0   Active   Apr 29 14:38:42 2019      331   AAAAAAAA                 
  11  /dev/sdf3  1   Active   Apr 29 14:38:42 2019      331   AAAAAAAA                 
  10  /dev/sdg3  2  Rebuild   Apr 29 14:38:42 2019      331   AAAAAAAA                 
   9  /dev/sdh3  3   Active   Apr 29 14:38:42 2019      331   AAAAAAAA                 
   8  /dev/sdi3  4   Active   Apr 29 14:38:42 2019      331   AAAAAAAA                 
   7  /dev/sdj3  5   Active   Apr 29 14:38:42 2019      331   AAAAAAAA                 
   6  /dev/sdk3  6   Active   Apr 29 14:38:42 2019      331   AAAAAAAA                 
   5  /dev/sdl3  7   Active   Apr 29 14:38:42 2019      331   AAAAAAAA                 
===============================================================================

closed as off-topic by Rui F Ribeiro, Mr Shunz, doneal24, Alexander, Kiwy May 7 at 13:24

  • This question does not appear to be about Unix or Linux within the scope defined in the help center.
If this question can be reworded to fit the rules in the help center, please edit the question.

  • It looks as if the rep has created a new array using the same disks (new uuid, new creation time). If the old array had something really important then take the array offline immediately. There is a reasonable chance you will be able to recover most of the data but it will be an enormous amount of work. If this is just a media server it will probably be more cost effective to just buy the movies again, based on 7 data disks * 6TB per disk * 1/3 full / 1GB per dvd = 14,000 movies. – icarus May 2 at 14:35
2

Near impossible to really answer as no one knows for sure what truly went on there between your before/after. Please do take my deductions below with a grain of salt - it's impossible to say for sure that things happened this way but this is what it looks like from the data you provided.

Your (BEFORE) shows 1 drive missing completely, and another significantly out of date. (Apr 23 vs. Apr 28, Event count 3515 vs. 3927).

Your (AFTER) is a complete mess. Event counts reset (331), drive order completely different (sde3 was #7, now is #0. sdf3 was #6, now is #1. and so on), unclear how the missing drive was restored and the out of date drive forced back into the array. Furthermore it shows the drive #2 to be rebuilding even though it was drive #3 that was missing and drive #5 that was out of date.

Basically it looks like someone re-created the RAID, which can work if done correctly, but only if you really know what you are doing but you don't know. Unless you can explain the change of drive order, it looks like it was done incorrectly here.

If these assumptions are correct then in your (BEFORE) state there were still very good chances of data recovery. Even if the filesystem itself was shot, any files not modified since the failure event (Apr 23) should have been undamaged and somewhat recoverable.

With a re-create that re-synced drives and rebuilds going on, likely destroying data on drive #7 and #2, chance of such recovery is now nil or rather reduced to files smaller than chunksize, which in your case happens to be only 64K. Good enough for code snippets but not much else.

One thing that could save you at this point is if the missing drive was not actually failed, just kicked out randomly and not kicked long before Apr 23. You did not actually state if this drive was physically replaced or not.

If the missing drive was not actually defective, and still had valid data, and still spinning in this array, then the re-create might not have caused additional damage even if it was with the wrong drive order. This is a magic trick possible with raid5 due to how XOR parity calculation works (any order).

  • I did not replace anything. it just kicked it out and showed no actual errors – Shadowed6 Apr 29 at 21:40
  • I stopped the rebuild which was at 10%. So looking around I would assume that I need to run something along the lines of: – Shadowed6 Apr 29 at 21:53
  • 1
    mdadm -CfR --assume-clean /dev/md1 -n 8 -l 5 -e 1.0 - 6 /dev/sdl3 /dev/sdk3 /dev/sdj3 /dev/sdi3 /dev/sdh3 /dev/sdg3 /dev/sdf3 /dev/sde3 – Shadowed6 Apr 29 at 21:53
  • The missing drive went missing yesterday. – Shadowed6 Apr 29 at 21:54
  • Use overlays for experiments (described in the link) and put the most outdated drive as missing. – frostschutz Apr 29 at 22:38

Not the answer you're looking for? Browse other questions tagged or ask your own question.