3

I wanted to replace a corrupted raid5-device on my linux system (opensuse 13.2) but missed the step of set-faulty/removing the device (/dev/sdc1).

So i accidently just switch the device with a cloned new disk witch leads to this output:

 mdadm --detail /dev/md0
/dev/md0:
        Version : 1.0
  Creation Time : Tue Dec 10 19:07:16 2013
     Raid Level : raid5
     Array Size : 17581589376 (16767.11 GiB 18003.55 GB)
  Used Dev Size : 2930264896 (2794.52 GiB 3000.59 GB)
   Raid Devices : 7
  Total Devices : 6
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Mon Jul 13 12:04:24 2015
          State : active, degraded
 Active Devices : 6
Working Devices : 6
 Failed Devices : 0
  Spare Devices : 0

     Layout : left-symmetric
 Chunk Size : 64K

       Name : linux:0
       UUID : fd2ae851:d7376f4f:72b8032d:2931522b
     Events : 24936

Number   Major   Minor   RaidDevice State
   9       8       17        0      active sync   /dev/sdb1
   2       0        0        2      removed
   2       8       33        2      active sync   /dev/sdc1
   3       8       49        3      active sync   /dev/sdd1
   4       8       65        4      active sync   /dev/sde1
   5       8       81        5      active sync   /dev/sdf1
   7       8       97        6      active sync   /dev/sdg1

Just removing sdc1 by mdadm --manage /dev/md0 -r /dev/sdc1 did not clear the "removed" entry on device 2. So the array is falsely marked as degraded.

Now i don't know how to cleanup the raid array.

Edit: Here is the output of mdadm --examine /dev/sd?1 :

/dev/sdb1:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x9
     Array UUID : fd2ae851:d7376f4f:72b8032d:2931522b
           Name : linux:0
  Creation Time : Tue Dec 10 19:07:16 2013
     Raid Level : raid5
   Raid Devices : 7

 Avail Dev Size : 5860529896 (2794.52 GiB 3000.59 GB)
     Array Size : 17581589376 (16767.11 GiB 18003.55 GB)
  Used Dev Size : 5860529792 (2794.52 GiB 3000.59 GB)
   Super Offset : 5860530160 sectors
   Unused Space : before=0 sectors, after=344 sectors
          State : clean
    Device UUID : 943b936b:57015945:ee697339:e7b51539

Internal Bitmap : -24 sectors from superblock
    Update Time : Mon Jul 13 12:06:38 2015
  Bad Block Log : 512 entries available at offset -8 sectors - bad blocks present.
       Checksum : fc05a9a3 - correct
         Events : 24938

         Layout : left-symmetric
     Chunk Size : 64K

   Device Role : Active device 0
   Array State : A.AAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdc1:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : fd2ae851:d7376f4f:72b8032d:2931522b
           Name : linux:0
  Creation Time : Tue Dec 10 19:07:16 2013
     Raid Level : raid5
   Raid Devices : 7

 Avail Dev Size : 5860529896 (2794.52 GiB 3000.59 GB)
     Array Size : 17581589376 (16767.11 GiB 18003.55 GB)
  Used Dev Size : 5860529792 (2794.52 GiB 3000.59 GB)
   Super Offset : 5860530160 sectors
   Unused Space : before=0 sectors, after=344 sectors
          State : clean
    Device UUID : 067ee8d8:b84ef028:edf40cb0:c5623351

Internal Bitmap : -24 sectors from superblock
    Update Time : Mon Jul 13 12:06:38 2015
  Bad Block Log : 512 entries available at offset -8 sectors
       Checksum : dba97145 - correct
         Events : 24938

         Layout : left-symmetric
     Chunk Size : 64K

   Device Role : Active device 2
 Array State : A.AAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdd1:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : fd2ae851:d7376f4f:72b8032d:2931522b
           Name : linux:0
  Creation Time : Tue Dec 10 19:07:16 2013
     Raid Level : raid5
   Raid Devices : 7

 Avail Dev Size : 5860529896 (2794.52 GiB 3000.59 GB)
     Array Size : 17581589376 (16767.11 GiB 18003.55 GB)
  Used Dev Size : 5860529792 (2794.52 GiB 3000.59 GB)
   Super Offset : 5860530160 sectors
   Unused Space : before=0 sectors, after=344 sectors
          State : clean
    Device UUID : ca751de8:dbb68f99:c741dcad:562058c0

Internal Bitmap : -24 sectors from superblock
    Update Time : Mon Jul 13 12:06:38 2015
  Bad Block Log : 512 entries available at offset -8 sectors
       Checksum : c871db99 - correct
         Events : 24938

         Layout : left-symmetric
     Chunk Size : 64K

   Device Role : Active device 3
   Array State : A.AAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sde1:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : fd2ae851:d7376f4f:72b8032d:2931522b
           Name : linux:0
  Creation Time : Tue Dec 10 19:07:16 2013
     Raid Level : raid5
   Raid Devices : 7

 Avail Dev Size : 5860529896 (2794.52 GiB 3000.59 GB)
     Array Size : 17581589376 (16767.11 GiB 18003.55 GB)
  Used Dev Size : 5860529792 (2794.52 GiB 3000.59 GB)
   Super Offset : 5860530160 sectors
   Unused Space : before=0 sectors, after=344 sectors
          State : clean
    Device UUID : b43c31d2:e4eeb74b:fe239884:7acc8f89

Internal Bitmap : -24 sectors from superblock
    Update Time : Mon Jul 13 12:06:38 2015
  Bad Block Log : 512 entries available at offset -8 sectors
       Checksum : 4a168e8 - correct
         Events : 24938

         Layout : left-symmetric
     Chunk Size : 64K

   Device Role : Active device 4
Array State : A.AAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdf1:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : fd2ae851:d7376f4f:72b8032d:2931522b
           Name : linux:0
  Creation Time : Tue Dec 10 19:07:16 2013
     Raid Level : raid5
   Raid Devices : 7

 Avail Dev Size : 5860529896 (2794.52 GiB 3000.59 GB)
     Array Size : 17581589376 (16767.11 GiB 18003.55 GB)
  Used Dev Size : 5860529792 (2794.52 GiB 3000.59 GB)
   Super Offset : 5860530160 sectors
   Unused Space : before=0 sectors, after=344 sectors
          State : clean
    Device UUID : acaa467e:9ec70468:ccf75f20:2aa6a058

Internal Bitmap : -24 sectors from superblock
    Update Time : Mon Jul 13 12:06:38 2015
  Bad Block Log : 512 entries available at offset -8 sectors
       Checksum : 37dc5d18 - correct
         Events : 24938

         Layout : left-symmetric
     Chunk Size : 64K

   Device Role : Active device 5
   Array State : A.AAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdg1:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : fd2ae851:d7376f4f:72b8032d:2931522b
           Name : linux:0
  Creation Time : Tue Dec 10 19:07:16 2013
     Raid Level : raid5
   Raid Devices : 7

 Avail Dev Size : 5860529896 (2794.52 GiB 3000.59 GB)
     Array Size : 17581589376 (16767.11 GiB 18003.55 GB)
  Used Dev Size : 5860529792 (2794.52 GiB 3000.59 GB)
   Super Offset : 5860530160 sectors
   Unused Space : before=0 sectors, after=344 sectors
          State : clean
    Device UUID : 710a4e23:ccad2620:5be95df1:8f868bbc

Internal Bitmap : -24 sectors from superblock
    Update Time : Mon Jul 13 12:06:38 2015
  Bad Block Log : 512 entries available at offset -8 sectors
       Checksum : c9ee7501 - correct
         Events : 24938

         Layout : left-symmetric
     Chunk Size : 64K

   Device Role : Active device 6
   Array State : A.AAAAA ('A' == active, '.' == missing, 'R' == replacing)

Thanks for your help!

  • Active 6, Working 6, Failed 0 - so what is the problem? Please clarify. Show /proc/mdstat and mdadm --examine /dev/sd?1 and use proper formatting if possible... – frostschutz Jul 13 '15 at 13:40
  • I edited my post with the output of mdadm --examine /dev/sd?1. The problem is that the array is shown as degraded because of two raid-devices #2 (one with label "removed", one with sdc1) which makes a total of 7 raid-devices (instead of 6). – user3308110 Jul 13 '15 at 15:49
  • Your RAID firmly believes that it's supposed to have 7 devices. So there is in reality one drive missing. Even so it should run & give you data access with 6 out of 7 devices. – frostschutz Jul 13 '15 at 15:57
  • I think that's because i accidently replaced the corrupt disk with a cloned new one. How can I get rid of this "false" device entry? Thanks – user3308110 Jul 13 '15 at 16:02
  • This is not something that happens by cloning a disk. It could have been a --grow --raid-devices=7. In which case you still have a 7 disk raid, as reported by the metadata of all disks. Look at the array size: 18000GB / 3000GB = 6 means a 7 disk RAID5 (6 data + 1 parity). Before doing anything else you should verify (in read-only mode) that your data is still there. – frostschutz Jul 13 '15 at 17:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.