4

I am setting up a new set of drives on a CentOS 7 installation (10x1TB), however am seeing something annoying during the build process.

When I attempt to build the array, it is immediately recognised but indicates an error state (degraded, one disk missing.)

The full build command was:

mdadm --create /dev/md1 --level=5 --raid-devices=9 /dev/sde1 /dev/sdg1 /dev/sdc1 /dev/sdd1 /dev/sdf1 /dev/sdl1 /dev/sdj1 /dev/sdi1 /dev/sdm1 --spare-devices=1 /dev/sdk1

However I then tried the same without the spare drive:

mdadm --create /dev/md1 --level=5 --raid-devices=9 /dev/sde1 /dev/sdg1 /dev/sdc1 /dev/sdd1 /dev/sdf1 /dev/sdl1 /dev/sdj1 /dev/sdi1 /dev/sdm1

then variants of a three drive array:

mdadm --create /dev/md1 --level=5 --raid-devices=3 /dev/sdj1 /dev/sdi1 /dev/sdm1

mdadm --create /dev/md1 --level=5 --raid-devices=3 /dev/sde1 /dev/sdg1 /dev/sdc1

In all cases, the new array initialises with the expected free size (8TB and 2TB in the variants.) The array is also useable during the rebuild, I can read and write files.

Curiously, when building with the spare, the spare drive shows significant read/write activity, implying it is somehow participating in the build.

Basically, is this normal and if not, what am I doing wrong?

Edit:

This is the output of: mdadm --detail /dev/md1

/dev/md1:
        Version : 1.2
  Creation Time : Sun Jan  4 00:00:22 2015
     Raid Level : raid5
     Array Size : 7813042176 (7451.10 GiB 8000.56 GB)
  Used Dev Size : 976630272 (931.39 GiB 1000.07 GB)
   Raid Devices : 9
  Total Devices : 10
    Persistence : Superblock is persistent

    Update Time : Sun Jan  4 00:05:33 2015
          State : clean, degraded, recovering 
 Active Devices : 8
Working Devices : 10
 Failed Devices : 0
  Spare Devices : 2

         Layout : left-symmetric
     Chunk Size : 512K

 Rebuild Status : 0% complete

           Name : localhost.localdomain:1  (local to host localhost.localdomain)
           UUID : f2d4f470:9d5022bf:a397b5a0:45b94d28
         Events : 72

    Number   Major   Minor   RaidDevice State
       0       8       65        0      active sync   /dev/sde1
       1       8       97        1      active sync   /dev/sdg1
       2       8       33        2      active sync   /dev/sdc1
       3       8       49        3      active sync   /dev/sdd1
       4       8       81        4      active sync   /dev/sdf1
       5       8      177        5      active sync   /dev/sdl1
       6       8      145        6      active sync   /dev/sdj1
       7       8      129        7      active sync   /dev/sdi1
      10       8      193        8      spare rebuilding   /dev/sdm1

       9       8      161        -      spare   /dev/sdk1
3
  • If you're not referring to the initial sync, which is normal, maybe you should show the exact output of the error state you're referring to. Also that of dmesg after building the RAID. The spare drive will see some metadata updates, but shouldn't be involved until it takes over for another failed drive. Jan 4, 2015 at 0:15
  • I may need some help getting the information you require. The error is shown in the 'Disks' utility, with a red 'ARRAY IS DEGRADED - 1 disk is missing'. The progress bar suggests 'Recovering: 1.7%' with about 45 hours remaining.
    – J Collins
    Jan 4, 2015 at 0:37
  • It sounds like you're referring to the initial sync after all. As for information, try dmesg, cat /proc/mdstat, mdadm --detail /dev/md42, mdadm --examine /dev/sd*, ... Jan 4, 2015 at 0:41

1 Answer 1

4

Yes, this is normal. When you first create the array the parity blocks have to be calculated and written, which is what you are seeing. If you know that the drives are already filled with zeros, then you can use the --assume-clean switch to direct mdadm to skip the initial resync. If the drives aren't actually filled with zeros, then performing a parity check on the array in the future will report many errors since you never calculated the correct parity.

Also FYI, it is not recommended to create such a large array using raid5 as the probability of a second drive failing before it can rebuild is getting high. You might want to use raid6 instead.

4
  • +1, raid6 would make a lot of sense here, considering there already is a spare drive available Jan 4, 2015 at 0:44
  • I have considered RAID6, however have swung back and forth as to which is best. With this configuration I'll have a hot spare available that is not being written/degraded thus is fresh. With the RAID6 the same drive is being written and degraded with the rest of them thus causing more wear. Perhaps that is silly though and I should just pre-emptively invest in a few more drives to leave on the shelf for the failovers.
    – J Collins
    Jan 4, 2015 at 1:02
  • But yes, thanks all, this is the answer I was looking for. Confirmation that the construction of the array was not immediate and required calculating parity for all the unused swarf on the drives. I'm not sure if I could have used 'assume-clean' as I'd had plenty of cracks at formatting and testing the drives.
    – J Collins
    Jan 4, 2015 at 1:04
  • @JCollins, yes, but the wear on the one drive isn't likely to matter; it is the wear on the other 9 active drives that will kill you. It's also not a bad idea to keep a cold spare or two on the shelf and swap them out after a year or so.
    – psusi
    Jan 4, 2015 at 1:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .