Problem with RAID5

Question

I have a CentOS server with RAID5. Every time RAID5 re-syncs my server stop working. The hosting company stopped the httpd service so that RAID5 can re-sync itself, a process which can take as long as 3-4 hours.

The problem reoccurs frequently, so the Hosting company swaped my server hardware and I migrated to new hardware. I still have this problem (on the new server).

Is this something normal in RAID5? How can we solve this issue permanently? If every time RAID5 wants to re-sync my server overloads and my website will not be accessible, then RAID5 sucks.

I would really appreciate if you can suggest a solution for this disaster.

Here is /proc/mdstat report:

root@host [~]# watch 'cat /proc/mdstat'
Every 2.0s: cat /proc/mdstat                            Mon May  9 01:25:30 2011

Personalities : [raid1]
md0 : active raid1 xvda1[0] xvdb1[1]
      104320 blocks [2/2] [UU]

md1 : active raid1 xvda2[0] xvdb2[1]
      2096384 blocks [2/2] [UU]

md2 : active raid1 xvda5[0] xvdb5[1]
      484086528 blocks [2/2] [UU]
      [=====>...............]  resync = 29.5% (142978880/484086528) finish=77.7m
in speed=73108K/sec

unused devices: <none>

Answer 1

RAID should only resync after a server crash or replacing a failed disk. It's always recommended to use a UPS and set the system to shutdown on low-battery so that a resync won't be required on reboot. NUT or acpupsd can talk to many UPSes and initiaite a shutdown before the UPS is drained. If the server is resyncing outside of a crash, you probably have a hardware issue. Check the kernel log at /var/log/kern.log or by running dmesg. I also recommend setting up mdadm to email the adminstrator and running smartd on all disk drives similarly set up to email the administrator. I receive an email about half the time before I see a failed disk. If you are having unavoidable crashes, you should enable a write-intent bitmap on the RAID. This keeps a journal of where the disk is being written to and avoids a full re-sync on reboot. Enable it with:

mdadm -G /dev/md0 --bitmap=internal