I'm trying to determine if I should re-setup my RAID array due to poor IO performance. First off, the system:
- i7 920
- 4 4TB WD 5400 Green drives
- CentOS 6.3 host
Secondly, the disk setup:
- /dev/sda2,b2,c2,d2 are individually LUKS encrypted
- /dev/mapper/a2,b2,c2,d2 are all part of a software RAID5 /dev/md1
- /dev/md1 has LVM on top of that
- LVM is used to separate /, /storage, and swap
I choose this structure to allow for multiple instances of kcryptd, thinking that by doing this, I would get multithread support on encryption since one instance is running per drive. However, I'm beginning to wonder if that was a good idea.
For instance, if I run a heavy decompression routine on a RAR file of random data, my IO Wait goes up to around 25% and it slows the overall system down. I'm wondering if all the instruction sets are getting backed up somehow due to all the kcryptd processes.
Therefore, I'm considering changing to:
- /dev/sda2,b2,c2,d2 are put into /dev/md1
- /dev/md1 is encrypted and mapped to /dev/mapper/1
- LVM on top of /dev/mapper/1
This would drop down to a single kcrpytd process, which could be a bottleneck in it's own right, too. Does anyone think this will help with my IO issue?