I am doing some experiments with hybrid RAID in Linux. My test consists of the following:
2x256GB SSD in RAID 0 (/dev/md1)
2x256GB HDD in RAID 0 (/dev/md2)
Then I made md1 and md2 into a RAID 1 (/dev/md127) and marking the slow HDD (md2) as --write-mostly.
Essentially, my goal is to get maximum performance AND disk space out of my SSDs, but at the same time be "safe" from drive failures. I understand that losing one of the SSDs would mean that I fall back on slow HDDs, but that's a price I am willing to pay compared to losing all data. Besides, it would only be for a few hours until the broken SSDs gets replaced and RAID repaired.
root@s1 / # cat /proc/mdstat Personalities : [raid0] [raid1] [linear] [multipath] [raid6] [raid5] [raid4] [raid10] md2 : active raid0 sdd1 sdc1 498802688 blocks super 1.2 512k chunks md127 : active raid1 md1 md2(W) 498671616 blocks super 1.2 [2/2] [UU] bitmap: 1/4 pages [4KB], 65536KB chunk md1 : active raid0 sdb2 sda2 498802688 blocks super 1.2 512k chunks
Now, running a simple throughput benchmark on the 3 raid devices gives a (for me) surprising results:
root@s1 / # hdparm -t /dev/md1 /dev/md1: Timing buffered disk reads: 2612 MB in 3.00 seconds = 870.36 MB/sec root@s1 / # hdparm -t /dev/md2 /dev/md2: Timing buffered disk reads: 812 MB in 3.01 seconds = 270.14 MB/sec root@s1 / # hdparm -t /dev/md127 /dev/md127: Timing buffered disk reads: 1312 MB in 3.00 seconds = 437.33 MB/sec
RAID 0 SSD gives 870 MB/sec
RAID 0 HDD gives 270 MB/sec
RAID 1 HYBRID gives 437 MB/sec.
As the HDD raid has been marked as --write-mostly, I would assume that a pure read test would not touch the HDD at all, so what is going on here? I would assume that the hybrid benchmark would give similar results as the pure RAID 0 SSD.
At a first glance, it looks like the HDD somehow is slowing down the RAID, by being partly used for the read (even though I told it not to do reads on the HDD). However, if I have a file copy running on the HDDs while running the hdparm benchmark, I get the same result! If the HDDs WERE used, I would assume the benchmark would give even slower results if the HDDs were used for other tasks during the benchmark.