3

I am doing some experiments with hybrid RAID in Linux. My test consists of the following:

2x256GB SSD in RAID 0 (/dev/md1)

2x256GB HDD in RAID 0 (/dev/md2)

Then I made md1 and md2 into a RAID 1 (/dev/md127) and marking the slow HDD (md2) as --write-mostly.

Essentially, my goal is to get maximum performance AND disk space out of my SSDs, but at the same time be "safe" from drive failures. I understand that losing one of the SSDs would mean that I fall back on slow HDDs, but that's a price I am willing to pay compared to losing all data. Besides, it would only be for a few hours until the broken SSDs gets replaced and RAID repaired.

root@s1 / # cat /proc/mdstat
Personalities : [raid0] [raid1] [linear] [multipath] [raid6] [raid5] [raid4] [raid10]

md2 : active raid0 sdd1[1] sdc1[0]
      498802688 blocks super 1.2 512k chunks

md127 : active raid1 md1[2] md2[1](W)
      498671616 blocks super 1.2 [2/2] [UU]
      bitmap: 1/4 pages [4KB], 65536KB chunk

md1 : active raid0 sdb2[1] sda2[0]
      498802688 blocks super 1.2 512k chunks

Now, running a simple throughput benchmark on the 3 raid devices gives a (for me) surprising results:

root@s1 / # hdparm -t /dev/md1

/dev/md1:
 Timing buffered disk reads: 2612 MB in  3.00 seconds = 870.36 MB/sec
root@s1 / # hdparm -t /dev/md2

/dev/md2:
 Timing buffered disk reads: 812 MB in  3.01 seconds = 270.14 MB/sec
root@s1 / # hdparm -t /dev/md127

/dev/md127:
 Timing buffered disk reads: 1312 MB in  3.00 seconds = 437.33 MB/sec

RAID 0 SSD gives 870 MB/sec

RAID 0 HDD gives 270 MB/sec

RAID 1 HYBRID gives 437 MB/sec.

As the HDD raid has been marked as --write-mostly, I would assume that a pure read test would not touch the HDD at all, so what is going on here? I would assume that the hybrid benchmark would give similar results as the pure RAID 0 SSD.

At a first glance, it looks like the HDD somehow is slowing down the RAID, by being partly used for the read (even though I told it not to do reads on the HDD). However, if I have a file copy running on the HDDs while running the hdparm benchmark, I get the same result! If the HDDs WERE used, I would assume the benchmark would give even slower results if the HDDs were used for other tasks during the benchmark.

  • The great difference in speed between the SSD and HDD could indeed be problematic. – K7AAY May 21 at 17:27
  • 1
    for the glory of science, what happens if you fail the hdd side from the RAID1? You could also try RAID1 below and RAID0 on top. This would also mean rewriting only 1 drive in failure case... – frostschutz May 21 at 17:51
  • Note, you might also look at bcache instead of raid as a solution. See questions with the tag bcache – meuh May 22 at 9:10
  • Thanks! I will look into bcache. – Daniele Testa May 22 at 10:51
  • The raid wiki says Read performance will be somewhere between the fastest device and the slowest device. This is because md WILL occasionally still attempt to read from the slower device – meuh May 22 at 12:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.