SysBench highlighting abysmal disk write performance on white box vs Tier1 server

Question

Have been testing white box system with SuperMicro X8DTL motherboard with various SATA hard disks such as 7,200RPM Seagate Constellation ES and 10,000 RPM Western Digital VelociRaptor.

Have tested Fedora 15, Fedora 16, Ubuntu 11, running SysBench with the command found here MySQL High Performance blog

Disk write results are abysmal - typically results such as:

Operations performed:  0 Read, 3635 Write, 3635 Other = 7270 Total
    Read 0b  Written 14.199Mb  Total transferred 14.199Mb  (145.39Kb/sec)
    36.35 Requests/sec executed

On the other hand, a tier1 server with exactly the same CPU and 7,200RPM SATA hard disks running Fedora 15 produces these test results:

Operations performed: 0 Read, 151453 Write, 151453 Other = 302906 Total
    Read 0b  Written 591.61Mb  Total transferred 591.61Mb  (5.9159Mb/sec)
    1514.48 Requests/sec executed

I cannot understand how there can be such a massive difference and that the SuperMicro based system is producing such awful disk write performance.

I have tested various things including tweaks to fstab, scheduling, disabling disk standby and using sar, iostat, vmstat and so on to look for potential problems. But %idle, %iowait and so on don't show anything unusual. Also vm.zone_reclaim_mode as indicated here Poor disk performance although default setting on Fedora is already 0.

I tried different BIOS settings for the hard disk including IDE and AHCI. Would expect that AHCI should be the best, but write performance difference between IDE and AHCI options is negligible.

Anyone have ideas?

Answer 1

First, to do general IO testing, I recommend using iozone: http://www.iozone.org/

To properly answer this question, we need more information about the IO subsystems in each server.

However, in general, if you're looking for good IO performance, you need a dedicated hardware RAID card with onboard cache and a battery to back up that cache. This allows the RAID card to perform write-back caching, which can dramatically improve IO performance. The RAID card may also provide better throughput in general compared to onboard controllers.

And finally, an AHCI setting in the BIOS would control an onboard SATA controller. Onboard means it's on the motherboard and is not a server-class standalone hardware RAID card. If IO is not a priority for the workload, a server (whitebox or otherwise) may not have a separate RAID card and may indeed use the onboard controller.

You most assuredly want this BIOS setting set to AHCI, as any other setting will not provide Linux fast, direct access to the drives. If this setting is not making any difference, you might not have any drives connected to the onboard controller, or there may be another misconfiguration which is causing Linux or the BIOS to fall back to IDE-compatibility mode. You can check the kernel boot messages to see what drives the kernel sees and what interface the kernel is using to access those drives.