What does Fedora Workstation 29 use as the default I/O scheduler?

Question

If it depends on the exact type of block device, then what is the default I/O scheduler for each type of device?

Background information

Fedora 29 includes a Linux kernel from the 4.19 series. (Technically, the initial release used a 4.18 series kernel. But a 4.19 kernel is installed by the normal software updates).

Starting in version 4.19, the mainline kernel has CONFIG_SCSI_MQ_DEFAULT as default y. I.e. that's what you get if you take the tree published by Linus, without applying any Fedora-specific patches. By default, SCSI and SATA devices will use the new multi-queue block layer. (Linux treats SATA devices as being SCSI, using a translation based on the SAT standard).

This is a transitional step towards removing the old code. All the old code will now be removed in version 4.21 5.0, the next kernel release after 4.20.

In the new MQ system, block devices use a new set of I/O schedulers. These include none, mq-deadline, and bfq. In the mainline 4.19 kernel, the default scheduler is set as follows:

/* For blk-mq devices, we default to using mq-deadline, if available, for single queue devices. If deadline isn't available OR we have multiple queues, default to "none". */

A suggestion has been made to use BFQ as the default in place of mq-deadline. This suggestion was not accepted for 4.19.

For the legacy SQ block layer, the default scheduler is CFQ, which is most similar to BFQ.

=> The kernel's default I/O scheduler can vary, depending on the type of device: SCSI/SATA, MMC/eMMC, etc.

CFQ attempts to support some level of "fairness" and I/O priorities (ionice). It has various complexities. BFQ is even more complex; it supports ionice but also has heuristics to classify and prioritize some I/O automatically. deadline style scheduling is simpler; it does not support ionice at all.

=> Users with the Linux default kernel configuration, SATA devices, and no additional userspace policy (e.g. no udev rules), will be subject to a change in behaviour in 4.19. Where ionice used to work, it will no longer have any effect.

However Fedora includes specific kernel patches / configuration. Fedora also includes userspace policies such as default udev rules.

What does Fedora Workstation 29 use as the default I/O scheduler? If it depends on the exact type of block device, then what is the default I/O scheduler for each type of device?

Answer 1

Fedora 29 ships with the 4.18.16 kernel. It appears that CFQ is the default.

$ grep CONFIG_DEFAULT_IOSCHED= /boot/config-4.18.16-300.fc29.x86_64 
CONFIG_DEFAULT_IOSCHED="cfq"
$ grep CONFIG_SCSI_MQ_DEFAULT /boot/config-4.18.16-300.fc29.x86_64 
# CONFIG_SCSI_MQ_DEFAULT is not set
$ cat /sys/block/sda/queue/scheduler
noop deadline [cfq] 

As of this writing (November 24, 2018), 4.19.3 is available as update for F29. But, the config options do not appear to have changed.

4.20.0 (RC1) is in the "Rawhide" devel tree. In that devel-tree kernel, CFQ is still the default, and CONFIG_SCSI_MQ_DEFAULT is still unset. The Fedora Kernel list at https://lists.fedoraproject.org/archives/list/[email protected]/ is the best place to discuss whether this should change.

Answer 2

Some information that might be useful for your choice

I'm one of the authors of BFQ, so I'm all but a disinterested party :) But I'll report only numbers obtained with repeatable test.

We have been testing BFQ on SD Cards, eMMC, HDDs, SATA SSDs, and NVMe SSDs. As for HDDs and SSDs, we have run tests with both single-disk and RAID configurations.

In terms of throughput, results can be summarized as follows. With SD Cards, eMMC and HDDs (single and RAID), there is no regression in terms of throughput. In contrast, with HDDs, there is a gain around 20-30% with some workload.

On SSDs, there is a loss of throughput only

• with random sync I/O: around 2-3 % on average SSDs, up to 10-15% on very fast NVMe SSDs. With a workload meant to put BFQ in the most difficult condition, we reached a loss of 18% [1], but in any other third-party test the loss is around 10% in the worst case. This loss is mainly due to the fact that BFQ is not a minimal I/O scheduler. We are working on this. It is not easy; we will need time to fill this gap.
• with only-write I/O on very fast SSDs: around 5-10%. This is due to a problem with I/O-request tags. We have already found a solution. Since we do not consider this issue critical, we are giving more priority to other items in our TODO list. If you think otherwise, we are willing to change our priorities.

Because of the above overhead, BFQ cannot process more than 400-500 KIOPS on a commodity CPU.

In terms of responsiveness and latency for time-sensitive applications (such as audio/video players), results are simply incomparable. For example, regardless of the I/O workload in the background, with BFQ applications start as quickly as if the drive was idle. With any of the other schedulers, applications may take ten times as long, or even not start at all (until the background workload is over) [1].

In addition, as for server-like workloads, BFQ enable, e.g., the desired fraction of the I/O bandwidth to be guaranteed to each client (or container, VM, or any other kind of entity sharing storage), while reaching a total throughput not comparable to that reached by any other solution for controlling I/O [2].

Finally, if you are in doubt about some particular workload, we will be glad to test it.

[1] http://algo.ing.unimo.it/people/paolo/disk_sched/results.php

[2] https://lwn.net/Articles/763603/