sudo dd if=/dev/sda of=/dev/null bs=1M iflag=direct
atopsar -d 5 # in a second terminal
top # in a third terminal
19:18:32 disk busy read/s KB/read writ/s KB/writ avque avserv _dsk_ ... 19:16:50 sda 18% 156.5 1024.0 0.0 0.0 5.0 1.15 ms 19:16:55 sda 18% 156.3 1024.0 0.0 0.0 4.9 1.15 ms ...
Why is disk utilization ("busy") reported as much less than 100% ?
dd process only uses 3% of a CPU or less.
top also provides an overall report of hardware and software interrupt (
si) usage of the system's CPU's, which shows as less than 1%. I have four CPUs (2 cores with 2 threads each).
/dev/sda is a SATA HDD. It is not an SSD, it is not even a hybrid SSHD drive. It cannot read faster than about 150 megabytes per second :-). So that part of the results makes sense: 156 read/s * 1024 KB/read = 156 MB/s
The kernel version is
5.0.9-200.fc29.x86_64. I have a fairly default, straightforward install of Fedora Workstation 29. It is not a VM. The IO scheduler is
Since kernel version 5.0, Fedora uses the multi-queue block layer. Because the single queue block layer has been removed :-).
I believe the disk utilization figure in
atopsar -d and
atop is calculated from one of the kernel iostat fields. The linked doc mentions "field 10 -- # of milliseconds spent doing I/Os". There is a more detailed definition as well, although I am not sure that the functions it mentions still exist in the multi-queue block layer. As far as I can tell, both
atopsar -d and
atop use common code to read this field 10. (I believe this field is also used by
sar -d /
iostat -x /
Variant 2: Changing to
bs=512k, but keeping
dd if=/dev/sda of=/dev/null bs=512k iflag=direct
19:18:32 disk busy read/s KB/read writ/s KB/writ avque avserv _dsk_ ... 19:18:00 sda 35% 314.0 512.0 0.0 0.0 2.1 1.12 ms 19:18:05 sda 35% 313.6 512.0 0.2 4.0 2.1 1.11 ms
Variant 3: Using
bs=1M, but removing
dd uses about 10% CPU, and 35% disk.
dd if=/dev/sda of=/dev/null bs=1M
19:18:32 disk busy read/s KB/read writ/s KB/writ avque avserv _dsk_ ... 19:21:47 sda 35% 242.3 660.2 0.0 0.0 5.4 1.44 ms 19:21:52 sda 31% 232.3 667.8 0.0 0.0 9.5 1.33 ms
How to reproduce these results - essential details
Beware of the last test, i.e. running
It is a bit of a hog. I saw it freeze the system (mouse cursor) for ten seconds or longer. Even when I had swap disabled. (The test fills your RAM with buff/cache. It is filling the inactive LRU list. I think the turnover evicts inactive cache pages relatively quickly. At the same time, the disk is busy with sequential reads, so it takes longer when you need to page something in. How bad this gets probably depends on whether the kernel ends up also turning over the active LRU list, or shrinking it too much. I.e. how well the current "mash of a number of different algorithms with a number of modifications for catching corner cases and various optimisations" is working in your case).
The exact results of the original test are difficult to reproduce.
KB/read shows as
512 instead of
1024. In this case, the other results look more like the results from
bs=512k. Including that it shows a disk utilization around 35%, instead of around 20%. So my question stands in either case, the difference is just a bit confusing.
I thought the difference might be due to some IO from other processes... e.g. when I open Firefox to start putting my results into this question... but I've also seen it without Firefox running.
I think the test tends to show
KB/read = 1024 if I run it after rebooting, logging in, and waiting for the system IO to settle down to zero (e.g. finish PackageKit checking) first.
Some additional information was requested. Here is the additional information, but in my opinion it does not shed any more light on the question.