0

I am monitoring disk utilization via prometheus-node-exporter which reads /proc/diskstats

My expectation would be that if the block device is 100% busy then the field 7 - time spent reading (ms) will increase 1000 (in miliseconds) per second.

But currently when the disk is busy,the increase is between 1200 - 1800 ms per second.

What could be the cause of this? Thanks.

13 - time spent doing I/Os (ms) is increasing as expected less than 1000ms per second.

the HDD is Seagate ST8000AS0022-1WL17Z Host-Aware SMR

Debian Bullseye, kernel version 5.4.0-4-amd64
32GB RAM i5 3570,

device structure:
sde -> loop1 -> dm-crypt -> Virtual machine Virtio -> ZFS
the loop device is used because otherwise dm-crypt doesn't work

scheduler mq-deadline
Improve this question

1 Answer 1

Reset to default
0

Disk stats are tracked per-CPU, and /proc/diskstats shows the sum of stats over all CPUs (apart from tick-based stats such as “time spent doing I/O”). The times spent reading and writing are the sums of the durations of all reads and writes across all CPUs, so on systems with multiple CPUs the stats can increase faster than real time.

Improve this answer
5
  • Is it possible that one disk at one time can serve several CPUs in parallel? If not, the sum of per-CPU per second cannot be more than second..
    – z0lupka
    Oct 4, 2021 at 17:58
  • Also I have 200000 difference per second on 11 time spent writing (ms) on virtual machine with just 4 cores. So it seems that disk is busy 50 second per second..
    – z0lupka
    Oct 4, 2021 at 18:03
  • 1
    Yes, one disk can serve multiple CPUs in parallel. I don’t understand what you mean by “200000 difference”, sorry.
    – Stephen Kitt
    Oct 4, 2021 at 18:50
  • I mean that f.e. now it is 100200000, but one second ago it was 100000000. So even if 200000/4 = 50000 ms, i.e. [50 seconds per cpu] per second. It is impossible. The disk can't serve one CPU more than second per second..
    – z0lupka
    Oct 5, 2021 at 8:31
  • 1
    Asked in separate question: unix.stackexchange.com/questions/671919/…
    – z0lupka
    Oct 5, 2021 at 10:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.