What do “writes / reads issued” refer to in atop's DSK label?

Question

Context: I'm writing a script that calculates a service's I/O usage from atop history for the last 2 minutes (where atop's sampling is configured to be per 1 minute). I'm using the following command to generate the history file:

atop -P DSK,PRD -b [time] -e [time] -r > somefile_to_read_from

I'm using atop's parseable output option (-P) and the labels DSK and PRD.

From atop's manual page, it says this about DSK:

For every logical volume/multiple device/hard disk one line is shown. Subsequent fields: name, number of milliseconds spent for I/O, number of reads issued, number of sectors transferred for reads, number of writes issued, and number of sectors transferred for write.

While for PRD it says:

For every process one line is shown. Subsequent fields: PID, name (between brackets), state, obsoleted kernel patch installed ('n'), standard io statistics used ('y' or 'n'), number of reads on disk, cumulative number of sectors read, number of writes on disk, cumulative number of sectors written, cancelled number of written sectors, TGID (group number of related tasks/threads) and is_process (y/n).

I assumed they would be the same thing. However, I almost always get values way above 100% for the I/O usage (for example when running ab for apache). I thought that it would be a problem coming from my programming logic and algorithm, however, I banged my head in the wall for hours and couldn't think of a mistake I might've done, tried a lot of different ways to calculate it, still getting the same results.

So then I opened and started reading the history file I generated line by line after filtering it to show me only the process that I've monitored to have such I/O usage (apache in this case, since I ran benchmarks on it). And I noticed something, that was the fact, that DSK's numbers of writes issued was way lower than the sum of all the apache's PRD lines' number of writes on disk.

I'm not sure if I've understood something wrong or what am I doing wrong. The history file is too large to show, however, I can upload it to something like pastebin if needed.

My questions is, what does DSK's numbers of writes/reads issued refer to, isn't it the same as PRD's number of reads/writes on disk? And if not, what would be a way to calculate the I/O usage for a single process by using atop's history?

Answer 1

First of all my man atop says:

The counters 'number of reads on disk' and 'number of writes on disk' are obsoleted anyhow.

atop Version: 2.3.0 - 2017/03/25 09:59:59

From man iostat:

A transfer is an I/O request to the device. Multiple logical requests can be combined into a I/O request to the device.

I think this explains why the sum of process I/O exceeds values from DSK.

So decently accurate I/O usage of a single process would be process_io / sum_of_all_process_io. It's not 100% accurate, since there is no way (that I know of) to determine how exactly logical requests where combined.

Answer 2

I can be absolutely wrong, but it may be related to the fact of file system IO buffering as well as drive sector size and the size of IO. For example, if a disk block size is 512 bytes and application is writing 1024 bytes, then 1 application IO equals to 2 IOs on the drive. Imagine now that between the application and the drive there are at least a filesystem and a volume manager and both of them may have their own block-sizes.

Answer 3

I think that your results are correct, and are a result of efficient disk IO. In a write-back (stackoverflow) system the number of writes issued should be less than the the actual writes to disk, whereas in a write-through system the sum of the quantity of writes issued should be equal to the number of writes to disk because there is no write-combining (wikipedia).

From Webopedia:

Write-back caching yields somewhat better performance than write-through caching because it reduces the number of write operations to main memory. With this performance improvement comes a slight risk that data may be lost if the system crashes.

Because of this, atop's DSK label is a more representative value for the actual disk IO taking place in write-back systems.

For per-process io this serverfault question might help.

This Huawei forum thread has a nice description of write-through vs write-back, assuming that is what's influencing your output.