5

I am reading a large file sequentially from the disk and trying to understand the iostat output while the reading is taking place.

  • Size of the file : 10 GB
  • Read Buffer : 4 KB
  • Read ahead (/sys/block/sda/queue/read_ahead_kb) : 128 KB

The iostat output is as follows

Device:         rrqm/s   wrqm/s     r/s     w/s    rMB/s    wMB/s avgrq-sz avgqu-sz await r_await w_await  svctm  %util
sda               0.00     0.00  833.00   14.00   103.88     0.05   251.30     6.07    5.69    2.33 205.71  1.18 100.00

Computing the average size of an I/O request = (rMB/s divided by r/s) gives ~ 128 KB which is then the read-ahead value. This seems to indicate that while the read system call has specified a 4KB buffer, the actual disk I/O is happening according to the read-ahead value.

When I increased the read-ahead value to 256KB, the iostat output was as follows

Device:         rrqm/s   wrqm/s     r/s     w/s    rMB/s    wMB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sda               0.00    28.00  412.00   12.00   102.50     0.05   495.32    10.78   12.15    4.76  265.83   2.36 100.00

Again the average I/O request size was 256 KB matching the read ahead.

This kept up until I set 512 KB as the read-ahead value and did not hold up when I moved up to a read-ahead value of 1024 KB - the average size of the I/O request was still 512 KB. Increasing max_sectors_kb (maximum amount of data per I/O request) from the default of 512 KB to 1024 KB also did not help here.

Why is this happening - ideally I would like to minimize my read IOPS as much as possible and read a larger amount of data per I/O request (larger than 512 KB per request). Additionally, I am hitting 100% disk utilization in all cases - I would want to throttle myself to read at 50-60% disk utilization with good sequential throughput. In short, what are the optimized application/kernel settings for sequential read I/O?

1 Answer 1

1

You're likely being limited by the max_sectors_kb setting for /dev/sda.

cat /sys/block/sda/queue/max_sectors_kb

almost certainly returns 512.

You should be able to change it to 1024 with something like:

echo 1024 > /sys/block/sda/queue/max_sectors_kb

Assuming you don't run into another limit, that should up the IO size you see to 1024 KB.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .