2

The concepts of IO scheduler and CPU scheduler confuse me. Below are my understanding:

  1. Linux uses CFS scheduler + nice values by default to schedule processes.
  2. Each process has an IO queue.
  3. There is an IO scheduler kernel thread.
  4. IO scheduler is in the block level, not in the file level.
  5. IO scheduler is a module of the file system.

Questions:

What is relationship between IO scheduler and CPU scheduler? Conceptually, it seems to me that CPU scheduler is superior over IO scheduler. CPU scheduling happens first. IO scheduler is a thread itself and subject to CPU scheduling.

A contrived scenario looks like this:

Step 1: CPU scheduler picks a process P1 to execute

Step 2: P1 puts IO requests in its own IO queue

Step 3+: CPU scheduler picks other threads to run. (Assuming no process has IO other than P1)

....(after a while)

Step n: CPU scheduler picks the IO scheduler thread to run.

Step n+1: IO scheduler thread 'notices' P1 has IO requests queued up and issues those requests to disk.

Does my understanding and the scenario make sense?

5

Let's start with the IO scheduler first. There's a IO scheduler per block device. Its job is to schedule (order) the requests that pile up in the device queue. There are three different algorithms currently shipped in the linux kernel: deadline, noop and cfq. cfq is the default, and according to its doc:

The CFQ I/O scheduler tries to distribute bandwidth equally among all processes in the system. It should provide a fair and low latency working environment, suitable for both desktop and server systems

You can configure which scheduler governs which device via the scheduler file corresponding to your block device under /sys/ (You can issue the following command to find it: find /sys | grep queue/scheduler).

What that short description doesn't say is that cfq is the only scheduler that looks at the ioprio of a process. ioprio is a setting that you can assign to the process, and the algorithm will take that into account when choosing a request before another. ioprio can be set via the ionice utility.

Then, there's the task scheduler. Its job is to allocate the CPUs amongst the processes that are ready to run. It takes into account things like the priority, the class and the niceness of a give process, as well as how long that process has run and other heuristics.

Now, to your questions:

What is relationship between IO scheduler and CPU scheduler?

Not much, besides the name. They schedule different shared resources. The first one orders the requests going to the disks, and the second one schedules the 'requests' (you can view a process as requesting CPU time to be able to run) to the CPU.

CPU scheduling happens first. IO scheduler is a thread itself and subject to CPU scheduling.

It doesn't happen like the the IO scheduler algorithm is run by whichever process is queuing a request. A good way to see this is to look at crashes that have elv_add_request() in their path. For example:

 [...]
 [<c027fac4>] error_code+0x74/0x7c
 [<c019ed65>] elv_next_request+0x6b/0x116
 [<e08335db>] scsi_request_fn+0x5e/0x26d [scsi_mod]
 [<c019ee6a>] elv_insert+0x5a/0x134
 [<c019efc1>] __elv_add_request+0x7d/0x82
 [<c019f0ab>] elv_add_request+0x16/0x1d
 [<e0e8d2ed>] pkt_generic_packet+0x107/0x133 [pktcdvd]
 [<e0e8d772>] pkt_get_disc_info+0x42/0x7b [pktcdvd]
 [<e0e8eae3>] pkt_open+0xbf/0xc56 [pktcdvd]
 [<c0168078>] do_open+0x7e/0x246
 [<c01683df>] blkdev_open+0x28/0x51
 [<c014a057>] __dentry_open+0xb5/0x160
 [<c014a183>] nameidata_to_filp+0x27/0x37
 [<c014a1c6>] do_filp_open+0x33/0x3b
 [<c014a211>] do_sys_open+0x43/0xc7
 [<c014a2cd>] sys_open+0x1c/0x1e
 [<c0102b82>] sysenter_past_esp+0x5f/0x85

Notice how the process enters the kernel calling open(), and this ends up involving the elevator (elv) algorithm.

  • A follow-up question: elv_add_request() adds the block requests to a queue and then flush the requests in order to disk. Is it preemptive? The process can be switched to another process that also pends IO requests to the same queue. Right? – Junji Zhi Apr 30 '15 at 12:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.