3

I've been reading Linux Kernel Development and there's something that's not entirely clear to me -- when an interrupt is triggered by the hardware, what's the criterion to decide on which CPU to run the interrupt handling logic?

I could imagine it having to be always the same CPU that raised the IO request, but as the thread is for all purposes now sleeping there would not really be that much of a point in doing that.

On the other hand, there may be timing interrupts (for the scheduler, for instance) that need to be raised. On an SMP system are they always raised on the same core (let's say, #0) or they're always pretty much raised at any core?

How does it actually work?

Thanks

3
+50

On a multiprocessor/multicore system, you might find a daemon process named irqbalance. Its job is to adjust the distribution of hardware interrupts across processors.

At boot time, when the firmware hands over the control of the system to the kernel, initially just one CPU core is running. The first core (usually core #0, sometimes called the "monarch CPU/core") initially takes over all the interrupt handling responsibilities from the firmware before initializing the system and starting up the other CPU cores. So if nothing is done to distribute the load, the core that initially started the system ends up with all the interrupt handling duties.

https://www.kernel.org/doc/Documentation/IRQ-affinity.txt suggests that on modern kernels, all CPU cores are allowed to handle IRQs equally by default. But this might not be the optimal solution, as it may lead to e.g. inefficient use of CPU cache lines with frequent IRQ sources. It is the job of irqbalance to fix that.

irqbalance is not a kernel process: it's a standalone binary /usr/sbin/irqbalance that can run either in one-shot mode (i.e. adjust the distribution of interrupts once as part of the boot process, and exit) or as a daemon. Different Linux distributions can elect to use it differently, or to omit it altogether. It allows easy testing and implementation of arbitrarily complex strategies for assigning IRQs to processors by simply updating the userspace binary.

It works by using per-IRQ /proc/irq/%i/smp_affinity files to control which IRQs can be handled by each CPU. If you're interested in details, check the source code of irqbalance: the actual assignment of IRQ settings happens in activate.c.

  • I can see irqbalance running in my Ubuntu 16.04 LTS. – devoured elysium Jun 22 at 15:19
  • irqbalance is not a kernel process: it's a standalone binary /usr/sbin/irqbalance that can run either in one-shot mode (i.e. adjust the distribution of interrupts once as part of the boot process, and exit) or as a daemon. – telcoM Jun 22 at 15:27
  • I'm curious how does userspace code do this, is there any /sys file you can read/write or there's some syscall I don't know? – 炸鱼薯条德里克 Jun 22 at 15:35
  • If I recall correctly, back when I encountered my first Linux multiprocessor systems, the IRQs definitely used to get dumped on CPU0 if irqbalance was not used. But as the multi-core CPUs became ubiquitous, obviously the kernel defaults have been updated at some point. kernel.org/doc/Documentation/IRQ-affinity.txt suggests that the kernel default is to allow all CPUs to handle IRQs. – telcoM Jun 22 at 16:28
  • And even the IRQ-affinity.txt is effectively from year 2011, so modern kernels may actually have some semi-smart default strategy to guarantee reasonable results even with a stripped-down userspace with no irqbalance. – telcoM Jun 22 at 16:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.