I have been reading about performance tuning of Linux to get the fastest packet processing times when receiving financial market data. I see that when the NIC receives a packet, it puts it in memory via DMA, then raises a HardIRQ - which in turn sets some NAPI settings and raises a SoftIRQ. The SoftIRQ then uses NAPI/device drivers to read data from the RX Buffers via polling, but this is only run for some limited time (net.core.netdev_budget, defaulted to 300 packets). These are in reference to a real server running ubuntu, with a solarflare NIC My questions are below:

  1. If each HardIRQ raises a SoftIRQ, and the Device Driver reads multiple packets in 1 go (netdev_budget), what happens to the SoftIRQs raised by each of the packets that were drained from the RX buffer in 1 go (Each pack received will raise a hard and then soft irq)? Are these queued?

  2. Why does the NAPI use polling to drain the RX_buffer? The system has just generated a SoftIRQ and is reading the RX buffer, then why the polling?

  3. Presumably, draining of the RX_Buffer via the softirq, will only happen from 1 specific RX_Buffer and not across multiple RX_Buffers? If so, then increasing the netdev_budget can delay the processing/draining of other RX_buffers? Or can this be mitigated by assigning different RX_buffers to different cores?

  4. There are settings to ensure that HardIRQs are immediately raised and handled. However, SoftIRQs may be processed at a later time. Are there settings/configs to ensure that SoftIRQs related to network RX are also handled at top priority and without delays?

  • VM or real server? If VM, which virtualisation solution? What is the brand and model of the NIC? – Rui F Ribeiro Jun 22 '16 at 12:49
  • 1
    Its a real server, running ubuntu. Its a solarflare NIC – Nidhi Jun 22 '16 at 13:00
  • @Nidhi what would solarflare have to do with this? I'm curious – Johnny V Aug 19 '16 at 1:15

Regarding answer to the 4th question,

Yes Network NIC cards are the peripheral which are associated with the softirq, As softirq is the highest priority in all bottom half mechanism.

So to prohibit the delay that might lead to the packet drop afterwards, softirqs are used.

Basically NAPI mechanism is also intended to handle the packets those who are coming with a speed which kernel can not handle with the interrupt mechanism due to some reasons.

I would suggest just go for the chapter of network drivers in LDD3. Here is the link which can be helpful for the same.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.