1

In my program, I have several threads that are started with the process and that remain until the program ends. They will meet different loads during the lifetime of the app, and at times they will all run at 100%.

By default the Linux thread scheduler will change affinity on a multi-core system for these threads quite frivolously IMO. When I look at the bouncing graphs in my graphical process monitor (the one in gnome) I can't help but think that this constitutes some kind of overhead.

EDIT: To clarify, even for very stable loads, the threads are scheduled on different cores, and even though it is not visible in the image provided it is at times very clear that the core selected for each thread is "swapped" frequently.

the threads here are actually running at constant loads

Will not this constant change in affinity affect performance adversely?

In that case, why is it implemented this way? What benefits does the changing affinity have?

My guesses are:

  • Wear levelling - Don't put all the work on one core
  • Unintentional - Some smart algorithm tries to optimize usage depending on load and it so happens that the overhead of is not significant enough to warrant keeping the affinity over changing it.
  • 1
    Let's not mix up affinity with multiprocessor scheduling in general. If your thread keeps jumping from one core to the next, then it has no affinity at all. The kernel does not change your thread's affinity: at time t, it just lets it run on a CPU which happens to be available. Scheduling is not about changing a process'/thread's affinity at regular intervals. – John WH Smith Oct 9 '14 at 22:07
  • You say affinity is not related to scheduling. To rephrase my question with this in mind; won't letting the scheduler pin affinity ("mixing them") have the potential of improve scheduling? – Lennart Rolland Oct 9 '14 at 22:40
1

If you're going to run all the threads on one core, buy cheaper hardware with a single core.

The scheduler tries to make maximum use of all the cores. This means dispatching threads to whatever core has some free time. Moving a thread from one core to another core has a small cost, so the scheduler does try to avoid this. But you'll typically not notice it much, because the benefit of not letting a core go idle is far more than the cost of migrating a thread. This is especially true for if the threads use more memory than you have core-local cache: if the memory used by a thread isn't in the core-local cache, there is very little to be lost by migrating it to another core.

Second-guessing an industrial-grade scheduler such as Linux's usually makes performance worse.

The graphs that you show indicate that the load on the various cores is not full and mildly variable, presumably because your system as a whole is limited by I/O for the tasks it's doing right now, not by CPU power. It doesn't say anything one way or the other as to how often threads move from one core to another.

  • I know many many use-cases where setting the affinity is beneficial. One example would be a render graph with many cpu intensive nodes. Distributing nodes smartly over cores and setting affinity would preserve caches all over, so the industrial-grade scheduler in Linux is missing out on plenty of optimizations! – Lennart Rolland Oct 30 '14 at 1:34
1

The snapshot provided here depends on the type(version) of kernel too. Older kernels with version 2.4 had poor affinity which caused lot of ping pong movement of threads impacting the performance of the system. Kernel versions from 2.5 have relatively better affinity.

On a multi-core based system, setting of affinity can improve performance, by avoiding the cache invalidation occurrence while moving a thread across the cores.

In case of linux based multi-core system, the affinity behavior of scheduler(natural affinity) can be overridden based on the type of application/requirement by using the sched_setaffinity/taskset for process and pthread_setaffinity_np for thread.

kernelshark appears to provide better visual representation of multi-core system & affinity.

Also, note that htop provide visual support to set affinity (to override the scheduler).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.