7

The Linux scheduler keeps evolving: How does the kernel schedule processes vs threads today? Do I need autogroups?

The previous similiar stack overflow question is many years old and likely obsolete.

I believe in 2018 the default is the Completely Fair Scheduler (CFS) instead of O(1) or other scheduler.

For Linux, some documentation mentions the OS juggles tasks which indifferently represent a process or a thread of a process, or does not clearly make the distinction though it's critical for scheduling.

For sake of clarity, a Process runs a program, and may spawn 1 or more threads when it is multi-threaded. For me the distinction is all threads of a process use the same virtual address space. However for scheduling on a CPU that distinction is irrelevant.

If I run P processes each with different Tp thread counts:

  • Process fairness: How can I guarantee a process with many threads won't use up all resources and overpower a process with only 1 thread? Formalized, fair scheduling would mean: process P should get 1/Pth of the CPU resource, and have that 1/Pth split evenly to 1/(Pth*Tp) per thread of P. Is it what CFS guarantees by default?
  • Do I need autogroups? Autogroups allow a process to schedule groups of tasks together. By default there's one group, but if you spin a 2nd one it will get 1/2 the CPU, spin 10 and each group of threads gets 1/10th. Simple (see gang scheduling).
  • multi-core: I do not care too much to muddy the answer with considerations on load balancing for multi-cores and process migration between cores.

Experimental result: CFS seems to be fair between processes.

I tried on 4.9.27 to monitor the runtime of a small process vs a long running process with many threads. It seems that the scheduler on my kernel schedules by process, not by threads. So the small process was treated fairly and got 50% of the CPU. When I used 2 long running processes I got 33%. The slowdowns were like 2.2x and 3.4x respectively.

This kernel doesn't seem to have autogroups compiled in; setsid() works but seems to do nothing, AND /proc/*/autogroup doesn't exist, so trying to use autogroup while CFS seems to do the right thing is an expensive undertaking. However some other data points seem to indicate sometimes the behavior is not fair to a process.

Side question

Is there a trick to guarantee one process will run more regularly than all others? I can't promote it to realtime because of LD_LIBRARY_PATH usage I believe. And even then, I suspect because it reads /proc to monitor the system, it sometimes still got delayed massively when the system is heavily over capacity.

Thanks!

1
  • Did you ever find answers by yourself?
    – drerD
    Commented Oct 26, 2021 at 8:56

1 Answer 1

1

Answering regarding cpu-bound tasks (since fairness considerations are more trivial to observe than with io-bound tasks) running concurrently on the same core under SCHED_OTHER scheduling policy (since it is the only true time-sharing scheduling policy)


Processes (in the meaning you give to that term inherited from UNIX systems) are not scheduling entities. Only threads are and CFS schedules them irrespective on any parentness considerations. Quoting man sched :

   The thread to run is chosen from the static priority 0 list based
   on a dynamic priority that is determined only inside this list.
   The dynamic priority is based on the nice value (see below) and
   is increased for each time quantum the thread is ready to run,
   but denied to run by the scheduler.  This ensures fair progress
   among all SCHED_OTHER threads.

As a consequence of this, whatever multi-threaded application can globally receive more cpu power than any single-threaded application running concurrently on the same core, as many times demonstrated (read from §3) period. 1


Linux kernel control group support :

If adequately configured (CONFIG_CROUPS=y) the kernel offers the facility to group tasks together into… just guess : tasks groups ! :-P and incidentally to make other programs (fill data structures) such as memory controllers and, of course, CFS, aware of these groupings.

Then, if adequately configured (CONFIG_CGROUP_SCHED=y), CFS will control CPU bandwidth allocation in order to ensure fairness amongst all existing tasks groups.2

In this context (CONFIG_CROUPS=y && CONFIG_CGROUP_SCHED=y) we can then rephrase the above statement :

Whatever multi-threaded application will globally receive more cpu power than any single-threaded application belonging to the same tasks group but without exceeding the cpu power allocated to any other co-existing tasks group. 3


Autogrouping

Because grouping tasks requires user explicit action (if not early specific system configuration) and because most common desktop users don't want to bother with that but still want their desktop to stay responsive irrespective of whatever user doing in its own session, the kernel offers the facility to automatically create and populate tasks groups on a per session basis.
If CONFIG_SCHED_AUTOGROUP is set, one task group is created per session and all tasks launched within this session will belong to it.4


Your side question ("is there a trick to guarantee one process will run more regularly than all others?") appears almost non sensible since the algorithm used for scheduling SCHED_NORMAL threads is deterministic and will ensure the best possible regularity. In that context, you just cannot expect reaching "more regularity", and even less more regularity for some threads unless wishing to… break regularity.
You might wish threads being scheduled more or less frequently though. In which case simply adjust their nice value.


1 : verbi gratia : without any extra care, do not expect enjoying your video launched while building chromium -j64 :-P

2 : No ! Contrarily to what the config label would suggest, CFS is not going to start scheduling tasks groups. It will still schedule threads but taking into account in its election the sum of cpu time allocated to each of the other threads belonging to the same task group; ensuring it does not exceed the total time allocated to the threads belonging to an other existing task group.

3 : verbi gratia : If you specifically take care to ensure your chromium build and your video player do not belong to the same task group, then you might enjoy your video running in parallel with a make -j64 even on a core 2. If you don't then… goto 1 :-P

4 : verbi gratia : If you just take care launching your chromium build and your player in two different sessions then just enjoy. If you launch them into the same session then… goto 1 :-P

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .