Differences between system processes, and user processes, kernel control paths and kernel thread

Question

My understanding is that

• a process always runs in user mode and uses user space only, and
• a kernel always runs in kernel mode and uses kernel space only.

But I feel that I might not be correct, after reading the following two books. Could you correct me if I am wrong?

1. In Linux Kernel Architecture by Maurer, the terms "system process" and "user process" are used without definitions, for example, when introducing division of virtual address space into kernel space and user space:

   Every user process in the system has its own virtual address range that extends from 0 to TASK_SIZE . The area above (from TASK_SIZE to 2 32 or 2 64 ) is reserved exclusively for the kernel — and may not be accessed by user processes. TASK_SIZE is an architecture-specific constant that divides the address space in a given ratio — in IA-32 systems, for instance, the address space is divided at 3 GiB so that the virtual address space for each process is 3 GiB; 1 GiB is available to the kernel because the total size of the virtual address space is 4 GiB. Although actual figures differ according to architecture, the general concepts do not. I therefore use these sample values in our further discussions.

   This division does not depend on how much RAM is available. As a result of address space virtualization, each user process thinks it has 3 GiB of memory. The userspaces of the individual system processes are totally separate from each other. The kernel space at the top end of the virtual address space is always the same, regardless of the process currently executing.

   ... The kernel divides the virtual address space into two parts so that it is able to protect the individual system processes from each other.

   You can read more examples, by searching either "user process" or "system process" in the book.

   Are both user processes and system processes processes, as opposed to kernel?

   What are their definitions? Do they differ by their owners (regular user or root?), by the user who started them, or by something else?

   Why does the book explicitly write "system process" or "user process", rather than just "process" to cover both kinds of "processes", for example, in the above quote? I guess what it says about "user process" also applies to "system process", and what it says about "system process" also applies to "user process".

2. In Understanding Linux Kernel by Bovet, there are concepts "kernel control path" and "kernel thread".

   A kernel control path denotes the sequence of instructions executed by the kernel to handle a system call, an exception, or an interrupt.

   ... Traditional Unix systems delegate some critical tasks to intermittently running processes, including flushing disk caches, swapping out unused pages, servicing network connections, and so on. Indeed, it is not efficient to perform these tasks in strict linear fashion; both their functions and the end user processes get better response if they are scheduled in the background. Because some of the system processes run only in Kernel Mode, modern operating systems delegate their functions to kernel threads, which are not encumbered with the unnecessary User Mode con- text. In Linux, kernel threads differ from regular processes in the following ways:

   • Kernel threads run only in Kernel Mode, while regular processes run alterna- tively in Kernel Mode and in User Mode.

   • Because kernel threads run only in Kernel Mode, they use only linear addresses greater than PAGE_OFFSET . Regular processes, on the other hand, use all four gigabytes of linear addresses, in either User Mode or Kernel Mode.

   You can read more by searching them at Google Books.

   Are "system process" in Maurer's book and in Bovet's book the same concept?

   Can "system process" mentioned in the two books run in user space, kernel space, or both?

   Is "system process" different from kernel control path and kernel thread?

Answer 1

Q: Are both user processes and kernel processes processes, as opposed to kernel?

I'm not sure if there is a single correct answer, but I'll give it a try.
Citing from "Operating Systems Design & Implementation" (A. Tanenbaum), 3rd edition, Chapter 2.1 says:

2.1. Introduction to Processes

All modern computers can do several things at the same time. While running a user program, a computer can also be reading from a disk and outputting text to a screen or printer. In a multiprogramming system, the CPU also switches from program to program, running each for tens or hundreds of milliseconds. While, strictly speaking, at any instant of time, the CPU is running only one program, in the course of 1 second, it may work on several programs, thus giving the users the illusion of parallelism. Sometimes people speak of pseudoparallelism in this context, to contrast it with the true hardware parallelism of multiprocessor systems (which have two or more CPUs sharing the same physical memory). Keeping track of multiple, parallel activities is hard for people to do. Therefore, operating system designers over the years have evolved a conceptual model (sequential processes) that makes parallelism easier to deal with. That model, its uses, and some of its consequences form the subject of this chapter.

2.1.1. The Process Model

In this model, all the runnable software on the computer, sometimes including the operating system, is organized into a number of sequential processes , or just processes for short. A process is just an executing program, including the current values of the program counter, registers, and variables.

(emphasis mine)

While I have not yet gotten around to finish reading the book, according to this explanation a "process" is a unit of work that's executed on the processor and holds all necessary resources (image, state, registers, counters...).

Answer to edited question

a kernel always runs in kernel mode and uses kernel space only.

That depends on the type of the kernel. A monolithic kernel runs its stuff in a single address space (kernel space), while microkernels may run their kernel processes in user space.

Can "system process" mentioned in the two books run in user space, kernel space, or both?

See above, a system process may run in both modes, depending on the type of kernel.

Are both user processes and system processes processes, as opposed to kernel?

Yes, both user processes and system processes are processes - hence the naming ;-) I don't understand the part after the comma, though.

Is "system process" different from kernel control path and kernel thread?

Yes. A process (user or system=kernel) is something different.

The kernel control path denotes the sequence of instructions, a kernel thread (aka LWP - lightweight process) is a thread that's created and scheduled directly be the kernel (as opposed to user threads, which are created by a threading lib).

Conclusion

A process is just a theoretical structure.
A kernel is a part of an OS that implements the concept of processes to allow eg. scheduling of said processes.
A thread is the smallest part of a process that can be scheduled independently.

Answer 2

In the case of Linux, a task (kernel internal idea of a thread; threads can share resources, like memory and open files; some run only inside the kernel) can run in userland, or (it's thread of execution) can transfer into the kernel (and back) to execute a system call. A user thread can be highjacked temporarily to execute an interrupt (but that isn't really that thread running).

That a process is a "system process" or a regular user process is completely irrelevant in Unix, they are handled just the same. In Linux case, some tasks run in-kernel to handle miscellaneous jobs. They are kernel jobs, not "system processes" however.

One big caveat: Text books on complex software products (compilers and operating systems are particularly egregious examples) tend to explain simplistic algorithms (often ones that haven't been used in earnest for half a century), because real world machines and user requirements are much too complex to be handled in some way that can be described in a structured, simple way. Much of a compiler is ad-hoc tweaks (particularly in the area of code optimization, the transformations are mostly the subset of possibilities that show up in practical use). In the case of Linux, most of the code is device drivers (mentioned in passing as device-dependent in operating system texts), and of this code a hefty slice is handling misbehaving devices, which do not comply to their own specifications, or which behave differently between versions of "the same device". Often what is explained in minute detail is just the segment of the job that can be reduced to some nice theory, leaving the messy, irregular part (almost) completely out. For instance, Cris Fraser and David Hanson in their book describing the LCC compiler state that typical compiler texts contain mostly explanations on lexical analysis and parsing, and very little on code generation. Those tasks are some 5% of the code of their (engineered to be simple!) compiler, and had negligible error rate. The complex part of the compiler is just not covered in standard texts.

Answer 3

Try to make a short but hopefully clear it all answer here. Only applied to modern linux kernel.

There are struct tasks, used internally by kernel as minimal schedulable unit, which is always created by kernel code using ring0 and starts running in ring0, but may or may not switch to ring3 later, or switch back to ring0(using the platform specific syscall instructions) later.

A task has many resources or say properties, the most important ones, are, memory space, task id(tid viewd from top-level pid namespace), task group id(pid viewd from top level of namespace). Although in ring3, tasks obviously can't access even it's own struct task directly but at least thoes properties I just mentioned are exposed to ring3 by kernel through /procfs. But modern kenerl might expose it in a weird way, like /proc/[pid]/status use the word "Pid" to refer to task id viewd from the pid namespace associated with its procfs instance.

A task may or may not share these properties with other tasks(except tid viewd from top-level pid namespace), like they might sharing exactly the same one memory space, or having the same task group id.

Now users in ring3, or say userspace, can invent the concept of "thread" and "process". But since they're purely userspace invention, different people/textbook might use different terminology. So here we only talk about commonly used terminology.

OS-Thread(thread): a synonym of task.

Kernel-Thread: a task that never switch to ring3 and always share the memory space with the kernel.

OS-Process(process): Kernel doesn't track process's existence, human just define it as one or more tasks that having the same task group id (pid viewd from top-level pid namespace). Once all these threads died, the process disappeared in human mind.

One important fun fact is, as you should be able to conclude from the definitions above, a OS-Process might contain one or more OS-Threads, these OS-Threads, or say tasks, in one OS-Process are actually more restricted by kernel about the properties they can unshare.

Once a task is created, it belongs to some certain process, and this relationship can never change until it dies.

Tasks belonging to one process must share the exact same one memory space.

Tasks belonging to one process can't stay in different user namespace or pid namespace, these must be shared.

Answer 4

Q. Are "system process" in Maurer's book and in Bovet's book the same concept?

I see Maurer uses system processes as a general term to cover both user processes and kernel threads.

From what I can see, Bovet is not using system process to define any more specific concept than Maurer. He might be using less strict language though. So I would take care not to equate them directly. I have difficulty with the sentence

Because some of the system processes run only in Kernel Mode, modern operating systems delegate their functions to kernel threads, which are not encumbered with the unnecessary User Mode con- text.

Whose functions are being delegated? Is it the operating system's functions, or the system processes' functions? It does not make sense to say that because a system process runs only in kernel mode, its function is delegated to a kernel thread. It makes some sense to think of an operating system "delegating" specific operating system functions to a kernel thread. However, the "because" would indeed not make sense if we use Maurers definition of system process. Therefore I discount any strict meaning of this sentence.

(And the lack of "encumberance" is pretty insignificant, and a detail that might happen to be contradicted by a specific implementation).

A reasonable loose re-interpretation is that the kernel does not support performing certain "critical tasks" outside the kernel, and that some of these tasks are handled by kernel threads.

Q. Can "system process" mentioned in the two books run in user space, kernel space, or both?

At a specific point in time, a system process can be running in user space or in kernel space.

When a user process makes a system call, the process transitions to running in kernel space. When the system call returns, the process transitions back to running in user space.

Q. Is "system process" different from kernel control path and kernel thread?

As per the above definition, a kernel thread is a system process, but a system process might not be a kernel thread.

Bovet says the spin_lock macro, when it succeeds, will cause the current kernel path to "acquire" the spin lock (exclusive of other kernel paths). spin_lock can be called from kernel threads, therefore a kernel thread would count as a kernel control path. As far as I can tell. And as far as I can tell there is not a contradiction of this. But since I cannot find this explicitly defined, of what is and is not a kernel control path, I would not rely on this implicit definition to apply everywhere he uses the phrase.

The kernel control paths for a system call, an exception, or an interrupt are not kernel threads.

Except that some drivers now use threads to handle practically all of the response to an interrupt. (The only part outside the thread is to dis-ambiguate interrupt lines which are shared between multiple devices).

Moving interrupts to threads - LWN.net, 2008

https://www.kernel.org/doc/html/v4.20/core-api/genericirq.html (search for "thread")