If a user-mode process wants to change to kernel mode, does it always succeed?

Question

Computer Systems: a Programmer's Perspective says

In order for the operating system kernel to provide an airtight process abstraction, the processor must provide a mechanism that restricts the instructions that an application can execute, as well as the portions of the address space that it can access.

Processors typically provide this capability with a mode bit in some control register that characterizes the privileges that the process currently enjoys. When the mode bit is set, the process is running in kernel mode (sometimes called supervisor mode). A process running in kernel mode can execute any instruction in the instruction set and access any memory location in the system.

When the mode bit is not set, the process is running in user mode. A process in user mode is not allowed to execute privileged instructions that do things such as halt the processor, change the mode bit, or initiate an I/O operation. Nor is it allowed to directly reference code or data in the kernel area of the address space. Any such attempt results in a fatal protection fault. User programs must instead access kernel code and data indirectly via the system call interface.

A process running application code is initially in user mode. The only way for the process to change from user mode to kernel mode is via an exception such as an interrupt, a fault, or a trapping system call. When the exception occurs, and control passes to the exception handler, the processor changes the mode from user mode to kernel mode. The handler runs in kernel mode. When it returns to the application code, the processor changes the mode from kernel mode back to user mode.

If a user-mode process wants to change to kernel mode, does it always succeed?

Does whether the change of mode succeed depend on what the process wants to do after change to kernel mode?

For examples,

• A process may or may not have the right to access a file, depending on the uid of the process and the access control bits of the file. When the process issues a system call to access the file, can it always succeed in changing from user mode to kernel mode? Does whether the process can change from user to kernel mode depends on whether it can access the file?

• Does sudo have nothing to do with whether a process can change from user to kernel mode?

Thanks.

Answer 1

Does whether the process can change from user to kernel mode depends on whether it can access the file?

No, the check to see if a program can access a file happens in the kernel. A program can call the kernel whenever it wants using an interrupt (in x86) or the syscall instruction (in amd64).

Does sudo have nothing to do with whether a process can change from user to kernel mode?

sudo has nothing to do with usermode/kernel mode. The superuser account still lives solely in usermode.

Answer 2

If a user-mode process wants to change to kernel mode, does it always succeed?

For all practical purposes, yes. Note that processes don’t necessarily “want” to change to kernel mode — the processor switches to kernel mode whenever a process attempts a privileged operation, or calls a privileged “gate”.

The details of what constitutes a privileged operation are architecture-specific, but broadly speaking, they include:

• privileged instructions, which affect the global system state in ways which shouldn’t be under non-kernel control (examples on x86 include the CLI instruction which disables non-maskable interrupts, and the HLT instruction which halts the CPU);
• certain types of input/output instructions, which depend on the privilege level (on x86, the IN/OUT family of instructions);
• attempting to access memory which the current process isn’t allowed to access;
• attempting to execute code in areas of memory which the current process isn’t allowed to execute.

Other instructions or circumstances will also end up trapping or faulting into the kernel, without necessarily being privileged per se — for example, attempting to divide by zero.

If the information required to switch to kernel mode isn’t correctly set up, the processor will fault, and if that fault isn’t handled correctly, it will usually reboot. (See double faults and triple faults.)

If you’re keen to understand more about this, see the relevant processor manuals (e.g. the Intel manuals).

On x86-64, the SYSCALL instruction which is used to call system calls sits somewhere in between “regular” instructions and privileged instructions; it is always callable, and always invokes code in privileged mode.

Does whether the change of mode succeed depend on what the process wants to do after change to kernel mode?

No.

When the process issues a system call to access the file, can it always succeed in changing from user mode to kernel mode? Does whether the process can change from user to kernel mode depends on whether it can access the file?

No, the switch to kernel mode happens before any of the kernel code runs, including deciding whether the caller is allowed to do what it asked to do.

Does sudo have nothing to do with whether a process can change from user to kernel mode?

No. Privileges here only concern the processor’s execution levels and memory protection; they are completely separate from user privileges such as the user/root distinction.

Answer 3

There are well defined entry points in to kernel space.

There exists a table (only writable in kernel mode). It contains the start addresses of many routines. These include the routine to run if there is a segmentation fault, a divide by zero, a hardware interrupt, a sys-call, etc.

So a process can trigger a transition by executing the sys-call (there may be 1 or more of these) instruction. However the user-mode code can not specify what code is run. In a sys-call, the kernel mode will look at the content of registers, to determine what the process wants, it will look at the processes user, group, capabilities, etc to decide if it can do it.