Is system call the only way that a user program can intentionally affect the state of a Linux kernel?

Question

I think there are several ways that a user program can intentionally affect the state of a Linux kernel.

1. By invoking a system call;
2. By invoking mmap() and writing the memory that has been mapped to the kernel;
3. By loading a kernel module via insmod.

I cannot come up with any other way. Hardware interrupts are not considered because they are not initiated by a user program. I think both mmap() and insmod are using system calls, so maybe a user program has to rely on system calls to affect the kernel's state. Am I correct?

If I'm correct, let's say there are some vulnerabilities in the kernel, and a malicious user program wants to exploit them to attack the kernel. Is it possible to verify every system call to defend such attacks, given that our verification can always tell the truth?

Answer 1

As it so happens, there is another significant interface with the kernel: the /proc and /sys virtual filesystems. While they do not hold regular files, their contents are direct gateways to the kernel: to act on them is to act directly on kernel-allocated memory. For instance, if you want to drop all memory caches, you may use...

echo 3 > /proc/sys/vm/drop_caches

... and the kernel will react immediately.

Now, a program will need system calls in order to interact with the filesystem : open, read, write, and so on... However, there is still a way to track all these system calls : the kernel provides a tracing mechanism under /sys/kernel/debug/tracing. More specifically, tracing of system calls is handled by /sys/kernel/debug/tracing/events/syscalls. This virtual directory contains two subdirectories for each system call. For instance, with the open system call, we have:

• sys_enter_open
• sys_exit_open

In these directories, you'll find a file called enable. If it contains "1", then the associated event (entering or exiting an open call) is being traced. I usually use the enter event, but you may choose whatever fits your needs more.

Once you've activated the system call trace, you'll find the log at /sys/kernel/debug/tracing/trace. Now, keep in mind that the open system call is used a lot. It is the final gateway between a program and a file, and files can be literraly anything on a Linux system. Also keep in mind that...

UNIX was not designed to stop its users from doing stupid things, as that would also stop them from doing clever things. — Doug Gwyn

While you may monitor what happens on your system, the kernel won't try hard preventing users from doing stupid things: that's more part of the sysadmin's work.

Managing the tracing mechanism requires permissions under /sys/kernel/debug/tracing/trace. You'll probably need to be root in order to activate and manipulate the trace. Don't forget to disable the trace when you're done.

Answer 2

It depends. What do you mean by 'verify'? If you just want to monitor what syscalls some process is triggering then it's possible.. usually.... But if you feel like digging deeper then you're in trouble... I have not heard of any tools that could do that.

You can use strace to see what syscalls some particular process is firing. Of course you'll have to run it as root... However you won't always succeed running strace as it's possible to protect application from monitoring it's activity -- ptrace calls are dropped. Try stracing on chromium - you'll see ;)

If strace is not enough than I suppose you're lift with disassembling each application binary and manually checking what it does. This should be fun :) (or, of course, you can get a source code and see all those algorithms as human-readable text.. but why pick an easier way when you can go with ASM :) )

Answer 3

This is a good first approximation. There is a strong boundary between userspace and the kernel, and most interactions must involve system calls. This model helps understand why strace is so powerful as a troubleshooting tool.

A large number of kernel vulnerabilities could be blamed on the absence of such verification. So long as you understand that such verification is extremely costly. This is particularly true for languages which can be used for direct hardware access etc. It is doubly true when using such a crude language as C, because the language design provides much less help for such verification than one would like.

A well-known mitigation is to use a microkernel. Less code allows more verification. With a microkernel architecture, the mass of device driver code can be significantly more contained. Perhaps entirely so, if you have an adequate IOMMU as well as MMU.

However, the statement is not correct.

If you search using the right terms, I think you can find a number of vulnerabilities which serve as counterexamples. Faults can be generated on memory ranges which have not even been mapped:

‘Linux Kernel i386 SMP Page Fault Handler Privilege Escalation’ [SecuriTeam].

In general, you need to verify all of the architecture-specific fault code. This includes the code which receives system calls and page faults. However there are other types of faults as well, e.g. floating point exceptions (divide by zero).

Also, vulnerabilities can arise from hardware details. The specific hardware details might not be known at the time you try to verify the code.

https://en.wikipedia.org/wiki/Row_hammer