OpenBSD introduced stack-register checking

"It is opportunistically enforced by the kernel." -> this means it is optional?

"When a system call happens, we check if the stack-pointer register points to such a page. If it doesn't, the program is killed. " - can someone please explain the stack-register to a non-programmer in a little more detail? if it is optional, how come it would kill all program that that doesn't have it?

2 Answers 2


The stack pointer register is a hardware register. It points to a memory location* that is in an area used for a stack. The stack pointer register is used when addressing data on the stack, typically incrementing or decrementing the value of the pointer before or after the memory access. The check, which is performed when a system call happens, checks that the register points to a valid stack address. "Opportunistically" does not mean "optional".

[*]: a valid address can also be one past the last address of the stack page on some arhitectures.


Every thread has stack. Stack is speical region of memory that is used by thread to store local variables, functon return address etc. When CPU executes thread, it must has its register (SP) point to memory address with stack.

In most cases kernel allocates stack of thread, but sometimes stacks are created by application: Apps may create custom stack for signal handlers (code that is executed when kernel sends signal to process) or they may do it to implement threading library.

To do that, application asks kernel for region of memory and then marks it as stack, so kernel knows this region is stack. But due to developer mistakes, application may ask kernel to use some memory for stack which was not registered for that. In this case something bad may happen: kernel may overwrite useful data because it thinks it wrtites to stack, but it is not stack! Hackers may use it to break programs.

In OpenBSD when you reserve memory (using mmap for example) you must EXPLICITLY say that:

Sunny day scenario:

  • App: kernel, may I have some memory which I will use as stack?
  • Kernel: sure, here is your memory
  • App: ok, please use it as stack for signals
  • Kernel: ok (configures internal structures to make CPU SP register to point to this memory when signal will be processed)

Rainy day scenario:

  • App: kernel, may I have some memory?
  • Kernel: sure, here is your memory
  • App: ok, please use it as stack for signals
  • Kernel: but this memory is not for stack! You are trying to fool me!
  • (Kernel kills app)

Here are some more examples of such flags:

PROT_EXEC: asks kernel to reserve memory to store code (something CPU may execute) PROT_WRITE: asks kernel to reserve memory to writable data

So, you can't execute your data or write to your code (this is called W^X and OpenBSD people proud of it)

You may read man mmap and map sigaltstack for more examples

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.