11

The flag GCC -fstack-protector flag enables the usage of stack canaries for stack overflow protection. The usage of this flag by default has been more prominent in recent years.

If a package is compiled with -fstack-protector, and we overflow a buffer in the program, we are likely to get an error such as:

*** buffer overflow detected ***: /xxx/xxx terminated

However, "who" is in charge of these error messages? Where do these messages get logged? Does the syslog daemon pick these messages?

10

Stack smashing is detected by libssp, which is part of gcc. It tries very hard to output the message to a terminal, and only if that fails does it log to the system log — so in practice you’ll see buffer overflow messages in the logs for daemons and perhaps GUI applications.

Once it’s output its message, libssp tries a variety of ways to exit, including crashing the application; this might be caught by one of the abnormal exit loggers, but that’s not guaranteed.

  • 1
    Let me pose a concrete example as a way to further explore this explanation. Let's pick nginx for this example. I've compiled nginx with stack canaries. When I run nginx, it starts a process but does not output anything to the shell. Instead, any messages are logged in its multiple log files. If nginx detects stack smashing, libssp will output its message by the stderr output used by nginx. Then, libssp might try to exit the process (or child process for nginx). If it "doesn't need to" crash the application, then abnormal exit loggers will not pick this up.Is this a correct interpretation? – aedcv Oct 16 '17 at 19:30
  • Not quite — it will try to crash the application, using __builtin_trap() first, then if that fails, trying to provoke a segment violation, and only if that fails, exiting with status 127. – Stephen Kitt Oct 16 '17 at 21:17
  • The printing of messages part hasn't better guarantees of success than the exit via a core yielding method (e.g. abort()). – maxschlepzig Oct 17 '17 at 8:51
7

Modern Linux distributions like CentOS/Fedora set up a crash handling daemon (e.g. systemd-coredump or abortd), by default.

Thus, when your program terminates in an abnormal fashion (segfault, uncaught exception, abort, illegal instruction etc.) this event is registered and logged by that daemon. Thus, you find some messages in the system journal and possibly a reference to a directory with some additional details (e.g. core file, logs, etc.).

Example

$ cat test_stack_protector.c 
#include <string.h>

int f(const char *q)
{
  char s[10];
  strcpy(s, q);
  return s[0] + s[1];
}

int main(int argc, char **argv)
{
  return f(argv[1]);
}

Compile:

$ gcc -Wall -fstack-protector test_stack_protector.c -o test_stack_protector

Execute:

$ ./test_stack_protector 'hello world'
*** stack smashing detected ***: ./test_stack_protector terminated
======= Backtrace: =========
/lib64/libc.so.6(+0x7c8dc)[0x7f885b4388dc]
/lib64/libc.so.6(__fortify_fail+0x37)[0x7f885b4dfaa7]
/lib64/libc.so.6(__fortify_fail+0x0)[0x7f885b4dfa70]
./test_stack_protector[0x400599]
./test_stack_protector[0x4005bd]
/lib64/libc.so.6(__libc_start_main+0xea)[0x7f885b3dc50a]
./test_stack_protector[0x40049a]
======= Memory map: ========
00400000-00401000 r-xp 00000000 00:28 1151979                            /home/juser/program/stackprotect/test_stack_protector
00600000-00601000 r--p 00000000 00:28 1151979                            /home/juser/program/stackprotect/test_stack_protector
00601000-00602000 rw-p 00001000 00:28 1151979                            /home/juser/program/stackprotect/test_stack_protector
0067c000-0069d000 rw-p 00000000 00:00 0                                  [heap]
7f885b1a5000-7f885b1bb000 r-xp 00000000 00:28 1052100                    /usr/lib64/libgcc_s-7-20170915.so.1
7f885b1bb000-7f885b3ba000 ---p 00016000 00:28 1052100                    /usr/lib64/libgcc_s-7-20170915.so.1
7f885b3ba000-7f885b3bb000 r--p 00015000 00:28 1052100                    /usr/lib64/libgcc_s-7-20170915.so.1
7f885b3bb000-7f885b3bc000 rw-p 00016000 00:28 1052100                    /usr/lib64/libgcc_s-7-20170915.so.1
7f885b3bc000-7f885b583000 r-xp 00000000 00:28 945348                     /usr/lib64/libc-2.25.so
7f885b583000-7f885b783000 ---p 001c7000 00:28 945348                     /usr/lib64/libc-2.25.so
7f885b783000-7f885b787000 r--p 001c7000 00:28 945348                     /usr/lib64/libc-2.25.so
7f885b787000-7f885b789000 rw-p 001cb000 00:28 945348                     /usr/lib64/libc-2.25.so
7f885b789000-7f885b78d000 rw-p 00000000 00:00 0 
7f885b78d000-7f885b7b4000 r-xp 00000000 00:28 945341                     /usr/lib64/ld-2.25.so
7f885b978000-7f885b97b000 rw-p 00000000 00:00 0 
7f885b9b0000-7f885b9b3000 rw-p 00000000 00:00 0 
7f885b9b3000-7f885b9b4000 r--p 00026000 00:28 945341                     /usr/lib64/ld-2.25.so
7f885b9b4000-7f885b9b6000 rw-p 00027000 00:28 945341                     /usr/lib64/ld-2.25.so
7ffc59966000-7ffc59987000 rw-p 00000000 00:00 0                          [stack]
7ffc5999c000-7ffc5999f000 r--p 00000000 00:00 0                          [vvar]
7ffc5999f000-7ffc599a1000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
zsh: abort (core dumped)  ./test_stack_protector 'hello world'

The exit status is 134 which is 128+6, i.e. 128 plus the abort signal number.

The system journal:

Oct 16 20:57:59 example.org audit[17645]: ANOM_ABEND auid=1000 uid=1000 gid=1000 ses=3 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 pid=17645 comm="test_stack_prot" exe="/home/juser/program/stackprotect/test_stack_protector" sig=6 res=1
Oct 16 20:57:59 example.org systemd[1]: Started Process Core Dump (PID 17646/UID 0).
Oct 16 20:57:59 example.org audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-coredump@21-17646-0 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Oct 16 20:57:59 example.org systemd-coredump[17647]: Process 17645 (test_stack_prot) of user 1000 dumped core.

                           Stack trace of thread 17645:
                           #0  0x00007f885b3f269b raise (libc.so.6)
                           #1  0x00007f885b3f44a0 abort (libc.so.6)
                           #2  0x00007f885b4388e1 __libc_message (libc.so.6)
                           #3  0x00007f885b4dfaa7 __fortify_fail (libc.so.6)
                           #4  0x00007f885b4dfa70 __stack_chk_fail (libc.so.6)
                           #5  0x0000000000400599 f (test_stack_protector)
                           #6  0x00000000004005bd main (test_stack_protector)
                           #7  0x00007f885b3dc50a __libc_start_main (libc.so.6)
                           #8  0x000000000040049a _start (test_stack_protector)
Oct 16 20:57:59 example.org audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-coredump@21-17646-0 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Oct 16 20:58:00 example.org abrt-notification[17696]: Process 17645 (test_stack_protector) crashed in __fortify_fail()

That means you get logging from the auditd auditing daemon and the systemd-coredump crash handler.

To verify whether a crash handling daemon is configured you can check /proc, e.g.:

$ cat /proc/sys/kernel/core_pattern
|/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %e

(everything tested on Fedora 26, x86-64)

  • 1
    I'm very glad you posted this example. The canaries are put in place by gcc. (Please correct me if I'm wrong) I assume that what happens is something like: gcc puts "extra code" in the program to implement the canary functionality; during execution, and before a function returns, the value is checked; if pollutted, the program will output the message "stack smashing detected" and raises an error. This error is picked up by the OS, recognizes a segmentation fault and prints the backtrace and memory map you posted. Lastly, OS kills the application, generates a core dump, and logs to sys journal – aedcv Oct 16 '17 at 19:20
  • @aedcv, this is pretty much the story - to be more precise: the stack smashing checking code calls abort() which yields an abort signal, i.e. there is no segmentation fault going on. It's just that the default signal handlers for abort/segmentation fault etc. yield the same action: write core and exit the process with an exit status unequal zero that also encodes the signal number. The core writing is done by the kernel and its behavior is configurable via /proc/.../core_pattern. In the above example a user-space helper is configured and thus called. The kernel also triggers the auditing. – maxschlepzig Oct 16 '17 at 19:43
  • @maxschlepzig it’s not quite abort(), the SSP code uses __builtin_trap() (but the effect is the same). – Stephen Kitt Oct 16 '17 at 21:14
  • 1
    @StephenKitt, well, take a look at the stack trace in the above example. There you clearly see how abort() is called. – maxschlepzig Oct 17 '17 at 8:48
  • 1
    @maxschlepzig yes, of course, but that’s an implementation detail (GCC code uses __builtin_trap() to avoid having an explicit dependency on abort()). Other distributions have different stack traces. – Stephen Kitt Oct 17 '17 at 9:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.