1

I am on an embedded Linux setup. I mount a squashfs with a writable overlay (using overlayfs) and then chroot into it, transferring control to systemd.

I have patches in systemd to send the coredumps to /var/log, instead of /var/lib/systemd/coredump.

When a coredump is requested, systemd-coredump is invoked, but gets this error.

Jul 30 08:54:14 evo4k-e6872f kernel: ClearApp[803]: segfault at 0 ip 000000000042bcb0 sp 00007ffcd4915f18 error 6 in ClearApp[400000+868000]
Jul 30 08:54:14 evo4k-e6872f kernel[359]: ClearApp[803]: segfault at 0 ip 000000000042bcb0 sp 00007ffcd4915f18 error 6 in ClearApp[400000+868000]
Jul 30 08:54:14 evo4k-e6872f systemd[1]: Started Process Core Dump (PID 804/UID 0).
Jul 30 08:54:14 evo4k-e6872f systemd-coredump[805]: Failed to create temporary file for coredump /var/log/coredump/core.ClearApp.0.54a13c5624ad4ed6b3>
Jul 30 08:54:14 evo4k-e6872f systemd-coredump[805]: Process 803 (ClearApp) of user 0 dumped core.
Jul 30 08:54:14 evo4k-e6872f systemd[1]: clearapp.service: Main process exited, code=dumped, status=11/SEGV
Jul 30 08:54:14 evo4k-e6872f systemd[1]: clearapp.service: Failed with result 'core-dump'.
Jul 30 08:54:15 evo4k-e6872f systemd[1]: clearapp.service: Service hold-off time over, scheduling restart.
Jul 30 08:54:15 evo4k-e6872f systemd[1]: clearapp.service: Scheduled restart job, restart counter is at 5.
Jul 30 08:54:15 evo4k-e6872f systemd[1]: Stopped MedX ClearApp.
Jul 30 08:54:15 evo4k-e6872f systemd[1]: clearapp.service: Start request repeated too quickly.
Jul 30 08:54:15 evo4k-e6872f systemd[1]: clearapp.service: Failed with result 'core-dump'.
Jul 30 08:54:15 evo4k-e6872f systemd[1]: Failed to start MedX ClearApp.

My /proc/sys/kernel/core_pattern is |/lib/systemd/systemd-coredump %P %u %g %s %t %c %e. If I set the core_pattern to /tmp/cores/core.%e.%p.%h.%t, it works. So, it is def something with systemd.

I am using systemd version 237. This previously worked with systemd version 234.

My file system is correct/clean. It is not corrupted in anyway (fsck comes back clean).

How do I get systemd to generate core dumps without errors?

EDIT #1

I recompiled systemd-coredump to log to different directories, even a mounted thumb drive, and it gives me the same error.

Jul 30 10:43:39 evo4k-e6872f systemd-coredump[1910]: Failed to create temporary file for coredump /run/media/Pauls/core.ClearApp.0.dd6557bb31264bf2b3773b534fd6e2b1.1908.1532961819000000: Read-only file system

I'm starting to think that the kernel or systemd is doing something other than simply creating a temp file.

EDIT #2

I ran the exact open call that systemd does from userspace, and it succeeds. Is there something about the context in which the kernel invokes the core dumper?

I added this line to systemd.

    fd = open(tmp, O_CREAT|O_EXCL|O_NOFOLLOW|O_NOCTTY|flags, 0640);
    if (fd < 0) {
            log_error("Couldn't open: %d: %s", fd, tmp);
            return -errno;
    }

I get this output.

Couldn't open: -1: /run/media/Pauls/.#core.ClearApp.0.7833dca6d3354c0e959b366df731bf9f.879.15329633730000000f174d1155a09d96
  • It has a writable overlay (tmpfs). It is completely writable. Also, if I manually mount --bind another partition to /var/log before chroot, the error still happens. – Paul Knopf Jul 30 '18 at 13:06
  • Also, this worked with systemd 234, with and without the mount --bind to /var/log before chroot. – Paul Knopf Jul 30 '18 at 13:08
3

This has been answered in GitHub (#9756).

For posterity: "the coredump service is run with ProtectSystem=strict and ProtectHome=yes set, which means it can't write anywhere except the path listed in StateDirectory= i.e. /var/lib/systemd/coredump/."

The suggested approach for saving coredumps elsewhere (such as a subdirectory of /var/log/) is to bind mount that directory into /var/lib/systemd/coredump/.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.