2

I am on an embedded Linux setup. I mount a squashfs with a writable overlay (using overlayfs) and then chroot into it, transferring control to systemd.

I have patches in systemd to send the coredumps to /var/log, instead of /var/lib/systemd/coredump.

When a coredump is requested, systemd-coredump is invoked, but gets this error.

Jul 30 08:54:14 evo4k-e6872f kernel: ClearApp[803]: segfault at 0 ip 000000000042bcb0 sp 00007ffcd4915f18 error 6 in ClearApp[400000+868000]
Jul 30 08:54:14 evo4k-e6872f kernel[359]: ClearApp[803]: segfault at 0 ip 000000000042bcb0 sp 00007ffcd4915f18 error 6 in ClearApp[400000+868000]
Jul 30 08:54:14 evo4k-e6872f systemd[1]: Started Process Core Dump (PID 804/UID 0).
Jul 30 08:54:14 evo4k-e6872f systemd-coredump[805]: Failed to create temporary file for coredump /var/log/coredump/core.ClearApp.0.54a13c5624ad4ed6b3>
Jul 30 08:54:14 evo4k-e6872f systemd-coredump[805]: Process 803 (ClearApp) of user 0 dumped core.
Jul 30 08:54:14 evo4k-e6872f systemd[1]: clearapp.service: Main process exited, code=dumped, status=11/SEGV
Jul 30 08:54:14 evo4k-e6872f systemd[1]: clearapp.service: Failed with result 'core-dump'.
Jul 30 08:54:15 evo4k-e6872f systemd[1]: clearapp.service: Service hold-off time over, scheduling restart.
Jul 30 08:54:15 evo4k-e6872f systemd[1]: clearapp.service: Scheduled restart job, restart counter is at 5.
Jul 30 08:54:15 evo4k-e6872f systemd[1]: Stopped MedX ClearApp.
Jul 30 08:54:15 evo4k-e6872f systemd[1]: clearapp.service: Start request repeated too quickly.
Jul 30 08:54:15 evo4k-e6872f systemd[1]: clearapp.service: Failed with result 'core-dump'.
Jul 30 08:54:15 evo4k-e6872f systemd[1]: Failed to start MedX ClearApp.

My /proc/sys/kernel/core_pattern is |/lib/systemd/systemd-coredump %P %u %g %s %t %c %e. If I set the core_pattern to /tmp/cores/core.%e.%p.%h.%t, it works. So, it is def something with systemd.

I am using systemd version 237. This previously worked with systemd version 234.

My file system is correct/clean. It is not corrupted in anyway (fsck comes back clean).

How do I get systemd to generate core dumps without errors?

EDIT #1

I recompiled systemd-coredump to log to different directories, even a mounted thumb drive, and it gives me the same error.

Jul 30 10:43:39 evo4k-e6872f systemd-coredump[1910]: Failed to create temporary file for coredump /run/media/Pauls/core.ClearApp.0.dd6557bb31264bf2b3773b534fd6e2b1.1908.1532961819000000: Read-only file system

I'm starting to think that the kernel or systemd is doing something other than simply creating a temp file.

EDIT #2

I ran the exact open call that systemd does from userspace, and it succeeds. Is there something about the context in which the kernel invokes the core dumper?

I added this line to systemd.

    fd = open(tmp, O_CREAT|O_EXCL|O_NOFOLLOW|O_NOCTTY|flags, 0640);
    if (fd < 0) {
            log_error("Couldn't open: %d: %s", fd, tmp);
            return -errno;
    }

I get this output.

Couldn't open: -1: /run/media/Pauls/.#core.ClearApp.0.7833dca6d3354c0e959b366df731bf9f.879.15329633730000000f174d1155a09d96
2
  • It has a writable overlay (tmpfs). It is completely writable. Also, if I manually mount --bind another partition to /var/log before chroot, the error still happens.
    – Paul Knopf
    Commented Jul 30, 2018 at 13:06
  • Also, this worked with systemd 234, with and without the mount --bind to /var/log before chroot.
    – Paul Knopf
    Commented Jul 30, 2018 at 13:08

1 Answer 1

3

This has been answered in GitHub (#9756).

For posterity: "the coredump service is run with ProtectSystem=strict and ProtectHome=yes set, which means it can't write anywhere except the path listed in StateDirectory= i.e. /var/lib/systemd/coredump/."

The suggested approach for saving coredumps elsewhere (such as a subdirectory of /var/log/) is to bind mount that directory into /var/lib/systemd/coredump/.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .