How should I manage the logs from my own systemd services?

Question

I have deployed a couple of daemons on ubuntu boxes and wrapped them as systemd services. Their standard output is redirected to the journal by default which is what I was hoping for.

Now, it turns out that the default configuration on ubuntu (I guess same on debian) is to make journald forward all events to /run/systemd/journal/syslog (see /etc/systemd/journald.conf: #ForwardToSyslog=yes) and have rsyslog pull data from there with the imuxsock module (see /etc/rsyslog.conf). By default, I also see that because in /etc/rsyslog.d/50-default.conf, we have this line:

.;auth,authpriv.none -/var/log/syslog

i.e., all non-auth data ends up in /var/log/syslog

Because I wanted to track the output of my services in journald, I made the journald log file persistent by creating /var/log/journal (in /etc/systemd/journald.conf, Storage's default value is auto).

The result is that now all my log data is duplicated: it is stored once in journald and once in rsyslog's /var/log/syslog.

So, for various reasons, I really want to keep my own service's data in journald but I really do not want to duplicate that data (there are a lot of logs !).

I can see a couple of options:

1. disable rsyslog entirely. I am worried I might miss a lot of data from other services if I do this: who knows what other code in my infrastructure reads /var/log/*
2. try to disable only daemon.info because this is what appears to be the default log facility/level picked for my services. I am worried the following might make me ignore other useful messages that just happen to have the same facility/level :/

.;auth,authpriv.none;daemon.!=info -/var/log/syslog

3. Change the rsyslog/journald integration to use the imjournal input module, ignore imuxsock (just like on fedora), and write rsyslog rules more specific to each of my services

Now, the question is: what would be the recommended way to proceed ?

Answer 1

Old question I know but you are almost there. To stop any log entries from any services going to syslog add daemon.none like this;

*.*;auth,authpriv.none,daemon.none              -/var/log/syslog

Log output from a daemon defaults to SyslogFacility=daemon which is why you get a daemon.log file.

I 'think' that if you just want it for one of your services, you can use the SyslogFacility= identifier with local0 (0-9 available) within that service file and then just include the relevant directive as above (e.g. local0.none) to just filter those log entries out of syslog.

https://www.freedesktop.org/software/systemd/man/systemd.exec.html