7

Please assume there's an app/script which prints a great deal of logging data to stdout. When something bad but expected happens within the script (exception handling), scripts reports an error to stderr and then continues doing what it was doing.

systemd for sure is capable to collect this data and put it into the journal:

# systemctl show sd_test.service | grep 'Standard[OE]'
StandardOutput=journal
StandardError=inherit

My sd_test.service:

[Unit]
Description=A Test Service simply printing to stdout and stderr

[Service]
Type=simple
ExecStart=/home/narunas/sd_test.py

[Install]
WantedBy=multi-user.target

journalctl now has required data:

# systemctl reenable sd_test.service
# systemctl restart sd_test.service
# journalctl -n -u sd_test
-- Logs begin at Fri 2016-11-11 15:49:33 GMT, end at Tue 2017-01-03 19:23:18 GMT. --
Jan 03 19:23:01 dev-box sd_test.py[13183]: This is "stderr": 2
Jan 03 19:23:02 dev-box sd_test.py[13183]: This is "stdout": 2
Jan 03 19:23:03 dev-box sd_test.py[13183]: This is "stderr": 3
Jan 03 19:23:07 dev-box sd_test.py[13183]: This is "stdout": 3
Jan 03 19:23:07 dev-box sd_test.py[13183]: This is "stderr": 4
Jan 03 19:23:11 dev-box sd_test.py[13183]: This is "stdout": 4
Jan 03 19:23:12 dev-box sd_test.py[13183]: This is "stderr": 5
Jan 03 19:23:15 dev-box sd_test.py[13183]: This is "stdout": 5
Jan 03 19:23:17 dev-box sd_test.py[13183]: This is "stdout": 6
Jan 03 19:23:18 dev-box sd_test.py[13183]: This is "stderr": 6

I know there's _TRANSPORT journal field, sadly stderr is tagged with the same tag:

stdout
for those read from a service's standard output or error output

I can of course change my ExecStart like so:

ExecStart=/bin/bash -c '/home/narunas/sd_test.py 2> /some/file/path'

But that is not ideal as I'm mixing up journal with files...

Could you recommend a better way to view stderr log only?

EDIT_1

As suggested in the comments I just tried -o verbose option:

Tue 2017-01-03 20:23:49.994171 GMT [s=0b15e5c69e2f476eb200d2bdda769465;i=12e28;b=db55b41f61144ae69cc86acfb75209fb;m=42e034d52dc;t=5453672322abb;x=8b696c5447bc2bce]
    PRIORITY=6
    _UID=0
    _GID=0
    _CAP_EFFECTIVE=3fffffffff
    _SYSTEMD_SLICE=system.slice
    _BOOT_ID=db55b41f61144ae69cc86acfb75209fb
    _MACHINE_ID=c5a9e78e2c854065a9b041c58f07c2b2
    _HOSTNAME=dev-box
    SYSLOG_FACILITY=3
    _TRANSPORT=stdout
    SYSLOG_IDENTIFIER=sd_test.py
    _COMM=python3
    _EXE=/usr/bin/python3.5
    _CMDLINE=python3 /home/narunas/sd_test.py
    _SYSTEMD_CGROUP=/system.slice/sd_test.service
    _SYSTEMD_UNIT=sd_test.service
    MESSAGE=This is "stdout": 729
    _PID=13183
Tue 2017-01-03 20:23:52.979981 GMT [s=0b15e5c69e2f476eb200d2bdda769465;i=12e29;b=db55b41f61144ae69cc86acfb75209fb;m=42e037ae22e;t=54536725fba0d;x=8802c0df56848907]
    PRIORITY=6
    _UID=0
    _GID=0
    _CAP_EFFECTIVE=3fffffffff
    _SYSTEMD_SLICE=system.slice
    _BOOT_ID=db55b41f61144ae69cc86acfb75209fb
    _MACHINE_ID=c5a9e78e2c854065a9b041c58f07c2b2
    _HOSTNAME=dev-box
    SYSLOG_FACILITY=3
    _TRANSPORT=stdout
    SYSLOG_IDENTIFIER=sd_test.py
    _COMM=python3
    _EXE=/usr/bin/python3.5
    _CMDLINE=python3 /home/narunas/sd_test.py
    _SYSTEMD_CGROUP=/system.slice/sd_test.service
    _SYSTEMD_UNIT=sd_test.service
    MESSAGE=This is "stderr": 745
    _PID=13183

-o json-pretty produce same results as -o verbose, but in a different format.

Apart my script's silly MESSAGE there's nothing useful that would let to distinguish what is stderr or stdout.

EDIT_2

With StandardError=journal.

Unit:

[Unit]
Description=A Test Service simply printing to stdout and stderr

[Service]
Type=simple
ExecStart=/home/narunas/sd_test.py
StandardError=journal

[Install]
WantedBy=multi-user.target

journalctl:

# systemctl reenable sd_test.service
# systemctl restart sd_test.service
# journalctl -n -u sd_test -o verbose
Tue 2017-01-03 22:34:55.381341 GMT [s=0b15e5c69e2f476eb200d2bdda769465;i=13ab7;b=db55b41f61144ae69cc86acfb75209fb;m=42fd81d9b7e;t=545384702735d;x=e65570c85dfec1dc]
    _TRANSPORT=stdout
    PRIORITY=6
    SYSLOG_FACILITY=3
    SYSLOG_IDENTIFIER=sd_test.py
    _UID=0
    _GID=0
    _COMM=python3
    _EXE=/usr/bin/python3.5
    _CMDLINE=python3 /home/narunas/sd_test.py
    _CAP_EFFECTIVE=3fffffffff
    _SYSTEMD_CGROUP=/system.slice/sd_test.service
    _SYSTEMD_UNIT=sd_test.service
    _SYSTEMD_SLICE=system.slice
    _BOOT_ID=db55b41f61144ae69cc86acfb75209fb
    _MACHINE_ID=c5a9e78e2c854065a9b041c58f07c2b2
    _HOSTNAME=dev-box
    MESSAGE=This is "stdout": 10
    _PID=21280
Tue 2017-01-03 22:34:57.397880 GMT [s=0b15e5c69e2f476eb200d2bdda769465;i=13ab8;b=db55b41f61144ae69cc86acfb75209fb;m=42fd83c6099;t=5453847213878;x=610237e701d596d4]
    _TRANSPORT=stdout
    PRIORITY=6
    SYSLOG_FACILITY=3
    SYSLOG_IDENTIFIER=sd_test.py
    _UID=0
    _GID=0
    _COMM=python3
    _EXE=/usr/bin/python3.5
    _CMDLINE=python3 /home/narunas/sd_test.py
    _CAP_EFFECTIVE=3fffffffff
    _SYSTEMD_CGROUP=/system.slice/sd_test.service
    _SYSTEMD_UNIT=sd_test.service
    _SYSTEMD_SLICE=system.slice
    _BOOT_ID=db55b41f61144ae69cc86acfb75209fb
    _MACHINE_ID=c5a9e78e2c854065a9b041c58f07c2b2
    _HOSTNAME=dev-box
    MESSAGE=This is "stderr": 11
    _PID=21280
Tue 2017-01-03 22:35:02.403014 GMT [s=0b15e5c69e2f476eb200d2bdda769465;i=13abc;b=db55b41f61144ae69cc86acfb75209fb;m=42fd888bfe7;t=54538476d97c6;x=a6362820cb52ce9]
    _TRANSPORT=stdout
    PRIORITY=6
    SYSLOG_FACILITY=3
    SYSLOG_IDENTIFIER=sd_test.py
    _UID=0
    _GID=0
    _COMM=python3
    _EXE=/usr/bin/python3.5
    _CMDLINE=python3 /home/narunas/sd_test.py
    _CAP_EFFECTIVE=3fffffffff
    _SYSTEMD_CGROUP=/system.slice/sd_test.service
    _SYSTEMD_UNIT=sd_test.service
    _SYSTEMD_SLICE=system.slice
    _BOOT_ID=db55b41f61144ae69cc86acfb75209fb
    _MACHINE_ID=c5a9e78e2c854065a9b041c58f07c2b2
    _HOSTNAME=dev-box
    MESSAGE=This is "stderr": 12
    _PID=21280
6
  • Check journalctl … -o verbose or -o json-pretty to see if there is anything...
    – derobert
    Jan 3, 2017 at 20:21
  • @derobert please see edited question.
    – NarūnasK
    Jan 3, 2017 at 20:38
  • Hmmm... Does setting StandardError=journal get the _TRANSPORT set to stderr?
    – derobert
    Jan 3, 2017 at 22:07
  • @derobert Please see Edit_2
    – NarūnasK
    Jan 3, 2017 at 22:39
  • 1
    I just added a feature request.
    – NarūnasK
    Jan 3, 2017 at 22:48

2 Answers 2

3

A pull-request implementing the feature is currently under review.

In the meanwhile, if you control the script run with this unit, you may use the python-systemd module to send messages from your script to the journal with the priority and options you may wish.

1

If you prepend your message with <priority-level> your message will be recorded in the journal with that priority. You can find a list with the levels, a brief description and some usage examples at this link.

Compiling this piece of code on my machine

#include <cstdio>
#include <thread>
#include <chrono>

const char* pri_msg[] {
    "EMERGENCY",
    "ALERT",
    "CRITICAL",
    "ERROR",
    "WARNING",
    "NOTICE",
    "INFO",
    "DEBUG"
};

int main(){
    int pri = 0;
    while(true) {
        std::printf("<%d>Test service priority level: %s", pri, pri_msg[pri]);
        pri = ++pri % 8;
        std::this_thread::sleep_for(std::chrono::seconds(5));
    }
}

and running it as a systemd service, each message is logged to journald with its priority level.

Then, you can retrieve them with journalctl -u *your-service* -p *priority-level*:

$ journalctl -u testd -p 3
-- Logs begin at mer 2020-10-21 17:31:13 CEST, end at ven 2020-10-23 16:01:01 CEST. --
ott 23 15:57:30 legion testd[21967]: Test service priority level: EMERGENCY
ott 23 15:57:35 legion testd[21967]: Test service priority level: ALERT
ott 23 15:57:40 legion testd[21967]: Test service priority level: CRITICAL
ott 23 15:57:45 legion testd[21967]: Test service priority level: ERROR
ott 23 15:58:10 legion testd[21967]: Test service priority level: EMERGENCY
ott 23 15:58:15 legion testd[21967]: Test service priority level: ALERT
ott 23 15:58:20 legion testd[21967]: Test service priority level: CRITICAL
ott 23 15:58:25 legion testd[21967]: Test service priority level: ERROR
ott 23 15:58:50 legion testd[21967]: Test service priority level: EMERGENCY
ott 23 15:58:55 legion testd[21967]: Test service priority level: ALERT
ott 23 15:59:00 legion testd[21967]: Test service priority level: CRITICAL

In your case I would just add a "<3>" behind every message logged with std::cerr (or fprintf(stderr, ...).

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .