4

systemd appears to split very long log lines in multiple log messages:

$ journalctl -u myunit
Nov 12 08:00:18 ovh7 uwsgi[32441]: SHORT LINE
Nov 12 08:00:18 ovh7 uwsgi[32441]: START of VERY VERY LONG LINE ON STDOUT
Nov 12 08:00:18 ovh7 uwsgi[32441]: CONTINUE VERY VERY LONG LINE
Nov 12 08:00:18 ovh7 uwsgi[32441]: SHORT LINE

Now, I do not really mind but I need to join the separate log messages to get back the original stdout from my process. I thought I could merely comb through the logs for the pid and join the MESSAGE strings using the journalctl json output:

def main():
    import optparse, json, sys
    parser = optparse.OptionParser()
    parser.add_option('--pid')
    parser.add_option('-f', '--file')
    options, args = parser.parse_args()

    with open(options.file, 'r') as f:
        for line in f:
            d = json.loads(line)
            if d['_PID'] == options.pid:
                sys.stdout.write(d['MESSAGE'].encode("utf-8"))

Sadly, the above does not work because systemd also appears to trim the trailing \n from stdout. It generates this:

SHORT LINE START of VERY VERY LONG LINE ON STDOUT CONTINUE VERY VERY LONG LINE SHORT LINE

Now, I can try to add an extra \n for each MESSAGE but this generates this:

SHORT LINE
START of VERY VERY LONG LINE ON STDOUT
CONTINUE VERY VERY LONG LINE
SHORT LINE

None of the two above results are very helpful. I need this:

SHORT LINE
START of VERY VERY LONG LINE ON STDOUT CONTINUE VERY VERY LONG LINE
SHORT LINE

And I see nothing in the output of journalctl that would allow me to infer that two consecutive messages are coming from the same line in the original stdout output. Any idea that would allow me to reconstruct this data correctly without having to generate a separate log file for my program's stdout ?

  • 1
    One heuristic is to concatenate when a message length is 2048 bytes (or a bit less as whitespace is removed). – meuh Nov 12 '16 at 15:33
  • good point meuh – mathieu Nov 12 '16 at 17:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.