How to add custom journal field value in a bash systemd unit

Question

I have systemd service that runs a bash file. All the output of this script is logger in the systemd journal, and correctly attributed to the corresponding unit by systemd:

# myservice.sh
echo "PROGRESS 1/5"
echo "PROGRESS 2/5"
echo "PROGRESS 3/10"
echo "PROGRESS 9/10"

> journalctl -eu my-service
oct. 19 16:01:38 my-computer systemd[1]: Started My service.
oct. 19 16:01:40 my-computer my-service[137083]: PROGRESS: 1/5
oct. 19 16:01:42 my-computer my-service[137083]: PROGRESS: 2/5
oct. 19 16:01:44 my-computer my-service[137083]: PROGRESS: 3/10
oct. 19 16:01:46 my-computer my-service[137083]: PROGRESS: 9/10
oct. 19 16:01:46 my-computer systemd[1]: my-service.service: Succeeded.

However, I would like to add additional information in my log entries, by adding new fields. I tried to use logger for that without success: the fields are not stored correctly and the logged lines are not flagged as comming from my-service.service.

# myservice.sh
function log_progress() {
    logger --tag my-service --priority user.info \
           --rfc5424
           --sd-id lcupdater@1 \
           --sd-param "PROGRESS_VALUE=\"$2\"" \
           --sd-param "PROGRESS_TO=\"$3\"" \
            "$1"
}

log_progress "Doing stuff" 1 5
log_progress "Doing more stuff" 2 5
log_progress "Going on..." 3 10
log_progress "Almost done..." 9 10

> journalctl -e -o json
{
  "_UID": "0",
  "_GID": "0",
  "_PID": "147370",
  "_SOURCE_REALTIME_TIMESTAMP": "1603120378691356",
  "MESSAGE": "1 2020-10-19T17:12:58.691328+02:00 my-computer my-service - - [timeQuality tzKnown=\"1\" isSynced=\"1\" syncAccuracy=\"132000\"][lcupdater@1 PROGRESS_VALUE=\"9\" PROGRESS_TO=\"10\"] Almost d
one...",
  "__MONOTONIC_TIMESTAMP": "28976809066",
  "_TRANSPORT": "syslog",
  "SYSLOG_RAW": "<14>1 2020-10-19T17:12:58.691328+02:00 my-computer my-service - - [timeQuality tzKnown=\"1\" isSynced=\"1\" syncAccuracy=\"132000\"][lcupdater@1 PROGRESS_VALUE=\"9\" PROGRESS_TO=\"10\"] A
lmost done...",
  "SYSLOG_FACILITY": "1",
  "PRIORITY": "6",
  "__REALTIME_TIMESTAMP": "1603120378691368"
}

The field _SYSLOG_UNIT is missing, and PROGRESS_VALUE and PROGRESS_TO are not stored correclty. Is there a way to achieve this?

EDIT:

A verbose but more correct way to achieve this is to use logger --journald:

function log_progress() {
    logger --journald <<EOF
MESSAGE=$1
PROGRESS_VALUE=$2
PROGRESS_TO=$3
SYSLOG_IDENTIFIER=my-service
SYSLOG_FACILITY=3
PRIORITY=6
_SYSTEMD_UNIT=my-service.service
EOF
}

However, the field _SYSTEMD_UNIT is stripped, as only systemd is allowed to manipulate it.

Answer 1

It is definitely possible to add additional fields according to the systemd.journal-fields documentation (“New fields may freely be defined by applications, […]” but the only solution I found is using native messaging via the journald-socket and a c-library. I haven't found anything which can easily used within shell-scripts, not even systemd-cat accepts json as input.

As of now, the easiest way is probably to rewrite your application in python and use the systemd.journal-module