0

Before a daemon process is to be executed by systemd, I need to create or change the destination of a symlink. The daemon process redirects its standard output to this symlink and therewith into a special file for each started daemon process. That is the idea. I set up the service unit file named "test_start.service":

[Unit]
Description=Test Server
After=network-online.target test_start.socket
Requires=network-online.target test_start.socket

[Service]
User=test1
Group=test1
Type=simple
ExecStartPre=/bin/bash /etc/testServer/Debug/makeOutfile.sh
ExecStart=/etc/testServer/Debug/testServer
StandardInput=socket
StandardOutput=file:/etc/testServer/Debug/test_outLink
Restart=on-failure
RestartSec=3

[Install] 
WantedBy=multi-user.target

The bash script "/etc/testServer/Debug/makeOutfile.sh" looks like this:

#!/usr/bin/env bash
timeStamp=$(date +%y-%m-%d_%Hh%Mm%Ss)
myfolder="/etc/testServer/Debug"
# create new logfile
touch $myfolder/started_$timeStamp
# remove old symlink if exists (owned by test1)
if [ -h $myfolder/test_outLink ]; then
    rm $myfolder/test_outLink
fi
# create new symlink
ln -s $myfolder/started_$timeStamp $myfolder/test_outLink

It works under certain circumstances. If a symlink exists and if the file it points to also exists, everything is just fine. The bash script "makeOutfile.sh" works via terminal for testing purposes and via a systemd service start.

But:

If the file pointed to (name: "started_atSomeDate") does not exist (because it got removed by someone meanwhile), a terminal call of the script works as expected, but starting the systemd service will add an additional file owned by root with group root with the original old file name: "started_atSomeDate".

Where does that come from?

If the old symlink "test_outLink" does not exist (because it got removed by someone meanwhile), a terminal call of the script works as expected, but starting the systemd service will add the newly created "test_outLink" as a regular file owned by root with group root and the service does not start.

Something goes terribly wrong here and although the systemd service unit User and Group is test1, root gets mixed in here. Can anybody explain? What am I doing wrong?

0

My suggestion is that you should use a shell script that will both set up the log file and execute the test server with an appropriate redirection, therefore bypassing the systemd StandardOutput= setup that doesn't work for your case.

Create a bash script /etc/testServer/Debug/runTestServer.sh like this:

#!/usr/bin/env bash
timeStamp=$(date +%y-%m-%d_%Hh%Mm%Ss)
myfolder="/etc/testServer/Debug"
# create new logfile
touch $myfolder/started_$timeStamp
# remove old symlink if exists (owned by test1)
if [ -h $myfolder/test_outLink ]; then
    rm $myfolder/test_outLink
fi
# create new symlink
ln -s $myfolder/started_$timeStamp $myfolder/test_outLink
# run the actual testServer
exec /etc/testServer/Debug/testServer >/etc/testServer/Debug/test_outLink

The only difference from your original makeOutfile.sh script is that this script executes the testServer with an appropriate redirection to send its stdout to the log file you just set up.

It also uses exec to ensure the shell script is not kept around and replaced by the testServer (so systemd will know what is the main process of the service.)

In your unit file, remove the ExecStartPre= and the StandardOutput= and replace the ExecStart= with:

ExecStart=/etc/testServer/Debug/runTestServer.sh

(Or calling the script with /bin/bash is also possible, although not necessary as long as the script has the executable bit set.)

By setting up your server through a script like this, you can even bypass the symlink altogether and do the redirection directly to the final started_$timeStamp log file.

I haven't looked at the systemd code directly and the documentation isn't clear on the interactions between StandardOutput= and ExecStartPre= in specific... But there's definitely an issue with the order in which the steps are executed by systemd (and which process executes them, which results in different permissions/ownership), so I'm not too surprised with the results you reported. I guess, regardless, my recommendation would be to simplify it with the single wrapper executor script, so hopefully that solves your problem.

  • 1
    Your workaround is good. Since I have a socket (named pipe) to stdin, I thought I definitely had to stick with systemd calling the daemon process via ExecStart alone. This is like you pointed out, not needed and the named pipe is handed to the daemon process even if it is called from a script. – bejo Dec 10 '18 at 19:00
  • Apparently by starting the daemon your way via an exec call in the script, systemd does not restart the service on failure. Or am I missing something? I can only kill the process as root and after killing there is no restart anymore. Do you have a workaround that also covers restarts on failure? – bejo Dec 10 '18 at 19:37
  • @bejo Restart on failure should work just fine. Also, the server should be running under user "test1" (so you should be able to kill it with that user.) Take a look at ps and pstree etc. to confirm this is working as supposed to... – filbranden Dec 10 '18 at 21:50
  • I can confirm that the daemon process can also be killed by its owner test1 - everything else would be a little strange;-) My user test1 is a system user with unable to log in. So I killed its process via sudo runuser -l test1 --shell /bin/bash -c 'kill "daemonPID"'. I wonder, why I was able to kill the daemon process as a different user (my regular user) before I made your suggested changes. I will go after that one later. – bejo Dec 11 '18 at 7:25
  • 1
    Hm - I see. Thanks for your patience and for clarifying that. I thought SIGKILL will be sent when a process gets killed and not SIGTERM. It was only for testing purposes, to check whether a restart occurs other than by a regular termination of the daemon by itself. So I am fine now. I learned that a process can be killed by specifying SIGKILL - tested that and could happily observe an automatic restart. Fantastic. Thanks. – bejo Dec 12 '18 at 19:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.