4

In SysV I can use a condition to ensure an application does not try to start before the database is up and running. I give the init script some time to wait, then finally give up after a period of time if the database service is still unavailable.

start() {
local exec=/path/to/exec
local tries=1
[ -x $exec ] || exit 5
echo -n $"Starting $prog: "


#check communication to database
if ! [ 2>/dev/null : < /dev/tcp/$dbHost/$dbPort ]
    then
        while ! [ 2>/dev/null : < /dev/tcp/$dbHost/$dbPort ] && [ ! $tries -ge 5 ]
            do
                >&2 echo -e "Could not connect to the database on $dbHost\nWaiting 10 seconds to check database status, attempt $tries"
                sleep 10
                ((tries++))
        done
        sleep 10
        if ! (: < /dev/tcp/$dbHost/$dbPort ) 2>/dev/null
            then
                >&2 echo -e "Could not connect to the database on $dbHost aborting startup of $exec"
                exit 1
        fi
fi

I've been searching for a similar scenario in documentation and google, but have not found anything that is not referencing local services.

  • Hm...maybe one option would be to put that into a regular shell script and kick that off through systemd. Or use Restart and RestartSec, maybe TimeoutStartSec in systemd to restart the service on failure. A more sophisticated way could be the use of ExecStartPre to check for the connection. – Thomas Mar 23 '18 at 18:51
2

Have you looked at ExecStartPre in the systemd service documentation?

I would suggest putting your database test in a script, have it use exit 0 on success and exit 1 on failure and then run it with ExecStartPre. You would then start your application using ExecStart.

  • I thought about that, but then thought, why not just leave it all in this one SysV script. – Michael Kelly Mar 23 '18 at 19:17
  • Then why use systemd? – GracefulRestart Mar 23 '18 at 20:29
  • I feel that resistance is futile, but would like to write elegant systemd init scripts as I have done with SysV for a couple of decades. :) – Michael Kelly Mar 23 '18 at 20:59
  • there are no init scripts in systemd, there are only service units which can easily execute init scripts. The writeup from @ErikF is a great example of a good one with the bonus that it is reusable for other services. – GracefulRestart Mar 23 '18 at 21:22
3

@GracefulRestart's answer is the best if you only have one service depending on the availability of the database. However, if you have multiple services that have this requirement, make a oneshot service that all of the services can then have a Requires= dependency to:

/etc/systemd/system/portopen@.service

[Unit]
Description=Checks database availability on %I
After=network.target
Requires=network.target

[Service]
Type=oneshot
RemainAfterExit=true
ExecStart=/path/to/portopen.sh %I

[Install]
WantedBy=default.target

/path/to/portopen.sh

#!/bin/bash
dbhost=${1:-localhost}
dbport=${2:-5678}
maxtries=${3:-5}
wait=${4:-10}
tries=1

# Check communication to database
while ! [ 2>/dev/null : < /dev/tcp/$dbhost/$dbport ]; do
   echo "Unable to connect to database on $dbhost TCP/$dbport (attempt $tries): retrying in $wait seconds" >&2
   (( tries++ ))
   if [[ $tries -le $maxtries ]]; then
      sleep $wait
   else
      echo "Unable to connect to database on $dbhost TCP/$dbport: aborting"
      exit 1
   fi
done

I made the script a bit more flexible in case you change or add database servers, or the port changes, or you want to change the number of retries on a per-service level. If you don't need that, just call the service portopen.service and remove the %I parts.

Let's say your database server is on foobar and your database application runs on foobarapp.service. Make the following changes to foobarapp.service:

# systemctl edit foobarapp.service

[in editor]

[Unit]
After=portopen@foobar.service
Requires=portopen@foobar.service

Then reload systemd and start and enable the check:

# systemctl daemon-reload
# systemctl enable portopen@foobar.service

You can then restart foobarapp.service whenever you want. It should only start if portopen@foobar.service returns successfully.

If it doesn't already exist, the database application service foobarapp.service would then look like this:

/etc/systemd/system/foobarapp.service

[Unit]
Description=Foobar database application
# Add any other dependencies here
After=portopen@foobar.service
Requires=portopen@foobar.service

[Service]
# If it is a daemon, use "forking" instead
Type=simple
ExecStart=/path/to/exec

[Install]
WantedBy=default.target
  • In reading through the documentation I had come to the conclusion that this would require an external script. So there is no way to have a condition within the single systemd service that says "only continue if 2>/dev/null : < /dev/tcp/$dbHost/$dbPort returns 0" and try x number of times, that you are aware of? I mean without it calling an external script to do so. – Michael Kelly Mar 23 '18 at 20:56
  • systemd has restart directives, but they only apply to services that aren't oneshot (like this one is); they're more designed to restart failed services that are long-running. An external script is going to be needed for what you are wanting to do. – ErikF Mar 24 '18 at 6:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.