9

I have a bunch of services (say C0, C1, … C9) that should only start after a service S has completed its initialization and is fully running and ready for the other services. How do I arrange that with systemd?

In Ordering services with path activation and target in systemd it is assumed that service S has a mechanism for writing out some sort of flag file. Assume here, in contrast, that I have full control over the program that service S runs, and can add systemd mechanisms into it if needs be.

7

One doesn't necessarily need this.

If the C services need to wait for S to be ready so that they can open a socket connection to it, then one doesn't necessarily need to do this at all. Rather, one can take advantage of early listening socket opening by service managers.

Several systems, including Laurent Bercot's s6, my nosh toolset, and systemd, have ways in which a listening socket can be opened early on, the very first thing in setting up the service. They all involve something other than the service program opening the listening socket(s), and the service program, when invoked, receiving the listening sockets(s) as already-open file descriptors.

With systemd, specifically, one creates a socket unit that defines the listening socket. systemd opens the socket unit and sets it up so that the kernel networking subsystem is listening for connections; and passes it to the actual service as an open file descriptor when it comes to spawn the process(es) that handle(s) connections to the socket. (It can do this in two ways, just like inetd could, but a discussion of the details of Accept=true versus Accept=false services is beyond the scope of this answer.)

The important point is that one does not necessarily need more ordering than that. The kernel batches up client connections in a queue until the service program is initialized, and ready to accept them and talk to clients.

When one does, readiness protocols are the thing.

systemd has a set of readiness protocols that it understands, specified service by service with the Type= setting in the service unit. The particular readiness protocol of interest here is the notify readiness protocol. With it, systemd is told to expect messages from the service, and when the service is ready it sends a message that flags readiness. systemd delays the activation of the other services until readiness is flagged.

Making use of this involves two things:

  • Modifying the code of S so that it calls something like Pierre-Yves Ritschard's notify_systemd() function or Cameron T Norman's notify_socket() function.
  • Setting up the service unit for the service with Type=notify and NotifyAccess=main.

The NotifyAccess=main restriction (which is the default) is because systemd needs to know to ignore messages from mischievous (or just plain faulty) programs, because any process on the system can send messages to systemd's notification socket.

One uses Pierre-Yves Ritschard's or Cameron T Norman's code for preference because it does not exclude the possibility of having this mechanism on UbuntuBSD, Debian FreeBSD, actual FreeBSD, TrueOS, OpenBSD, and so forth; which the code supplied by the systemd authors does exclude.

One trap to avoid is the systemd-notify program. It has several major problems, not the least of which is that messages sent with it can end up being thrown away unprocessed by systemd. The most major problem in this case is that it doesn't run as the "main" process of the service, so one has to open up the readiness notifications for the service S to every process on the system with NotifyAccess=all.

Another trap to avoid is thinking that the forking protocol is simpler. It is not. Doing it correctly involves not forking and exiting the parent until (for one thing) all of the program's worker threads are running. This does not match how the overwhelming majority of dæmons that fork actually fork.

Further reading

| improve this answer | |
  • 1
    According to man systemd.service(5), NotifyAccess=all will accept messages from all members of the service's control group, which does not imply just any rogue process on the system. This is secure enough for most use cases. Also, your concern about portability to other operating systems is not relevant to OP, since we are already on the topic of Systemd here. – Amir Jan 4 '18 at 7:18
1

Referring to the manual page for systemd.service(5), specifically the section about Type=, each service type has a different way for Systemd to determine that it is ready to offer functionality to other services:

  • If Type=simple, its communication channels should be installed before the daemon is started up (e.g. sockets set up by systemd, via socket activation).

  • If Type=forking, the parent process is expected to exit when start-up is complete and all communication channels are set up.

  • If Type=dbus, it is expected that the daemon acquires a name on the D-Bus bus, at which point systemd will proceed with starting follow-up units.

  • If Type=notify, it is expected that the daemon sends a notification message via sd_notify(3) or an equivalent call when it has finished starting up. systemd will proceed with starting follow-up units after this notification message has been sent.

For the last option (sending a message via sd_notify), you can use the systemd-notify utility, and remember to grant it access with NotifyAccess=all.

Given that you have control over service S, you are free to choose the best option for your use case, or simply the one that is easiest to implement.

| improve this answer | |
1

like this:

S.service

[Unit]
Description=My main Service

[Service]
Type=notify
ExecStart=/usr/bin/myBinary

C0.service

[Unit]
Description=Dependent service number 0
PartOf=S.service

C1.service

[Unit]
Description=Dependent service number 1
PartOf=S.service

C9.service

[Unit]
Description=Dependent service number 9
PartOf=S.service

Where /usr/bin/myBinary makes an sd_notify READY=1 call when it's initialization is complete.

Depending on how you want the dependency to behave, you can use PartOf, Requires, or BindsTo or others.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.