Have SystemD spawn N processes?

Question

At my organization, we have a number of queue consuming worker processes. We're currently using SupervisorD to manage them, but would like to use SystemD if possible for certain advantages therein. I'm fairly experienced with writing custom units, but I don't immediately have an analog in SystemD land for this.

In the SupervisorD documentation a parameter called numprocs is detailed which allows one to set the number of processes they'd like to be started with the service. If I want 30 processes started, it's a one-line change.

Is there a setting in SystemD units that will allow me to specify how many of these processes I'd like started?

Answer 1

What Munir mentioned is exactly how you do this. Basically you create a service file, and start it 30 times. Now that may seem a little unweildy, but it has advantages, such as being able to shut one of them down if it's misbehaving, and not have to shut all of them down. There are also some things you can do to make management easier.

First, the unit file. Create a file, such as /etc/systemd/system/[email protected]. The important bit is the @ symbol.

It's contents might look like:

[Service]
ExecStart=/bin/sleep 600 %I

[Install]
WantedBy=multi-user.target

Then start it with systemctl start [email protected], systemctl start [email protected].
The processes that get launched will look like:

root     17222  19   0  0.0  0.0 Ss         00:05 /bin/sleep 600 1
root     17233  19   0  0.0  0.0 Ss         00:02 /bin/sleep 600 2

Notice that the %I got substituted with whatever you put after the @ when you started it.

You can start all 30 with a little shell-fu:

systemctl start test@{1..30}.service

You can also enable them at boot like any normal service: systemctl enable [email protected].

 

Now, what I meant by things you can do to make management easier: Maybe you don't want to have to use test@{1..30}.service for managing them all. It is a little unwieldy. You can instead create a new target for your service.

Create /etc/systemd/system/test.target with:

[Install]
WantedBy=multi-user.target

Then adjust the /etc/systemd/system/[email protected] so that it looks like:

[Unit]
StopWhenUnneeded=true

[Service]
ExecStart=/bin/sleep 600 %I

[Install]
WantedBy=test.target

Reload systemd with systemctl daemon-reload (only necessary if you are modifying the unit file, and didn't skip the earlier version of it). And now enable all the services you want to be managed by doing systemctl enable test@{1..30}.service.
(If you had previously enabled the service while it had WantedBy=multi-user.target, disable it first to clear out the dependency)

You can now do systemctl start test.target and systemctl stop test.target, and it will start/stop all 30 processes.
And again, you can enable at boot like any other unit file: systemctl enable test.target.

Answer 2

Here is my example using a python script that runs in a virtualenv:

/etc/systemd/system/[email protected]

[Unit]
Description=manages my worker service, instance %i
After=multi-user.target

[Service]
PermissionsStartOnly=true
Type=idle
User=root
ExecStart=/usr/local/virtualenvs/bin/python /path/to/my/script.py
Restart=always
TimeoutStartSec=10
RestartSec=10

Disable: sudo systemctl disable my-worker\@{1..30}.service

Enable N workers: sudo systemctl enable my-worker\@{1..2}.service

Reload: sudo systemctl daemon-reload

Start: sudo systemctl start [email protected]

Check status: sudo systemctl status my-worker@1