1

I have a systemd service which spawns an ffmpeg process any time it detects a new event. These processes are meant to survive and run to completion even if the service is restarted (hence them being detached as soon as possible).

Currently, I have it set up with the following service unit file:

[Unit]
Description=My Service

[Service]
Environment="VAR1=val1" "VAR2=val2"
Type=exec
KillMode=process
ExecStart=/path/to/service/executable
Restart=on-failure

[Install]
WantedBy=default.target

While this works, it's not "proper" as per systemd's warnings in journald.

systemd[1]: Found left-over process 789794 (ffmpeg) in control group while starting unit. Ignoring.
systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.

So, what is the right way to handle detaching processes that should outlive their parent in systemd?

5
  • Can you tell us more about what you're doing with ffmpeg? That might help us come up with a more appropriate answer.
    – larsks
    Jul 5, 2023 at 4:01
  • @larsks It really shouldn't matter, but sure. It's capturing live streams when it detects a monitored stream go live. The service is the thing watching for the "stream went live" event. I know know you could have another service just for capture, but there are multiple parameters passed into ffmpeg that this is the best way I've found.
    – plunker
    Jul 5, 2023 at 4:07
  • 1
    Maybe you want a template service/unit file that spawns a separate service for each ffmpeg-instance?
    – FelixJN
    Jul 5, 2023 at 11:15
  • There are several parameters that need to be passed in which are dynamically determined by the main service. As far as I'm aware, templating does not allow more than one parameter and pulling out the code that determines the paramters is bad since I can coalesce requests in the service.
    – plunker
    Jul 6, 2023 at 18:13
  • It's not "proper" because you lose control over those processes. You can't check the status, and can't manage them within systemd. You'll need things like ps and kill <PID>. I'm not sure if they log to the journal either. It's a valid warning. templating is the way to go. There are other methods to pass data into a template other than the instance name. A *.path or a *.socket are examples.
    – Stewart
    Jul 14, 2023 at 9:24

2 Answers 2

2
+25

I suggest you run the long running process in another systemd unit. For that you create a second systemd unit that is a template unit. That way you can start different instances of it with different arguments. Normally systemd runs each unit only once (and if you start it while it runs nothing else happens). Template units end in @ and you can start them with any string after the @. So different instances with different strings after the @ can run at the same time.

Unit 1 is meant to detect new streams:

[Unit]
Description=My Service

[Service]
Environment="VAR1=val1" "VAR2=val2"
Type=exec
# KillMode=process  # don't do this read the man page about it
ExecStart=/path/to/service/executable
Restart=on-failure

[Install]
WantedBy=default.target

And in your program /path/to/service/executable as soon as you detect a new event/live stream you start your template service with the name of the live stream as instance argument: systemctl start [email protected]

The service file [email protected] might look something like this:

[Unit]
Description=Process one live stream

[Service]
# the string "%i" is substituted by systemd with the stuff you give it after the @ in the unit name.
ExecStart=/usr/bin/ffmpeg --some-args --live-stream=%i

EDIT:

If you have many parameters you could write them to a temp file and read them in the template service via EnvironmentFile:

[Service]
EnvironmentFile=/my/folder/%i
ExecStart=/usr/bin/ffmpeg --some-args $ARGS_FROM_ENV_FILE
6
  • I've mentioned this in the comments to my original question, but I have more than a single parameter that needs to be set. in the call to FFMPEG. It's literally impossible to template it via systemd templates.
    – plunker
    Jul 13, 2023 at 21:05
  • This is definitely the way to go. Much better than leaving orphaned processes because you maintain control and status of each process. It is not literally impossible. What I would do is create a *.socket with Accept=yes and send the start command to the socket. Then when I need to start a new ffmpeg, it will launch a template which can read the socket and fork a new process based on the contents received. Maybe I can make an example this evening.
    – Stewart
    Jul 14, 2023 at 9:19
  • If you have to pass several arguments that can not be put into the single templating parameter and can also not be written to a temp file (which could be used as templating parameter) and can not be passed via a socket, then there is also systemd-run. It allows you to run a shell command as an ad-hoc unit. That way your ffmpeg process does not escape the process manager.
    – Lucas
    Jul 14, 2023 at 9:38
  • It is literally impossible to pass more than a single parameter to a systemd template. It is possible to work around the restriction (by using temp files or using sockets), but that would require yet another process instead of running ffmpeg directly as the main process. systemd-run is closer to solving it, but still has other issues that I've run into (ie. I had already tried it as a possible solution).
    – plunker
    Jul 14, 2023 at 14:00
  • @plunker why do you need an extra process to read the temp file? What about EnvironmentFile= does that idea not work (see my edit)?
    – Lucas
    Jul 16, 2023 at 7:39
1

You can make a couple of changes to your service unit file:

  1. Change Type=exec to Type=simple. This indicates that your service will directly start the ffmpeg process without forking or backgrounding it.

  2. Add RemainAfterExit=yes to the [Service] section. This tells systemd to consider the service as still active even after the main process has exited.

Here's the updated service unit file:

[Unit]
Description=My Service

[Service]
Environment="VAR1=val1" "VAR2=val2"
Type=simple
KillMode=process
ExecStart=/path/to/service/executable
Restart=on-failure
RemainAfterExit=yes

[Install]
WantedBy=default.target

systemd will no longer complain about left-over processes. It acknowledges that the ffmpeg process is intentionally detached and allows it to run to completion even if the service is restarted. Hope that helps!

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .