26

Is there a way to dynamically assign environment variables in a systemd service unit file?

We have a machine that has 4 GPUs, and we want to spin up multiple instances of a certain service per GPU. E.g.:

  • gpu_service@1:1.service
  • gpu_service@2:1.service
  • gpu_service@3:1.service
  • gpu_service@4:1.service
  • gpu_service@1:2.service
  • gpu_service@2:2.service
  • gpu_service@3:2.service
  • gpu_service@4:2.service
  • ad nauseam

So the 1:1, 2:1, etc. are effectively the %i in the service unit file.

In order for the service to bind to a particular GPU, the service executable checks a certain environment variable, e.g.:

USE_GPU=4

Is there a way I can take %i inside the service unit file and run it through some (shell) function to derive the GPU number, and then I can set the USE_GPU environment variable accordingly?

Most importantly, I don't want the hassle of writing multiple /etc/systemd/system/gpu_service@x:y.service/local.conf files just so I can spin up more instances.

Improve this question

4 Answers 4

Reset to default
23

If you are careful you can incorporate a small bash script sequence as your exec command in the instance service file. Eg

ExecStart=/bin/bash -c 'v=%i; USE_GPU=$${v%:*} exec /bin/mycommand'

The $$ in the string will become a single $ in the result passed to bash, but more importantly will stop ${...} from being interpolated by systemd. (Earlier versions of systemd did not document the use of $$, so I don't know if it was supported then).

Improve this answer
4
  • 1
    I ended up doing something like that. :)
    – Kal
    Nov 18, 2016 at 8:10
  • 10
    Call a bash -c to start a program from the unit file? Call exec? This is like using a forklift on top of a forklift (maybe with another forklift on top) because the first forklift has trouble actually forklifting.
    – David Tonhofer
    Dec 31, 2018 at 18:50
  • Something like this would work. Or a wrapper script to split that variable :) The other bizarre option would be to create another service to "run beforehand" and setup an EnvironmentFile, not sure how that would work with templates tho: stackoverflow.com/a/42841480/32453
    – rogerdpack
    Jul 6, 2020 at 15:18
  • 2
    @DavidTonhofer How would a solution without "double forklifter" look like? I see that the bash is there to setup and mangle %i into USE_GPU. Then exec replaces the shell process with the real one.
    – U. Windl
    Feb 10, 2021 at 13:34
10

No built in way. You need to do these things before your service starts. One way would be putting it to an environment file.

[Service]
# Note you need to escape percentage sign
ExecStartPre=/bin/sh -c "my_awesome_parser %%i > /run/gpu_service_%i"
EnvironmentFile=/run/gpu_service_%i
ExecStart=...
Improve this answer
1
  • 5
    Unfortunately you can't use an ExecStartPre to write an env file, then use it, apparently it has to be written beforehand.
    – rogerdpack
    Jul 6, 2020 at 15:15
3

It looks like you can indeed set environment variables inside a systemd unit file...

Per suggestions from commenters, here is the solution:

Using environment variables in systemd units

Environment directive

systemd has an Environment directive which sets environment variables for executed processes. It takes a space-separated list of variable assignments. This option may be specified more than once in which case all listed variables will be set. If the same variable is set twice, the later setting will override the earlier setting. If the empty string is assigned to this option, the list of environment variables is reset, all prior assignments have no effect. Environments directives are used in built-in Container Linux systemd units, for example in etcd2 and flannel.

With the example below, you can configure your etcd2 daemon to use encryption. Just create /etc/systemd/system/etcd2.service.d/30-certificates.conf drop-in for etcd2.service:

[Service]
# Client Env Vars
Environment=ETCD_CA_FILE=/path/to/CA.pem
Environment=ETCD_CERT_FILE=/path/to/server.crt
Environment=ETCD_KEY_FILE=/path/to/server.key
# Peer Env Vars
Environment=ETCD_PEER_CA_FILE=/path/to/CA.pem
Environment=ETCD_PEER_CERT_FILE=/path/to/peers.crt
Environment=ETCD_PEER_KEY_FILE=/path/to/peers.key

Then run sudo systemctl daemon-reload and sudo systemctl restart etcd2.service to apply new environments to etcd2 daemon.

Quoted text taken from the following URL: https://coreos.com/os/docs/latest/using-environment-variables-in-systemd-units.html

Improve this answer
5
  • 3
    While this may theoretically answer the question, it would be preferable to include the essential parts of the answer here, and provide the link for reference.
    – Stephen Rauch
    Feb 23, 2017 at 5:12
  • 3
    While your comment may theoretically improve my future responses in stackexchange, it would be preferable for you to have included the essential parts of the answer in your comment instead of just commenting to point out how incompetent someone may be :)
    – CyberK
    Mar 1, 2017 at 3:43
  • 2
    Welcome to Stack Exchange! Thanks for the comment, you made me smile. Also thanks for taking the time to edit your answer. We are trying to build something that will have value over time, and link only answers just don't age very well.
    – Stephen Rauch
    Mar 2, 2017 at 8:06
  • If you add Environment=ABC=%i it does set that env. variable "to the entirety of %i". I guess you could make a wrapper to strip off the "stuff beyond the quote" you don't want, and it call the real executable. But if you're making a wrapper, you could even just pass %i in as an argument to it ex: ExecStart=my_wrapper %i
    – rogerdpack
    May 28, 2019 at 20:02
  • 5
    The question was for "dynamic" variables; you've simply given us the answer for the static solution.
    – Otheus
    Feb 12, 2020 at 11:38
1

It's ugly and not quite what you asked for, nor does it allow for autostart, but for followers it is possible to do something using the systemctl environment:

$ sudo systemctl set-environment USE_GPU=4 # add it to the env. variables for future services
$ sudo systemctl start gpu_service@4:2.service

One problem is if you run/start more than one "at once" (in parallel) then it could cause confusion/race conditions.

Just trying to list all the options possible, some of the other answers work as well :)

Improve this answer
1
  • 4
    +1: ExecStartPre=/bin/bash -c "/bin/systemctl set-environment USE_GPU=4" is the best way to do this of the answers posted here.
    – ZachB
    Jul 3, 2020 at 18:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .