15

I'm trying to run celery as a service as described in the docs.

The documentation uses %n%I specifiers for the log files:

celery.service:

ExecStart= [...] --logfile=${CELERYD_LOG_FILE}

celery.service.d/celery.conf

Environment="CELERYD_LOG_FILE=/var/log/celery/%n%I.log"

When doing this, I can see in the service status that %n is resolved at start time and %I remains at this stage:

systemctl status celery.service
[...]
    --logfile=/var/log/celery/worker%I.log

And I get those log files:

/var/log/celery/worker.log
/var/log/celery/worker-1.log
/var/log/celery/worker-2.log

If I use %i, however, the whole thing resolves at start time

celery.service.d/celery.conf

Environment="CELERYD_LOG_FILE=/var/log/celery/%n%i.log"

produces this:

systemctl status celery.service
[...]
    --logfile=/var/log/celery/celery.service.log

And I get only one log file:

/var/log/celery/celery.service.log

This is troubling.

From systemd documentation, the only difference should be about escaping:

  • "%i" | Instance name | For instantiated units: this is the string between the "@" character and the suffix of the unit name.
  • "%I" | Unescaped instance name | Same as "%i", but with escaping undone

Is there something I'm missing, here?

Also, I noticed that if I set the log path directly in the .service file, only the %n%i form is accepted.

ExecStart= [...] --logfile=/var/log/celery/%n%i.log

will do, and results in

/var/log/celery/celery.service.log

while

ExecStart= [...] --logfile=/var/log/celery/%n%I.log

triggers an error:

celery.service failed to run 'start' task: Operation not supported
Failed to start Celery worker.

How come?

I'm using systemd 215-17 on Debian Jessie.

Edit 1:

It seems %I is not understood by systemd at all. What we see when using %I is specific to Celery. (See Celery docs). So %i is managed by systemd while %I is ignored and passed transparently by systemd, then managed by Celery.

This explains a lot but leaves a few questions open:

  • Why doesn't systemd understand %I here?
  • Conversely, what if I wanted to pass Celery a %i?
  • Why does it differ if I pass the option directly in the .service file rather than in the .conf file?

I went through systemd changelog and didn't find anything about %I being more recent than the version I am using.

Edit 2:

I saw this error message while running systemctl status celery.service:

[/etc/systemd/system/celery.service.d/celery.conf:18] Failed to resolve specifiers, ignoring: "CELERYD_LOG_FILE=/var/log/celery/%n%I.log"

I can't reproduce it, though. I can't tell why it happened once and and not every time.

1
  • 1
    The celery doc you refer to is not using a service template, or the unit file would be named [email protected] (which you can try). You should probably replace every % with %% so systemd does no processing of the specifiers, since your celery application seems to be wanting to do it.
    – meuh
    Oct 9, 2017 at 17:50

1 Answer 1

18

From the documentation you already cited, the difference between %i and %I is the "escaping" of the passed parameter.

The next question is: What does "escaping" mean?

Escaping means changing the special characters. %i replaces the special characters / by dash - and by \x20.

WARNING: if the instance name contains a dash -, this dash gets replaced in the variable %I. That's why the documentation says "with escaping undone".

Example

Let's take [email protected] containing the following lines:

[Service]
Environment="OPTIONS=%I %i"
ExecStart=/usr/bin/bash -c "echo $OPTIONS >> /tmp/test-specifier"

Let's then start the service with: systemctl start some-service@/etc/path/to/some-conf The output in /tmp/test-specifier will be:

/etc/path/to/some/conf -etc-path-to-some-conf

Notice that some-conf has become some/conf in the variable %I.

Conclusion

Thus, the behavior for the instance name /etc/path/to/some-conf is:

  • %i specifier with escaping: -etc-path-to-some-conf
  • %I specifier without escaping: /etc/path/to/some/conf

Test setup

CentOS 7 with systemd-219-67

String escaping for systemd can be tried using systemd-escape from the command line.

References

https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Specifiers http://0pointer.de/blog/projects/instances.html https://www.freedesktop.org/software/systemd/man/systemd-escape.html

1
  • Very informative summary of the docs with a hands on example.
    – Alex
    Jun 15, 2023 at 21:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .