systemd service which fails restart shows up as inactive rather than failed

Question

Consider the following service:

[Unit]
Description=foobar

[Service]
Type=simple
ExecStart=/usr/bin/false
Restart=on-failure

[Install]
WantedBy=multi-user.target

I'd expect it restart a few times, and then quiesce as a failed unit. However, this is what actually happens on systemd 229 with Ubuntu Xenial:

root@monitoring:/etc/systemd/system# systemctl status foobar
● foobar.service - foobar
   Loaded: loaded (/etc/systemd/system/foobar.service; disabled; vendor preset: 
   Active: inactive (dead)

Apr 01 21:08:43 monitoring.ci.ezyang.com systemd[1]: foobar.service: Service hol
Apr 01 21:08:43 monitoring.ci.ezyang.com systemd[1]: Stopped foobar.
Apr 01 21:08:43 monitoring.ci.ezyang.com systemd[1]: Started foobar.
Apr 01 21:08:43 monitoring.ci.ezyang.com systemd[1]: foobar.service: Main proces
Apr 01 21:08:43 monitoring.ci.ezyang.com systemd[1]: foobar.service: Unit entere
Apr 01 21:08:43 monitoring.ci.ezyang.com systemd[1]: foobar.service: Failed with
Apr 01 21:08:44 monitoring.ci.ezyang.com systemd[1]: foobar.service: Service hol
Apr 01 21:08:44 monitoring.ci.ezyang.com systemd[1]: Stopped foobar.
Apr 01 21:08:44 monitoring.ci.ezyang.com systemd[1]: foobar.service: Start reque
Apr 01 21:08:44 monitoring.ci.ezyang.com systemd[1]: Failed to start foobar.

Importantly, it doesn't list as a failed unit in systemctl --failed, or degrade the status of systemctl status. Why?

Answer 1

This is a bug in systemd, and it is fixed systemd 238, and possibly earlier versions.

[root@866d9b9835e0 system]# systemctl status foobar
* foobar.service - foobar
   Loaded: loaded (/etc/systemd/system/foobar.service; disabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Sun 2018-04-01 21:06:21 UTC; 14min ago
  Process: 143 ExecStart=/usr/bin/false (code=exited, status=1/FAILURE)
 Main PID: 143 (code=exited, status=1/FAILURE)

Apr 01 21:06:21 866d9b9835e0 systemd[1]: foobar.service: Service hold-off time over, scheduling restart.
Apr 01 21:06:21 866d9b9835e0 systemd[1]: foobar.service: Scheduled restart job, restart counter is at 5.
Apr 01 21:06:21 866d9b9835e0 systemd[1]: Stopped foobar.
Apr 01 21:06:21 866d9b9835e0 systemd[1]: foobar.service: Start request repeated too quickly.
Apr 01 21:06:21 866d9b9835e0 systemd[1]: foobar.service: Failed with result 'exit-code'.
Apr 01 21:06:21 866d9b9835e0 systemd[1]: Failed to start foobar.
[root@866d9b9835e0 system]# systemctl --version
systemd 238
+PAM -AUDIT -SELINUX -IMA -APPARMOR +SMACK -SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN +PCRE2 default-hierarchy=hybrid

You can work around the problem by avoiding use of Restart in systemd.