I just switched to debian jessie, and most things run okay, including my graphical display manager wdm.

The thing is, I just don't understand how this works. Obviously my /etc/init.d/wdm script is called, because when I put an early exit in there, wdm is not not started. But when I alternatively rename the /etc/rc3.d directory (my default runlevel used to be 3), then wdm is still started.

I could not find out how systemd finds this script and I do not understand what it does to all the other init.d scripts.

  • When and how does systemd run init.d scrips?
  • In the long run, should I get rid of all init.d scripts?
up vote 124 down vote accepted

chaos' answer is what some documentation says. But it's not what systemd actually does. (It's not what System V rc did, either. Linux System V rc most definitely did not ignore LSB headers, which insserv used to calculate static orderings, for starters.) The Freedesktop documentation, such as that "Incompatibilities" page, is in fact wrong, on these and other points. (The HOME environment variable in fact is often set, for example. This went wholly undocumented anywhere for a long time. It's now documented in the manual, at least, but that Freedesktop WWW page still hasn't been corrected.)

The native service format for systemd is the service unit. systemd's service management proper operates solely in terms of those, which it reads from one of nine directories where (system-wide) .service files can live. /etc/systemd/system, /run/systemd/system, /usr/local/lib/systemd/system, and /usr/lib/systemd/system are four of those directories.

The compatibility with System V rc scripts is achieved with a conversion program, named systemd-sysv-generator. This program is listed in the /usr/lib/systemd/system-generators/ directory and is thus run automatically by systemd early in the bootstrap process at every boot, and again every time that systemd is instructed to re-load its configuration later on.

This program is a generator, a type of ancillary utility whose job is to create service unit files on the fly, in a tmpfs where three more of those nine directories (which are intended to be used only by generators) are located. systemd-sysv-generator generates the service units that run the System V rc scripts from /etc/init.d, if it doesn't find a native systemd service unit by that name already existing in the other six locations.

systemd service management only knows about service units. These automatically (re-)generated service units are written to invoke the system 5 rc scripts. They have, amongst other things:

[Unit]
SourcePath=/etc/init.d/wibble
[Service]
ExecStart=/etc/init.d/wibble start
ExecStop=/etc/init.d/wibble stop

Received wisdom is that the System V rc scripts must have an LSB header, and are run in parallel without honouring the priorities imposed by the /etc/rc?.d/ system. This is incorrect on all points.

In fact, they don't need to have an LSB header, and if they do not systemd-sysv-generator can recognize the more limited old RedHat comment headers (description:, pidfile:, and so forth). Moreover, in the absence of an LSB header it will fall back to the contents of the /etc/rc?.d symbolic link farms, reading the priorities encoded into the link names and constructing a before/after ordering from them, serializing the services. Not only are LSB headers not a requirement, and not only do they themselves encode before/after orderings that serialize things to an extent, the fallback behaviour in their complete absence is actually significantly non-parallelized operation.

The reason that /etc/rc3.d didn't appear to matter is that you probably had that script enabled via another /etc/rc?.d/ directory. systemd-sysv-generator translates being listed in any of /etc/rc2.d/, /etc/rc3.d/, and /etc/rc4.d/ into a native Wanted-By relationship to systemd's multi-user.target. Run levels are "obsolete" in the systemd world, and you can forget about them.

Further reading

  • 2
    In Debian the system-generators directory doesn't live on /usr/lib, but /lib packages.debian.org/sid/amd64/systemd/filelist – Braiam May 25 '16 at 13:55
  • 1
    This is a straight up amazing answer. Well done sir. – peelman Jan 13 '17 at 12:13
  • Thank you, thank you, thank you for this! Dealing with a mix of Debian 8 and RH/CentOS 7 systems has made management of SysVInit vs Systemd service dependency management a bit of a headache but this explanation of what systemd is doing has helped my understanding greatly. – Toby Mar 29 '17 at 17:22

Systemd is backward compatible with SysV init scripts. According to LSB 3.1, the init script must have informational Comment Conventions, defining when the script has to start/stop and what is required for the script to start/stop. This is an example:

### BEGIN INIT INFO
# Provides: my-service
# Required-Start: $local_fs $network $remote_fs
# Required-Stop: $local_fs $network $remote_fs
# Default-Start:  2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: start and stop service my-service
# Description: my-service blah blah ...
### END INIT INFO

This is a commented section which is ignored by SysV. On the other hand, systemd reads that dependency information and runs those scripts depending on that.

But there is one point, where systemd and SysV differ in terms of init scripts. SysV executes the scripts in sequential order based on their number in the filename. Systemd doesn't. If dependencies are met, systemd runs the scripts immediately, without honoring the numbering of the script names. Some of them will most probably fail because of the ordering. There are a lots of other incompatibilities that should be considered.


If there are init scripts and .service files for the same service, systemd will execute both, as soon as the dependencies are met (in case of the init script, those defined in the LSB header).

  • Okay, but I also have a whole bunch of .service files in /lib/systemd/system/. What does systemd actually execute? Whatever is specified in the service files (in dependency order), the init.d scripts or both? – Martin Drautzburg Oct 2 '15 at 13:46
  • @MartinDrautzburg I added that to the answer – chaos Oct 2 '15 at 13:58
  • 1
    As a sidenote, Debian has just announced to dump LSB compatibility: article.gmane.org/gmane.linux.debian.devel.lsb/1103 – Jan Oct 2 '15 at 14:22

FYI: What to do when SysD silently fails to start SysV process?

Yes, that happens. SysD shows System V process has been started with exit code 0 (success), still the process has failed to start correctly and service is actually down. The service will start correctly if you test running it on command line, i.e. no problem with configuration of that service. systemd-sysv-generator creates on the fly a file when starting something having SysV init. You can find the result at

/run/systemd/generator.late/

This example is related to "opendmark" service at Ubuntu. You can override for example ExecStart= to fix a startup problem. Run

sudo systemctl edit opendmarc.service    (<--use the actual service name)

That will open an editor. Type there whatever is needed to fix the thing:

[unit]
# with an override you must "clear" the ExecStart first.
ExecStart=
ExecStart=/usr/sbin/opendmarc -p localhost:54321 -u opendmarc -P /var/run/opendmarc/opendmarc.pid

In the example above I needed to start opendmarc as inet socket at localhost:54321 (instead of Unix socket).

Credits and useful to read:

https://p5r.uk/blog/2017/dmarc-with-postfix.html

https://www.turnkeylinux.org/blog/debugging-systemd-sysv-init-compat

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.