7

I'm having troubles getting a systemd service unit file to run correctly when the server reboots. This server is a fresh CentOS 7.0.1406 install. I'm trying to set it up with multiple Apache httpd instances. On an older CentOS install I had the httpd instances working fine and so now I'm trying to port those configurations over to the new CentOS which uses systemd.

I can enable the service, start it, restart it and stop it without any issues. When I do a reboot it says "No such file or directory". If I reenable the service, everything works again until the next reboot.



Service file:

[Unit]
Description=Apache web server instance apache01
After=network.target remote-fs.target nss-lookup.target

[Service]
Type=forking
PIDFile=/data/apacheinstances/apache01/logs/httpd.pid
ExecStart=/data/apacheinstances/apache01/bin/apachectl start
ExecStop=/data/apacheinstances/apache01/bin/apachectl stop
ExecReload=/data/apacheinstances/apache01/bin/apachectl reload
PrivateTmp=true
LimitNOFILE=infinity

[Install]
WantedBy=multi-user.target



How I enabled it:

$ sudo systemctl enable /data/apacheinstances/apache01/bin/apache01.service
ln -s '/data/apacheinstances/apache01/bin/apache01.service' '/etc/systemd/system/multi-user.target.wants/apache01.service'
ln -s '/data/apacheinstances/apache01/bin/apache01.service' '/etc/systemd/system/apache01.service'


$ sudo systemctl status apache01
apache01.service - Apache web server instance apache01
   Loaded: loaded (/data/apacheinstances/apache01/bin/apache01.service; enabled)
   Active: inactive (dead)



How I started it:

$ sudo systemctl start apache01


$ sudo systemctl status apache01
apache01.service - Apache web server instance apache01
   Loaded: loaded (/data/apacheinstances/apache01/bin/apache01.service; enabled)
   Active: active (running) since Wed 2014-10-08 14:37:56 PDT; 13s ago
  Process: 1740 ExecStart=/data/apacheinstances/apache01/bin/apachectl start (code=exited, status=0/SUCCESS)
 Main PID: 1746 (httpd)
   CGroup: /system.slice/apache01.service
           +-1746 /usr/local/httpd-2.4.10-apache01/bin/httpd -f /data/apacheinstances/apache01/conf/httpd.conf -k start
           +-1747 /usr/local/sbin/cronolog /data/apacheinstances/apache01/logs/error_log-%Y%m
           +-1748 /usr/local/sbin/cronolog /data/apacheinstances/apache01/logs/access_log-%Y%m
           +-1749 /usr/local/httpd-2.4.10-apache01/bin/httpd -f /data/apacheinstances/apache01/conf/httpd.conf -k start
           +-1750 /usr/local/httpd-2.4.10-apache01/bin/httpd -f /data/apacheinstances/apache01/conf/httpd.conf -k start
           +-1751 /usr/local/httpd-2.4.10-apache01/bin/httpd -f /data/apacheinstances/apache01/conf/httpd.conf -k start

Oct 08 14:37:56 server105 apachectl[1740]: Starting apache01: [  OK  ]
Oct 08 14:37:56 server105 systemd[1]: PID file /data/apacheinstances/apache01/logs/httpd.pid not readable (yet?) after start.
Oct 08 14:37:56 server105 systemd[1]: Started Apache web server instance apache01.



How I restarted it:

$ sudo systemctl restart apache01


$ sudo systemctl status apache01
apache01.service - Apache web server instance apache01
   Loaded: loaded (/data/apacheinstances/apache01/bin/apache01.service; enabled)
   Active: active (running) since Wed 2014-10-08 14:38:40 PDT; 12s ago
  Process: 1836 ExecStop=/data/apacheinstances/apache01/bin/apachectl stop (code=exited, status=0/SUCCESS)
  Process: 1844 ExecStart=/data/apacheinstances/apache01/bin/apachectl start (code=exited, status=0/SUCCESS)
 Main PID: 1850 (httpd)
   CGroup: /system.slice/apache01.service
           +-1850 /usr/local/httpd-2.4.10-apache01/bin/httpd -f /data/apacheinstances/apache01/conf/httpd.conf -k start
           +-1851 /usr/local/sbin/cronolog /data/apacheinstances/apache01/logs/error_log-%Y%m
           +-1852 /usr/local/sbin/cronolog /data/apacheinstances/apache01/logs/access_log-%Y%m
           +-1853 /usr/local/httpd-2.4.10-apache01/bin/httpd -f /data/apacheinstances/apache01/conf/httpd.conf -k start
           +-1855 /usr/local/httpd-2.4.10-apache01/bin/httpd -f /data/apacheinstances/apache01/conf/httpd.conf -k start
           +-1856 /usr/local/httpd-2.4.10-apache01/bin/httpd -f /data/apacheinstances/apache01/conf/httpd.conf -k start

Oct 08 14:38:40 server105 apachectl[1844]: Starting apache01: [  OK  ]
Oct 08 14:38:40 server105 systemd[1]: PID file /data/apacheinstances/apache01/logs/httpd.pid not readable (yet?) after start.
Oct 08 14:38:40 server105 systemd[1]: Started Apache web server instance apache01.



How I stopped it:

$ sudo systemctl stop apache01


$ sudo systemctl status apache01
apache01.service - Apache web server instance apache01
   Loaded: loaded (/data/apacheinstances/apache01/bin/apache01.service; enabled)
   Active: inactive (dead) since Wed 2014-10-08 14:39:44 PDT; 12s ago
  Process: 1940 ExecStop=/data/apacheinstances/apache01/bin/apachectl stop (code=exited, status=0/SUCCESS)
  Process: 1844 ExecStart=/data/apacheinstances/apache01/bin/apachectl start (code=exited, status=0/SUCCESS)
 Main PID: 1850 (code=exited, status=0/SUCCESS)

Oct 08 14:38:40 server105 apachectl[1844]: Starting apache01: [  OK  ]
Oct 08 14:38:40 server105 systemd[1]: PID file /data/apacheinstances/apache01/logs/httpd.pid not readable (yet?) after start.
Oct 08 14:38:40 server105 systemd[1]: Started Apache web server instance apache01.
Oct 08 14:39:44 server105 systemd[1]: Stopping Apache web server instance apache01...
Oct 08 14:39:44 server105 apachectl[1940]: Stopping apache01: [  OK  ]
Oct 08 14:39:44 server105 systemd[1]: Stopped Apache web server instance apache01.



Status after reboot:

$ sudo systemctl status apache01
apache01.service
   Loaded: not-found (Reason: No such file or directory)
   Active: inactive (dead)

Oct 08 14:44:58 server105 systemd[1]: Cannot add dependency job for unit apache01.service, ignoring: Unit apache01.service failed to load: No such file or directory.



How I reenable and start it:

$ sudo systemctl reenable /data/apacheinstances/apache01/bin/apache01.service
rm '/etc/systemd/system/apache01.service'
rm '/etc/systemd/system/multi-user.target.wants/apache01.service'
ln -s '/data/apacheinstances/apache01/bin/apache01.service' '/etc/systemd/system/multi-user.target.wants/apache01.service'
ln -s '/data/apacheinstances/apache01/bin/apache01.service' '/etc/systemd/system/apache01.service'


$ sudo systemctl start apache01


$ sudo systemctl status apache01
apache01.service - Apache web server instance apache01
   Loaded: loaded (/data/apacheinstances/apache01/bin/apache01.service; enabled)
   Active: active (running) since Wed 2014-10-08 14:52:20 PDT; 5s ago
  Process: 1737 ExecStart=/data/apacheinstances/apache01/bin/apachectl start (code=exited, status=0/SUCCESS)
 Main PID: 1743 (httpd)
   CGroup: /system.slice/apache01.service
           +-1743 /usr/local/httpd-2.4.10-apache01/bin/httpd -f /data/apacheinstances/apache01/conf/httpd.conf -k start
           +-1744 /usr/local/sbin/cronolog /data/apacheinstances/apache01/logs/error_log-%Y%m
           +-1745 /usr/local/sbin/cronolog /data/apacheinstances/apache01/logs/access_log-%Y%m
           +-1746 /usr/local/httpd-2.4.10-apache01/bin/httpd -f /data/apacheinstances/apache01/conf/httpd.conf -k start
           +-1747 /usr/local/httpd-2.4.10-apache01/bin/httpd -f /data/apacheinstances/apache01/conf/httpd.conf -k start
           +-1748 /usr/local/httpd-2.4.10-apache01/bin/httpd -f /data/apacheinstances/apache01/conf/httpd.conf -k start

Oct 08 14:52:20 server105 apachectl[1737]: Starting apache01: [  OK  ]
Oct 08 14:52:20 server105 systemd[1]: PID file /data/apacheinstances/apache01/logs/httpd.pid not readable (yet?) after start.
Oct 08 14:52:20 server105 systemd[1]: Started Apache web server instance apache01.

I'm not sure why it work without issue before a reboot but fail afterwards. If I don't do the reenable I continue to get the same error about "No such file or directory".

I've tried to research logging functionality within systemd to determine which file or directory it cannot find but have drawn a blank.

Has anyone else experienced this kind of behaviour before?

Improve this question
3
  • 1
    Try moving the service to /etc/systemd/system/ and see if the problem still persists.
    – remmy
    Oct 8, 2014 at 22:32
  • @kyrias that worked. One thing I forgot to mention was that /data is mounted on a different drive. I thought having the "After=remote-fs.target" would make sure the /data drive was mounted before the service was started (assuming that is the reason for the "No such file or directory"). Are the unit files required to be in the /etc/systemd/system directory? If not, I would still prefer the files to be with the rest of the httpd instance configurations just to make backup and documentation a little easier.
    – Richard Morris
    Oct 9, 2014 at 1:24
  • systemd will need to read the .service file in order to parse After=remote-fs.target. systemd starts early in the boot process and reads all the .service files (or symlinks to these files). At this point in time, your .service file is on an unmounted remote filesystem therefore systemd complains that it can't find it when it tries to follow the symlink in /etc/systemd. It hasn't even got to the After statement yet. The bottom line is that all .service files need to be on a filesystem that's accessible very early in the boot process.
    – garethTheRed
    Oct 9, 2014 at 9:01

1 Answer 1

Reset to default
9

All services need to be loaded and parsed before starting them, and your service isn't available early on in the boot process when systemd parses all services since it's on a different partition.

To solve it either keep the local services in /etc/systemd/system/ or make a package for your distribution and place them in /usr/lib/systemd/system/

Improve this answer
1
  • oh god this was my problem too, finally understood it after many hours
    – peedee
    Sep 29, 2016 at 6:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.