We encountered the following problem in our company. We have multiple Red Hat Enterprise Linux Servers on which "SAP HANA S/4" is running. We created a systemd Service to automatically start and stop the daemon so that we don't need to manually interact with the system on a reboot or shutdown.
Autostart works well, but there seems to be a problem with stopping the daemon correctly on shutdown. The daemons are running with another user (individual for each server). It seems that systemd starts to kill the user sessions before the actual service is stopped; as a result, the service won't stop properly.
[Unit] Description=saphana After=remote-fs.target user.slice sapinit.service multi-user.target Requires=user.slice [Service] KillMode=none Type=oneshot ExecStart=/hana/source/scripts/sapHanaControl.pl start ExecStop=/hana/source/scripts/sapHanaControl.pl stop RemainAfterExit=yes [Install] WantedBy=multi-user.target
The script wich is called in ExecStart and ExecStop basically executes the following command.
"sudo -u $username csh -c "sapcontrol -nr $instance -function Start"
On Stop: "sudo -u $username csh -c "sapcontrol -nr $instance -function Stop"
Output of the Systemd Log shows the following:
Jun 20 16:23:05 host123 systemd: Stopping Session c4 of user **userxy**. Jun 20 16:23:05 host123sapHanaControl.pl: sudo -u **userxy** csh -c "sapcontrol -nr 00 -function Stop" Jun 20 16:23:05 host123 sapHanaControl.pl: 20.06.2018 16:23:05 Jun 20 16:23:05 host123 sapHanaControl.pl: Stop Jun 20 16:23:05 host123 sapHanaControl.pl: FAIL: NIECONN_REFUSED (Connection refused), NiRawConnect failed in plugin_fopen()
I see the following processes running when the system is running normally:
[root@wsstadt325 ~]# ps -ef | grep sapstartsrv d61adm 1740 1 0 11:56 ? 00:00:01 /usr/sap/D61/HDB05/exe/sapstartsrv pf=/usr/sap/D61/SYS/profile/D61_HDB05_wsstadt325 -D -u d61adm sapadm 1741 1 0 11:56 ? 00:00:04 /usr/sap/hostctrl/exe/sapstartsrv pf=/usr/sap/hostctrl/exe/host_profile -D d21adm 1946 1 0 11:56 ? 00:00:02 /usr/sap/D21/ASCS01/exe/sapstartsrv pf=/usr/sap/D21/SYS/profile/D21_ASCS01_wsstadt325 -D -u d21adm d21adm 2182 1 0 11:56 ? 00:00:02 /usr/sap/D21/D00/exe/sapstartsrv pf=/usr/sap/D21/SYS/profile/D21_D00_wsstadt325 -D -u d21adm`
Chnaged my script to log the "ps -ef | grep sapstartsrv" output when the system gets rebooted / powered off.
ps -ef | grep sapstartsrv sapadm 1683 1 0 13:52 ? 00:00:01 /usr/sap/hostctrl/exe/sapstartsrv pf=/usr/sap/hostctrl/exe/host_profile -D root 5706 5522 0 14:00 ? 00:00:00 sh -c ps -ef | grep sapstartsrv root 5708 5706 0 14:00 ? 00:00:00 grep sapstartsrv
The sapstartsrv Service is started by a default SAP Service (sapinit) wich gets startet before my own Systemd Service (So on a reboot it is the reversed order [Stop my Systemd Service -> stop the Sapinit Service]) The problem seems to be that systemctl starts to kill the user session (In my case for the user: d21adm and d61adm) where the sapstartsrv process is running before my actual Systemd service is stopped. (Hope that makes at least a bit sense)
Here's an image of the whole systemd chain (my services are at the very end): The services involved: - sapinit.service (The default one) - saphana.service (My custom one)