linux watchdog and systemd watchdog

Question

Any way to register application with systemd watchdog at runtime ? I mean don't use systemd unit file, via systemd API for example

Linux watchdog is used for system reset only ? Can it be used for application reset ?

Answer 1

Systemd's watchdog can be mainly used for 3 different actions:

• hardware reset (leveraging the CPU hardware watchdog exposed at /dev/watchdog). This is enabled by the RuntimeWatchdogSec= option in /etc/systemd/system.conf
• application reset, as long as this is foreseen in the systemd unit definition
• system reset as a fallback measure in response to multiple unsuccessful application resets. Also defined in the systemd unit

example unit file:

[Unit]
Description=My Little Daemon
Documentation=man:mylittled(8)

[Service]
ExecStart=/usr/bin/mylittled
WatchdogSec=30s
Restart=on-failure
StartLimitInterval=5min
StartLimitBurst=4
StartLimitAction=reboot-force

The example is taken from: http://0pointer.de/blog/projects/watchdog.html, which gives a pretty complete overview of what and how you can use the watchdog service.

Answer 2

The Linux watchdog daemon should be used for system reset jobs, though it can also run a "repair binary" on persistent errors that could be used to fix or restart a process. Generally speaking, to monitor daemon processes and restart them you should use the init/upstart/systemd supported methods as already answered and keep the watchdog operation for the most serious "only a reboot is likely to fix things" situations.

Answer 3

You have multiple questions or expectations in your question.

System watchdog

When it comes to the Linux service watchdog it is intended to detect system (OS, Linux) issues (broken, frozen, ...) and it works in the following way:

1. there is a special service (supported with hardware, ideally) that is "telling" basically: "All is ok, I am alive!"
2. it tells it constantly, periodically
3. there is an observer (hardware ideally) which, if not informed for a longer than expected period, will decide that the system became unresponsive and will reboot the system (by software or hardware reboot)
4. hopefully if it was a specific freezing with a new reboot the system would recover
5. if something is seriously wrong the system will again trigger the watchdog and the system will repeat rebooting

Application watchdog

On the other hand, there is an interest in watching an application/service health.

1. for example you might want to be sure if the application dies you would like to restart it (for that you should start it as a service, ideally through systemd mechanism) or
2. if you want to be sure that the app is not frozen you want to have an external watchdog service to which your app would report "All good, I am alive". One of these app-supporting watchdogs is monit