I'm looking for ideas on how to automatically receive an alert (by email would be nice) if a specific periodic email were not to arrive. I would not necessarily have access to the system that generates the email.
In a recent real world use case I have a summary report that is sent daily to a half dozen or so technicians and managers giving system status of dozens of field locations. We have a second channel that sends alarms, but there are many maintenance issues that can only be discovered by reviewing this report.
Last night I found that the script generating the report had stopped working weeks ago - and no one noticed. Please, before anyone spends much time berating the people who failed to notice (and this would include me) understand that 90% to 100% of the people involved are very good at what they do, and really do care, but are on the border of information overload. I think it is just human nature not to notice when one of dozens of emails that aren't normally critical just happens not to arrive. When it arrives that does seems to trigger us to review it with good results, but no arrival - no mental trigger. We notice and act on emails but fail to notice and act on lack of emails.
My current thought is to send any report emails that I want to monitor to additional "watchdog" email addresses. I would then have independent machines - maybe a couple of Raspberry Pis or VM in the cloud receive the emails and send an alert if too much time goes by without receiving the email. Sort of a watchdog timer for emails.
Maybe something like this already exists and I just don't have the right search terms and someone can point me to the information, but I'd also be interested in any suggestions on how others might approach it. For now I'm heading in a specialized script direction, but would be interested in making it more general purpose.