That all depends on what you mean by "goes down", the details of what you are running and doing do tend to matter when monitoring as well....
The most thorough form of monitoring for your "service" is to have an external automated system do what your clients do and report to you when anything unexpected occurs.
From the brief description of your email service, my first end to end test would be:
- Send an email
#id via SMTP to a local
- Wait the maximum amount of seconds you want that to take.
- Check for email
monitor on the POP server.
That one check from an external monitoring host is going to pick up about 99% of the problems that could occur in a simple email system.
These service or transaction monitors tend be custom scripts written in something like Ruby, Python or Perl which have modules to easily implement things like SMTP or POP programatically. The scripts then generally plug into a monitoring solution but even triggering a simple email or SMS via a gateway from a cron job would suffice if you want something simple. If you pay for a monitoring solution you will generally get their attempt at a GUI designer for the same type of monitor.
Of course in the real world this quickly becomes more complicated. You might offer secure ports for both POP and SMTP requiring another check. Maybe IMAP is added in with a bit of Carddav and Caldav, you might have services on multiple hosts.
What an overall service check like the above won't readily tell you, is where the problem lies, just that there is a problem somewhere.
Lower level monitoring
By monitoring individual components of the you service you basically make it easier to identify (or predict) where an issue lies from monitoring before doing any leg work. This type of component monitoring is what systems like Nagios, Zabbix or big ones like Tivoli Monitoring are good at.
This can be an ever expanding tree of things, depending on how detailed you make it and again, how complex the system is that supports your "service"
"Your mail Service" depends on
Services: POP:110 SMTP:25
OS: linux Z
Host: server Y
Components: diskA diskB cpu1 cpu2 memory
Ntwork: ethernetA, Switch B, Router C, Firewall X
Each component has metrics or a state you can report on.
POP service - Are we accepting connections on 110,995
SMTP service - Are we accepting connections on 25,587
devmail process(es) (is it running, memory, cpu, handles, io)
JMX parameters of the java process (memory, threads, performance, garbage collection)
Disk, Memory, Cpu, IO
What if the monitoring host goes down? Or it's just the network between the service and monitor.
It's generally good to run the service checks from two (or more) external hosts which are as close to where the clients are coming from (without impacting your monitoring). Then also run the checks locally on the host or at least from local network. This way you get a better idea of most network based problems.
- If one external client fails, probably an external network.
- If local client is working, but all external clients are failing, probably local network.
- If all clients are failing, probably local issue.
I see a lot of people tend to build their monitoring solutions the wrong way around. They come up with a lot of the lower level system metrics and 1000's of monitors and level's they think are appropriate to alarm at, when none if that really matters. I mean they're nice for analysis and capacity management, you can make some swanky graphs out of all those values and they can be extremely useful but don't really mean much when you missed metric x at level y that means no one can receive emails.