3

I've recently set Debian 9 up on a home server and I would like to run some S.M.A.R.T checks against my HDD's and then get alerted if there are any issues. Ideally I'd like the alerts to come into my Slack instance as I have that connected to my smartphone and find it very useful (I already have alerts for my UPS and failed pings coming into there).

I've been looking into smartd/smartctl, and I can't seem to find a way to get the notifications into Slack.

I was hoping it would be possible to call some bash script somewhere to process a notification and call another (python) script that sends the notification to Slack.

(edit: just to clarify I already have the Python script for Slack Notifications as I'm using that elsewhere so we're good there)

Edit Both the below solutions were tested and worked well for me. I opted for the mail + script solution as it covered me in the event of Slack not working for whatever reason, but both were solid and I thank RalfFriedl for including the environmental variables for me.

2
  • So what exactly is the question? smartd is a good choice and the default setting are sufficient for many use cases, just add your email address for notifications. You can also write a script around smartctl. – RalfFriedl Jan 27 '19 at 16:53
  • @RalfFriedl how to get smartd to run the script. – sourcejedi Jan 27 '19 at 17:07
3
# /etc/smartd.conf
/dev/sda -a -m <nomailer> -M exec /my/script

This is for sda. Add one line per device. Or, replace /dev/sda with DEVICESCAN to monitor all detected devices.

Make sure that /my/script does not take too long, as it will hang smartd until it is finished.

/my/script will be run with various environment variables. Search man smartd.conf for SMARTD_DEVICE. For example another variable: "SMARTD_FULLMESSAGE is set to the contents of the entire email warning message string from smartd."

To test the messaging, start smartd using this temporary configuration file, to send a test message:

# /etc/smartd.conf
/dev/sda -a -m <nomailer> -M exec /my/script -M test

I have not tested any of the above. I assume you have already worked out if you need to include a specific -d option by testing using smartctl. (I do not know how -d and DEVICESCAN interact). I also assume that the default tests from -a match what you need for your device. For example, one of the things mentioned in the long man page is that some drives will show false positives after a bad sector has been re-assigned: https://superuser.com/a/539040/110495

3
  • Thank-you! It seems so simple now you've broken it down for me. Yes I've run smartctl and know what devices I would like to monitor. I'll be using specific drives as my NVME drives make smartctl sad :( One (potentially stupid) question SMARTD_FULLMESSAGE and other such commands, will these basically pass information onto my script? – Steve Brown Jan 27 '19 at 20:04
  • @SteveBrown environment variable. Where you would have written $1 to use the positional argument 1, write $SMARTD_FULLMESSAGE to use the env var. Note, as per the man page, you probably want to write "$SMARTD_FULLMESSAGE" i.e. put it in quotes. Otherwise sh will treat each word in the message as a separate parameter. – sourcejedi Jan 27 '19 at 20:08
  • That is absolutely brilliant. Thank you very much for your help! – Steve Brown Jan 27 '19 at 20:16
0

If you want smartd to call your script, see the option -w:

-w PATH, --warnexec=PATH

Run the executable PATH instead of the default script when smartd needs to send warning messages. PATH must point to an executable binary file or script. The default script is /etc/smartd_warning.sh.

The script is normally used to send an email. It is called with these environment variables:

SMARTD_MAILER
is set to the argument of -M exec, if present or else to 'mail' (examples: /usr/local/bin/mail, mail).

SMARTD_DEVICE
is set to the device path (example: /dev/sda).

SMARTD_DEVICETYPE
is set to the device type specified by '-d' directive or 'auto' if none.

SMARTD_DEVICESTRING
is set to the device description. It starts with SMARTD_DEVICE and may be followed by an optional controller identification (example: /dev/sda [SAT]). The string may contain a space and is NOT quoted.

SMARTD_DEVICEINFO
is set to device identify information. It includes most of the info printed by smartctl -i but uses a brief single line format. This device info is also logged when smartd starts up. The string contains space characters and is NOT quoted.

SMARTD_FAILTYPE
gives the reason for the warning or message email. The possible values that it takes and their meanings are:
EmailTest: this is an email test message.
Health: the SMART health status indicates imminent failure.
Usage: a usage Attribute has failed.
SelfTest: the number of self-test failures has increased.
ErrorCount: the number of errors in the ATA error log has increased.
CurrentPendingSector: one of more disk sectors could not be read and are marked to be reallocated (replaced with spare sectors).
OfflineUncorrectableSector: during off-line testing, or self-testing, one or more disk sectors could not be read.
Temperature: Temperature reached critical limit (see -W directive).
FailedHealthCheck: the SMART health status command failed.
FailedReadSmartData: the command to read SMART Attribute data failed.
FailedReadSmartErrorLog: the command to read the SMART error log failed.
FailedReadSmartSelfTestLog: the command to read the SMART self-test log failed.
FailedOpenDevice: the open() command to the device failed.

SMARTD_ADDRESS
is determined by the address argument ADD of the '-m' Directive. If ADD is , then SMARTD_ADDRESS is not set. Otherwise, it is set to the comma-separated-list of email addresses given by the argument ADD, with the commas replaced by spaces (example:admin@example.com root). If more than one email address is given, then this string will contain space characters and is NOT quoted, so to use it in a shell script you may want to enclose it in double quotes.

SMARTD_MESSAGE
is set to the one sentence summary warning email message string from smartd. This message string contains space characters and is NOT quoted. So to use $SMARTD_MESSAGE in a shell script you should probably enclose it in double quotes.

SMARTD_FULLMESSAGE
is set to the contents of the entire email warning message string from smartd. This message string contains space and return characters and is NOT quoted. So to use $SMARTD_FULLMESSAGE in a shell script you should probably enclose it in double quotes.

SMARTD_TFIRST
is a text string giving the time and date at which the first problem of this type was reported. This text string contains space characters and no newlines, and is NOT quoted. For example: Sun Feb 9 14:58:19 2003 CST

SMARTD_TFIRSTEPOCH
is an integer, which is the unix epoch (number of seconds since Jan 1, 1970) for SMARTD_TFIRST.

SMARTD_PREVCNT
is an integer specifying the number of previous messages sent. It is set to '0' for the first message.

SMARTD_NEXTDAYS
is an integer specifying the number of days until the next message will be sent. It it set to empty on '-M once' and set to '1' on '-M daily'.

For more details see man smartd and man smartd.conf.

3
  • Thank you! That seems very simple. One (potentially stupid) question: what exactly is passed to that script? (as in what would the script need to pull in from the command line args etc? Or is there some documentation I can read? :) – Steve Brown Jan 27 '19 at 20:06
  • The script gets information in environment variables, I included the description. See the manual pages for more information. – RalfFriedl Jan 28 '19 at 6:23
  • If it is what you were looking for, you could accept the answer. – RalfFriedl Jan 29 '19 at 6:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.