smartd only send email on attribute change

Question

I setup smartd to send me mail reports in case my disks fail. Unfortunately I am being spammed each day by the same attribute with the same (not changing) value:

The following warning/error was logged by the smartd daemon:

Device: /dev/sdb [SAT], 1 Offline uncorrectable sectors

Device info:
ST32000542AS, S/N:XXXXX, WWN:5-XXXXXX-XXXXXXXX, FW:XXXX, 2.00 TB

I am fully aware of the dangers implied with a disk with uncorrectable sector(s). (This disk is used in a RAID10 configuration)

I just do not want to receive the same email every day, I just want to receive an email if a value changes / increases.

This is the current configuration of my /etc/smartd.conf:

DEVICESCAN -d removable -n standby -t -m root -M exec /usr/share/smartmontools/smartd-runner

What do I have to change in order to achieve the desired behavior of smartd?

Answer 1

The default behaviour when state persistence is enabled, which I think is the default in Debian, is to send an email daily when a critical event is logged (or rather, when a critical event is logged, if no corresponding email has been sent in at least a day, an email is sent). You can change this behaviour using the -M option; add

-M once

to your smartd.conf, and critical events will only result in a single email, with no repeats.

(The -M once option is in addition to the -M exec option.)

To check whether state persistence really is enabled, check the contents of /var/lib/smartmontools; you should see state files for all your drives, updated recently.

Answer 2

You're using -M exec and the smartd-runner which is apparently a Debian specialty.

Package Maintainers and system administrators can put scripts to be run
when smartd detects an error into /etc/smartmontools/run.d. These
scripts will be run by smartd-runner using run-parts(8). The script will
receive the filename of the file containing the errormessage as first
parameter. See /etc/smartmontools/run.d/10mail for an example.

The main purpose of this runner seems to be making sending mail optional (depending on whether a mailer is installed in the first place) and separately, it seems to trigger a pop-up notification on a desktop (if the desktop notifier is installed).

So I guess you could modify that 10mail script to filter out duplicate mails.

The alternative would be running smartd with the --savestates option, and then use -M once in the smartd.conf - this is described in more detail in the smartd.conf manpage, to summarize:

          once - send only one warning email for each type of disk problem

          daily - send additional warning reminder emails, once  per  day,

          diminishing - send additional warning reminder emails,  after  a
          one-day  interval,  then  a  two-day  interval,  then a four-day

          test - send a single test email immediately upon smartd startup.

          exec PATH - run the executable PATH instead of the default  mail

And if that doesn't work, you'll either have to - live with it, or write your own mail handler exec logic that filters accordingly after all.

---

I am fully aware of the dangers implied with a disk with uncorrectable sector(s). (This disk is used in a RAID10 configuration)

I still recommend replacing such drives immediately. Whatever promises RAID makes regarding redundancy is only valid as long as each drive is in 100% working order.

Once the drive is replaced you can do a full write test on the removed drive without risking data loss - and depending how the drive fares in this test you can make a much more informed decision whether to keep using that drive (preferably outside of raid) or not.

You don't know how broken your drive really is until you put it to the test - and your smartd.conf doesn't look like you're running regular self-tests, so errors may stay undetected for a long time. And this is how RAIDs die during rebuild.

Answer 3

This is an old thread, but I'll post what I've found in the manual of smartd:

-U ID[+]

          [ATA only] Report if the number of offline uncorrectable sectors
          is  non-zero. Here ID is the id number of the Attribute whose
          raw value  is  the  Offline  Uncorrectable  Sector  count.   The
          allowed  range  of  ID  is 0 to 255 inclusive.  To turn off this
          reporting, use ID = 0.  If the -U ID option is not  given,  then
          it  defaults to -U 198 (since Attribute 198 is generally used to
          monitor offline uncorrectable sectors).  If  the  name  of  this
          Attribute  is  changed  by  a  '-v  198,FORMAT,NAME' (except '-v
          198,FORMAT,Offline_Scan_UNC_SectCt'), directive, the default  is
          changed to -U 0.

          If  '+'  is specified, a report is only printed if the number of
          sectors has increased since the last check cycle. **Some disks  do
          not  reset this attribute when a bad sector is reallocated.**  See
          also '-v 198,increasing' below.

So adding an option -U 198+ should do exactly what you want.