Are there any tools available to store SMART data over time?

Question

I'd like to start storing the SMART data over time and see any trends based on disk ID/serial number. Something that would let me, for example just get the smart information from disks once a day and put it in a database. Is there already a tool for this in Linux, or do I have to roll my own?

Answer 1

There are already tools which can do this, often as part of a more general monitoring tool. One I find useful is Munin, which has a SMART plugin to trace the available attributes:

Munin is available in many distributions.

smartmontools itself contains a tool which can log attributes periodically, smartd. You might find that that’s all you need.

Answer 2

It is easy to "roll your own". Run smartctl -A drive-specifier (as root) daily, via an AWK script, with output to a file.

gnuplot is good for drawing graphs of this file.

A slight expansion on this, with an example:-

1. Place an entry to run the following script in /etc/cron.daily

#!/bin/sh

# SMART DISK PROCESSING
# =====================
tmpfile=$(mktemp -q)
today=$(date -u +%d-%m-%Y)

smartctl -A /dev/disk/by-id/ata-Samsung_SSD_870_QVO_1TB_S5SVNG0NB22319L > $tmpfile

# Output log as a single line - note "Unknown_Attribute" is "POR_Recovery_Count" [unexpected shutdown]
echo -n $today ', ' >> /var/log/disk-monitor.d/sdb-errors.csv
awk 'NR>=8 && NR<=21 {print $1,",",$2,",",$10,",";}' $tmpfile | tr -d '\n' | sed 's/Unknown_Attribute/POR_Recovery_Count/;s/\,$/\n/' >> /var/log/disk-monitor.d/sdb-errors.csv

exit 0

The AWK "NR>=8 && NR<=21" picks out the correct line numbers, the print statement picks out the appropriate columns; the tr removes new-lines; sed fixes a SMART attribute problem and adds a single newline.

So that on each day one record is written to the CSV log file in a date, [attribute-id, attribute-name, attribute-value]*N format.

07-06-2021 , 5 , Reallocated_Sector_Ct , 0 ,9 , Power_On_Hours , 2900 , ...

2. I choose to plot selected values [the ones that would ideally be zero] on demand... The script I use for gnuplot script-name is as follows,

set title "SDA Errors which should be ZERO"
set xdata time
set timefmt "%d-%m-%Y"
set format x "%d/%m"
set datafile separator ","

set colorsequence default

set ytics 2 nomirror tc lt 2
set ylabel 'POR' tc lt 2
set yrange [0:30<*]  
set y2tics 1 nomirror tc lt 1
set y2label 'Errors' tc lt 1
set y2range [-1:10]

set key left top 

set grid ytics lt 0 lw 1 lc rgb "#bbbbbb"
set grid xtics lt 0 lw 1 lc rgb "#bbbbbb"
plot "/var/log/disk-monitor.d/sda-errors.csv" using 1:4 title "Reallocated Sector Count" with lines axes x1y2, '' using 1:13 title "Wear Levelling Count" with lines  axes x1y2, '' using 1:16 title "Used Rsvd Blk Cnt Total" with lines  axes x1y2, '' using 1:19 title "Program Fail Cnt Total" with lines  axes x1y2, '' using 1:22 title "Erase Fail Count Total" with lines  axes x1y2, '' using 1:25 title "Runtime Bad Block" with lines   axes x1y2, '' using 1:28 title "Reported Uncorrect" with lines  axes x1y2, '' using 1:34 title "Hardware ECC Recovered" with lines dt 3  axes x1y2, '' using 1:40 title "POR Recovery Count" with lines dt 1 linetype rgb "green" axes x1y1

pause -1 "Hit any key to continue"

I'm sure better scripts are available!

Answer 3

If you're looking for something a little more advanced, my tools of choice for this are Telegraf + InfluxDB + Grafana.

• Telegraf is a monitoring agent that can pull stats from hundreds of sources, including SMART data
• InfluxDB is a time-series database, optimized for storing things that are measured over time (ie exactly this!)
• Grafana connects to Influx to display graphs and set up alerting

There's obviously more to setup and maintain, but there's a lot more power and flexibility. eg you can have multiple devices sending their stats to one Influx server.

With everything installed, you can then set up a dashboard: