7

I'd like to start storing the SMART data over time and see any trends based on disk ID/serial number. Something that would let me, for example just get the smart information from disks once a day and put it in a database. Is there already a tool for this in Linux, or do I have to roll my own?

10

There are already tools which can do this, often as part of a more general monitoring tool. One I find useful is Munin, which has a SMART plugin to trace the available attributes:

Graph showing SMART attributes over time

Munin is available in many distributions.

smartmontools itself contains a tool which can log attributes periodically, smartd. You might find that that’s all you need.

7

It is easy to "roll your own". Run smartctl -A drive-specifier (as root) daily, via an AWK script, with output to a file.

gnuplot is good for drawing graphs of this file.

A slight expansion on this, with an example:-

  1. Place an entry to run the following script in /etc/cron.daily
#!/bin/sh

# SMART DISK PROCESSING
# =====================
tmpfile=$(mktemp -q)
today=$(date -u +%d-%m-%Y)

smartctl -A /dev/disk/by-id/ata-Samsung_SSD_870_QVO_1TB_S5SVNG0NB22319L > $tmpfile

# Output log as a single line - note "Unknown_Attribute" is "POR_Recovery_Count" [unexpected shutdown]
echo -n $today ', ' >> /var/log/disk-monitor.d/sdb-errors.csv
awk 'NR>=8 && NR<=21 {print $1,",",$2,",",$10,",";}' $tmpfile | tr -d '\n' | sed 's/Unknown_Attribute/POR_Recovery_Count/;s/\,$/\n/' >> /var/log/disk-monitor.d/sdb-errors.csv

exit 0

The AWK "NR>=8 && NR<=21" picks out the correct line numbers, the print statement picks out the appropriate columns; the tr removes new-lines; sed fixes a SMART attribute problem and adds a single newline.

So that on each day one record is written to the CSV log file in a date, [attribute-id, attribute-name, attribute-value]*N format.

07-06-2021 , 5 , Reallocated_Sector_Ct , 0 ,9 , Power_On_Hours , 2900 , ...

  1. I choose to plot selected values [the ones that would ideally be zero] on demand... The script I use for gnuplot script-name is as follows,
set title "SDA Errors which should be ZERO"
set xdata time
set timefmt "%d-%m-%Y"
set format x "%d/%m"
set datafile separator ","

set colorsequence default

set ytics 2 nomirror tc lt 2
set ylabel 'POR' tc lt 2
set yrange [0:30<*]  
set y2tics 1 nomirror tc lt 1
set y2label 'Errors' tc lt 1
set y2range [-1:10]

set key left top 

set grid ytics lt 0 lw 1 lc rgb "#bbbbbb"
set grid xtics lt 0 lw 1 lc rgb "#bbbbbb"
plot "/var/log/disk-monitor.d/sda-errors.csv" using 1:4 title "Reallocated Sector Count" with lines axes x1y2, '' using 1:13 title "Wear Levelling Count" with lines  axes x1y2, '' using 1:16 title "Used Rsvd Blk Cnt Total" with lines  axes x1y2, '' using 1:19 title "Program Fail Cnt Total" with lines  axes x1y2, '' using 1:22 title "Erase Fail Count Total" with lines  axes x1y2, '' using 1:25 title "Runtime Bad Block" with lines   axes x1y2, '' using 1:28 title "Reported Uncorrect" with lines  axes x1y2, '' using 1:34 title "Hardware ECC Recovered" with lines dt 3  axes x1y2, '' using 1:40 title "POR Recovery Count" with lines dt 1 linetype rgb "green" axes x1y1

pause -1 "Hit any key to continue"

I'm sure better scripts are available!

3

If you're looking for something a little more advanced, my tools of choice for this are Telegraf + InfluxDB + Grafana.

  • Telegraf is a monitoring agent that can pull stats from hundreds of sources, including SMART data
  • InfluxDB is a time-series database, optimized for storing things that are measured over time (ie exactly this!)
  • Grafana connects to Influx to display graphs and set up alerting

There's obviously more to setup and maintain, but there's a lot more power and flexibility. eg you can have multiple devices sending their stats to one Influx server.

With everything installed, you can then set up a dashboard:

Grafana dashboard example

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.