EDIT: I'm sorry, the output I claimed is wrong. There are more spaces than I previously thought (something happened when the output was saved to html file to remove these) The real output is as follows:

user@Debian:~$ sudo smartctl -l selftest /dev/sda | grep -e "#"
# 1  Short offline       Completed without error       00%      7264         -
# 2  Short offline       Completed without error       00%      7240         -
# 3  Short offline       Completed without error       00%      7219         -
# 4  Short offline       Completed without error       00%      7192         -
# 5  Short offline       Completed without error       00%      7168         -
# 6  Short offline       Completed without error       00%      7144         -
# 7  Extended offline    Completed without error       00%      7125         -
# 8  Short offline       Completed without error       00%      7096         -
# 9  Short offline       Completed without error       00%      7072         -
#10  Short offline       Completed without error       00%      7049         -
#11  Short offline       Completed without error       00%      7004         -

I'm not sure if I'm using the correct terminology as I'm rather new to Linux/bash.

Anyway, I'm using Smartmontools to detect and notify me if there are any SMART errors. It's working as I want it to but I would like to get some daily stats on the HDD's so I made my own script that collects info from smartmontools and other interesting things (like temps, SMART values and HDD space used). Might not be the best way to go about something like this but I enjoy doing it and I'm learning as I go.

The email I'm sending, is formatted as HTML to make tables and adding font colors for positive/negative results (green/red). But as I tried to make one table for displaying self tests, I got some issues.

The command I'm using is: sudo smartctl -l selftest $HDD | grep '#' >> $SMARTFILE (in a loop where $HDD is all the HDD in my system and $SMARTFILE is the html file I'm saving it to.

The output of this command looks like this:

# 1 Short offline Completed without error 00% 7264 -

# 2 Short offline Completed without error 00% 7240 -

And so on. I am using the following code to get the Serial number of the drive:

HDDinfo="$(sudo smartctl --info $HDD | grep -e 'Serial Number')"
IFS=':' read -r -a array <<< "$HDDinfo"

Since sudo smartctl --info $HDD | grep -e 'Serial Number' normally outputs

Serial Number: WD-RESTOFS/N123

But to put it in a table, I separated the string using the ':' char and get an array like this:

Serial Number,WD-RESTOFS/N123

But with the output I get for sudo smartctl -l selftest $HDD | grep '#' >> $SMARTFILE, there is no (to me) obvious way to separate them and the way I did it before won't work as the strings I want have spaces in them and can therefore not be separated using a space char.

TL;DR, I have the following command sudo smartctl -l selftest /dev/sda | grep '#' >> $SMARTFILE that has an output like this:

# 1 Short offline Completed without error 00% 7264 -

# 2 Short offline Completed without error 00% 7240 -

I want to make an array (or similar) to store them individually like this:

# 1,Short offline,Completed without error,00%,7264,-

So that I can easily put it into and HTML table. Can this be done? If an error does occur, it might look something like this:

# 1 Short offline Completed: read failure 20% 717 555027747

Please let me know if something is unclear or if there is any other information needed.

  • We/you would need to know how all field 2 might appear – Jeff Schaller Jan 6 '17 at 23:15
  • @JeffSchaller I'm sorry, what do you mean by "all field 2"? – L.Johnson Jan 6 '17 at 23:19
  • What possible values could show up there, so you know how to recognize it / split it. – Jeff Schaller Jan 6 '17 at 23:22
  • It'd be even better if smartctl had alternative output formats – Jeff Schaller Jan 6 '17 at 23:23
  • 1
    The first line of the table has the column width info, but you exclude that using grep. This makes interpreting the rest harder. – Jasen Jan 7 '17 at 1:49

From your (small) sample of smartctl messages above, it seems their parts are basically separated by "<space><anything but a lowercase>" (except for the "# nnn" field at the very beginning of the line).

sed could help separate the parts:

$ smartctl_output="\                                           
# 1 Short offline Completed without error 00% 7264 -
# 2 Short offline Completed without error 00% 7240 -
# 1 Short offline Completed: read failure 20% 717 555027747"

$ csv="$( sed 's/ //; s/ \([^[:lower:]]\)/,\1/g' <<< "$smartctl_output" )"

$ echo "$csv"
#1,Short offline,Completed without error,00%,7264,-
#2,Short offline,Completed without error,00%,7240,-
#1,Short offline,Completed: read failure,20%,717,555027747

If this is what you want, you can now populate your array as you did with HDDinfo.


Here is an explanation on the sed part that does the splitting: the sed program is made of two parts that I have put on one line. Here is the expanded version:

sed '
    s/ //
    s/ \([^[:lower:]]\)/,\1/g

A sed program operates on each line of input: it reads one line, applies a set of transformations, and prints the line. Then it starts over with the next line until there isn't any more lines to read.

Here the first sed command s/ // deletes the first space to put together "#" and the following number.

Then, the second sed command s/ \([^[:lower:]]\)/,\1/g search the beginning of each field (as defined by "<space><anything but a lowercase>") and substitutes the space with a colon. The \1 refers to the regex between parentheses "\([^[:lower:]]\)", which represents the first character of the next field.

The remaining part is a test: instead of feeding sed with the content of a file or the output of a command, I fed it with the variable smartctl_output (a string made of your samples) and I assigned the result to the csv variable.

[update #2]

It appears now that the fields are separated by two or more spaces. It is even easier than previously. The sed command becomes:

sed 's/  \+/,/g'

Which means: replace all series of two or more spaces with a colon.

  • Wow, ok. Thank you for this. I will try it out and see if it works. I have no idea what it does though so I will have to spend some time googling this and try to learn at least some of it. But again, thank you. This might make things a lot easier for me. I'll just have to double check with what Jeff Schaller talked about so that there aren't any error messages that does start with a small letter. – L.Johnson Jan 6 '17 at 23:56
  • I just tried it, and it does seem to work for what I want to use it for. Thank you very much! – L.Johnson Jan 7 '17 at 0:01
  • Actually, I was wrong about the output. When it was saved as text and I opened it in the html file, there was only one space between all values, but I do have some issues as the real output has more than one spaces in between the strings I want. I've updated the question with the true output of the command, if you could please update your answer to make it work for what the real output is, I would really appreciate it as I've yet to learn what it is that you actually did. – L.Johnson Jan 7 '17 at 0:22
  • @L.Johnson "I have no idea what it does though so I will have to spend some time googling" => I have added an explanation to my answer – xhienne Jan 7 '17 at 0:33
  • Thank you for that! Did you see my update? Unfortunately I was wrong about how the output was formatted and there are actually more spaces in between the desired strings that was lost (somehow, probably in how it was displayed on the html site I copied the output from). Could you help me with an updated version of the code that handles this? Thank you – L.Johnson Jan 7 '17 at 0:37

I can't think of a way to do it natively in the shell, but in perl for example you could define a regular expression for field splitting, and use that to insert a single delimiter of your choice which could then be read simply using IFS=, or whatever.

Based on your sample, the fields might be split at a space followed by either:

  1. an upper case character or hyphen; or
  2. a sequence of at least two digits

So piping your command something like

. . . | 
  perl -F'[[:space:]](?=[[:upper:]-]|[[:digit:]]{2,})' -anle 'print join ",", @F'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.