bash - Separate "table" values into strings in array

Question

EDIT: I'm sorry, the output I claimed is wrong. There are more spaces than I previously thought (something happened when the output was saved to html file to remove these) The real output is as follows:

user@Debian:~$ sudo smartctl -l selftest /dev/sda | grep -e "#"
# 1  Short offline       Completed without error       00%      7264         -
# 2  Short offline       Completed without error       00%      7240         -
# 3  Short offline       Completed without error       00%      7219         -
# 4  Short offline       Completed without error       00%      7192         -
# 5  Short offline       Completed without error       00%      7168         -
# 6  Short offline       Completed without error       00%      7144         -
# 7  Extended offline    Completed without error       00%      7125         -
# 8  Short offline       Completed without error       00%      7096         -
# 9  Short offline       Completed without error       00%      7072         -
#10  Short offline       Completed without error       00%      7049         -
#11  Short offline       Completed without error       00%      7004         -

I'm not sure if I'm using the correct terminology as I'm rather new to Linux/bash.

Anyway, I'm using Smartmontools to detect and notify me if there are any SMART errors. It's working as I want it to but I would like to get some daily stats on the HDD's so I made my own script that collects info from smartmontools and other interesting things (like temps, SMART values and HDD space used). Might not be the best way to go about something like this but I enjoy doing it and I'm learning as I go.

The email I'm sending, is formatted as HTML to make tables and adding font colors for positive/negative results (green/red). But as I tried to make one table for displaying self tests, I got some issues.

The command I'm using is: sudo smartctl -l selftest $HDD | grep '#' >> $SMARTFILE (in a loop where $HDD is all the HDD in my system and $SMARTFILE is the html file I'm saving it to.

The output of this command looks like this:

# 1 Short offline Completed without error 00% 7264 -

# 2 Short offline Completed without error 00% 7240 -

And so on. I am using the following code to get the Serial number of the drive:

HDDinfo="$(sudo smartctl --info $HDD | grep -e 'Serial Number')"
IFS=':' read -r -a array <<< "$HDDinfo"

Since sudo smartctl --info $HDD | grep -e 'Serial Number' normally outputs

Serial Number: WD-RESTOFS/N123

But to put it in a table, I separated the string using the ':' char and get an array like this:

Serial Number,WD-RESTOFS/N123

But with the output I get for sudo smartctl -l selftest $HDD | grep '#' >> $SMARTFILE, there is no (to me) obvious way to separate them and the way I did it before won't work as the strings I want have spaces in them and can therefore not be separated using a space char.

TL;DR, I have the following command sudo smartctl -l selftest /dev/sda | grep '#' >> $SMARTFILE that has an output like this:

# 1 Short offline Completed without error 00% 7264 -

# 2 Short offline Completed without error 00% 7240 -

I want to make an array (or similar) to store them individually like this:

# 1,Short offline,Completed without error,00%,7264,-

So that I can easily put it into and HTML table. Can this be done? If an error does occur, it might look something like this:

# 1 Short offline Completed: read failure 20% 717 555027747

Please let me know if something is unclear or if there is any other information needed.

Answer 1

From your (small) sample of smartctl messages above, it seems their parts are basically separated by "<space><anything but a lowercase>" (except for the "# nnn" field at the very beginning of the line).

sed could help separate the parts:

$ smartctl_output="\                                           
# 1 Short offline Completed without error 00% 7264 -
# 2 Short offline Completed without error 00% 7240 -
# 1 Short offline Completed: read failure 20% 717 555027747"

$ csv="$( sed 's/ //; s/ \([^[:lower:]]\)/,\1/g' <<< "$smartctl_output" )"

$ echo "$csv"
#1,Short offline,Completed without error,00%,7264,-
#2,Short offline,Completed without error,00%,7240,-
#1,Short offline,Completed: read failure,20%,717,555027747

If this is what you want, you can now populate your array as you did with HDDinfo.

[update]

Here is an explanation on the sed part that does the splitting: the sed program is made of two parts that I have put on one line. Here is the expanded version:

sed '
    s/ //
    s/ \([^[:lower:]]\)/,\1/g
'

A sed program operates on each line of input: it reads one line, applies a set of transformations, and prints the line. Then it starts over with the next line until there isn't any more lines to read.

Here the first sed command s/ // deletes the first space to put together "#" and the following number.

Then, the second sed command s/ \([^[:lower:]]\)/,\1/g search the beginning of each field (as defined by "<space><anything but a lowercase>") and substitutes the space with a colon. The \1 refers to the regex between parentheses "\([^[:lower:]]\)", which represents the first character of the next field.

The remaining part is a test: instead of feeding sed with the content of a file or the output of a command, I fed it with the variable smartctl_output (a string made of your samples) and I assigned the result to the csv variable.

[update #2]

It appears now that the fields are separated by two or more spaces. It is even easier than previously. The sed command becomes:

sed 's/  \+/,/g'

Which means: replace all series of two or more spaces with a colon.

Answer 2

I can't think of a way to do it natively in the shell, but in perl for example you could define a regular expression for field splitting, and use that to insert a single delimiter of your choice which could then be read simply using IFS=, or whatever.

Based on your sample, the fields might be split at a space followed by either:

1. an upper case character or hyphen; or
2. a sequence of at least two digits

So piping your command something like

. . . | 
  perl -F'[[:space:]](?=[[:upper:]-]|[[:digit:]]{2,})' -anle 'print join ",", @F'