Split the file and put it in corresponding data structure?

Question

I have a file which contains the single line below:

{machineA=[0, 1024, 4, 1028], machineB=[1, 1025, 5, 1029]}

in which I have two datasets:

machineA=[0, 1024, 4, 1028]
machineB=[1, 1025, 5, 1029]

Now, I need to read the above file and split it in such a way that I can extract each machine's information as mentioned above and store it in some data structure.

Currently, I am confused which data structure I should be using in a bash shell script. If I was doing this in Java, I would use Map<String, Set<String>> but I'm not sure what should I use in a shell script.

And after storing it in some data structure, I need to iterate it and print out the result.

I am able to read the above file using the shell script below:

#!/bin/bash

while read -r line; do
       echo "$line"
    done < data.txt

But I am not sure how to split the above line data in such a way so that I am able to extract each machine informaiton and then store it in some data structure?

UPDATE:-

Below is my shell script I have got after following the suggestion given by glenn -

#!/bin/bash

while read -r line; do
       echo "$line"
       declare -A "$(
            echo "x=("
            grep -oP '(\w+)(?==)|(?<==\[).*?(?=\])' <<< "$line" |
              tr -d , |
              paste - - |
              sed 's/^/[/; s/\t/]="/; s/$/"/'
            echo ")"
        )"

        for key in "${!x[@]}"; do           # need quotes here
            for element in ${x[$key]}; do   # no quotes here
                printf "%s\t%s\n" "$key" "$element"
            done
        done    
    done < primary.txt

Answer 1

bash data structures are too rudimentary for even 2-dimensional arrays. Are you sure this is the tool you want to use?

With bash and GNU grep, you can do this distasteful mess:

line='{machineA=[0, 1024, 4, 1028], machineB=[1, 1025, 5, 1029]}'
declare -A "$(
    echo "x=("
    grep -oP '(\w+)(?==)|(?<==\[).*?(?=\])' <<< "$line" |
      tr -d , |
      paste - - |
      sed 's/^/[/; s/\t/]="/; s/$/"/'
    echo ")"
)"

for key in "${!x[@]}"; do           # need quotes here
    for element in ${x[$key]}; do   # no quotes here
        printf "%s\t%s\n" "$key" "$element"
    done
done

machineA    0
machineA    1024
machineA    4
machineA    1028
machineB    1
machineB    1025
machineB    5
machineB    1029

This is pretty fragile. I'd use Perl for something like this: still ugly but more concise

echo "$line" | perl -MData::Dumper -ne '
    s/=\[/=>[/g; 
    eval "\$x=$_";
    # do something with your data structure (a hash of arrays) 
    print Dumper($x)
'

$VAR1 = {
          'machineB' => [
                          1,
                          1025,
                          5,
                          1029
                        ],
          'machineA' => [
                          0,
                          1024,
                          4,
                          1028
                        ]
        };

Answer 2

The shell text processing utilities are primarily designed to manipulate data represented with one record per line and fields separated by either whitespace, or a fixed character. This format is completely different and you won't be able to process it in a straightforward way.

One approach is to preprocess the file to fit the type of format that can be processed easily. I assume that brackets and braces aren't used in any way other than portrayed here (braces around the whole text, brackets around machine value lists).

<data.txt sed -e 's/^{//' -e 's/}$//' -e 's/ *= *\[/,/g' -e 's/, */,/g' -e 's/\] *$//' -e 's/] *, */\n/g'

The result has one machine per line and commas to separate records. The following snippet parses out the machine name on each line and leaves a comma-separated list of values in values.

… | while IFS=, read -r machine values; do …

The following bash-specific snippet puts the values in an array.

… | while IFS=, read -r -a values; do
  machine=${values[0]}; shift values
  echo "There are ${#values[@]} on machine $machine"
done

Answer 3

You can use awk to complete the task.

awk -F "], " '/[a-zA-Z]=\[[0-9]/ {gsub(/{|}/,""); for(i=1; i<=NF; i++) if($i !~ /\]$/) print $i"]"; else print $i}' data.txt

machineA=[0, 1024, 4, 1028]
machineB=[1, 1025, 5, 1029]

Answer 4

This looks a bit like JSON. You can fix it to be proper JSON and use JSON tools:

$ echo '{machineA=[0, 1024, 4, 1028], machineB=[1, 1025, 5, 1029]}' |  perl -pe 's!\b!"!g; s/=/:/g' | json_pp
{
   "machineB" : [
      "1",
      "1025",
      "5",
      "1029"
   ],
   "machineA" : [
      "0",
      "1024",
      "4",
      "1028"
   ]
}