10

I have a file which contains the single line below:

{machineA=[0, 1024, 4, 1028], machineB=[1, 1025, 5, 1029]}

in which I have two datasets:

machineA=[0, 1024, 4, 1028]
machineB=[1, 1025, 5, 1029]

Now, I need to read the above file and split it in such a way that I can extract each machine's information as mentioned above and store it in some data structure.

Currently, I am confused which data structure I should be using in a bash shell script. If I was doing this in Java, I would use Map<String, Set<String>> but I'm not sure what should I use in a shell script.

And after storing it in some data structure, I need to iterate it and print out the result.

I am able to read the above file using the shell script below:

#!/bin/bash

while read -r line; do
       echo "$line"
    done < data.txt

But I am not sure how to split the above line data in such a way so that I am able to extract each machine informaiton and then store it in some data structure?

UPDATE:-

Below is my shell script I have got after following the suggestion given by glenn -

#!/bin/bash

while read -r line; do
       echo "$line"
       declare -A "$(
            echo "x=("
            grep -oP '(\w+)(?==)|(?<==\[).*?(?=\])' <<< "$line" |
              tr -d , |
              paste - - |
              sed 's/^/[/; s/\t/]="/; s/$/"/'
            echo ")"
        )"

        for key in "${!x[@]}"; do           # need quotes here
            for element in ${x[$key]}; do   # no quotes here
                printf "%s\t%s\n" "$key" "$element"
            done
        done    
    done < primary.txt

4 Answers 4

4

bash data structures are too rudimentary for even 2-dimensional arrays. Are you sure this is the tool you want to use?

With bash and GNU grep, you can do this distasteful mess:

line='{machineA=[0, 1024, 4, 1028], machineB=[1, 1025, 5, 1029]}'
declare -A "$(
    echo "x=("
    grep -oP '(\w+)(?==)|(?<==\[).*?(?=\])' <<< "$line" |
      tr -d , |
      paste - - |
      sed 's/^/[/; s/\t/]="/; s/$/"/'
    echo ")"
)"

for key in "${!x[@]}"; do           # need quotes here
    for element in ${x[$key]}; do   # no quotes here
        printf "%s\t%s\n" "$key" "$element"
    done
done
machineA    0
machineA    1024
machineA    4
machineA    1028
machineB    1
machineB    1025
machineB    5
machineB    1029

This is pretty fragile. I'd use Perl for something like this: still ugly but more concise

echo "$line" | perl -MData::Dumper -ne '
    s/=\[/=>[/g; 
    eval "\$x=$_";
    # do something with your data structure (a hash of arrays) 
    print Dumper($x)
'
$VAR1 = {
          'machineB' => [
                          1,
                          1025,
                          5,
                          1029
                        ],
          'machineA' => [
                          0,
                          1024,
                          4,
                          1028
                        ]
        };
2
  • Thanks for suggestion. I might go with shell script option as finally I need to use scp so I believe doing scp in shell script will be easy. But anyways, let see how this goes out. I have updated my question with the actual shell script I might be using after incorporating your suggestion. Please take a look and let me know if it looks correct and if there are anything you would like to modify then let me know as well.
    – SSH
    Dec 17, 2013 at 1:49
  • +1 Pretty slick move with the eval, there.
    – Joseph R.
    Dec 17, 2013 at 8:38
1

The shell text processing utilities are primarily designed to manipulate data represented with one record per line and fields separated by either whitespace, or a fixed character. This format is completely different and you won't be able to process it in a straightforward way.

One approach is to preprocess the file to fit the type of format that can be processed easily. I assume that brackets and braces aren't used in any way other than portrayed here (braces around the whole text, brackets around machine value lists).

<data.txt sed -e 's/^{//' -e 's/}$//' -e 's/ *= *\[/,/g' -e 's/, */,/g' -e 's/\] *$//' -e 's/] *, */\n/g'

The result has one machine per line and commas to separate records. The following snippet parses out the machine name on each line and leaves a comma-separated list of values in values.

… | while IFS=, read -r machine values; do …

The following bash-specific snippet puts the values in an array.

… | while IFS=, read -r -a values; do
  machine=${values[0]}; shift values
  echo "There are ${#values[@]} on machine $machine"
done
2
  • @Giles: Thanks for suggestion. Is it also possible to get the total number of files for each machines? meaning the total count using the same above command? Like, for above example, machineA has four files and machineB has also four files
    – SSH
    Dec 17, 2013 at 8:50
  • @SSH See my edit. Dec 17, 2013 at 13:56
0

You can use awk to complete the task.

awk -F "], " '/[a-zA-Z]=\[[0-9]/ {gsub(/{|}/,""); for(i=1; i<=NF; i++) if($i !~ /\]$/) print $i"]"; else print $i}' data.txt

machineA=[0, 1024, 4, 1028]
machineB=[1, 1025, 5, 1029]
1
  • Thanks John. Is it possible to get the total number of files as well for each machine. Like, for above example, machineA has four files and machineB has also four files. Is it possible to get that as well?
    – SSH
    Dec 17, 2013 at 8:32
0

This looks a bit like JSON. You can fix it to be proper JSON and use JSON tools:

$ echo '{machineA=[0, 1024, 4, 1028], machineB=[1, 1025, 5, 1029]}' |  perl -pe 's!\b!"!g; s/=/:/g' | json_pp
{
   "machineB" : [
      "1",
      "1025",
      "5",
      "1029"
   ],
   "machineA" : [
      "0",
      "1024",
      "4",
      "1028"
   ]
}

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .