Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a file which contains the single line below:

{machineA=[0, 1024, 4, 1028], machineB=[1, 1025, 5, 1029]}

in which I have two datasets:

machineA=[0, 1024, 4, 1028]
machineB=[1, 1025, 5, 1029]

Now, I need to read the above file and split it in such a way that I can extract each machine's information as mentioned above and store it in some data structure.

Currently, I am confused which data structure I should be using in a bash shell script. If I was doing this in Java, I would use Map<String, Set<String>> but I'm not sure what should I use in a shell script.

And after storing it in some data structure, I need to iterate it and print out the result.

I am able to read the above file using the shell script below:

#!/bin/bash

while read -r line; do
       echo "$line"
    done < data.txt

But I am not sure how to split the above line data in such a way so that I am able to extract each machine informaiton and then store it in some data structure?

UPDATE:-

Below is my shell script I have got after following the suggestion given by glenn -

#!/bin/bash

while read -r line; do
       echo "$line"
       declare -A "$(
            echo "x=("
            grep -oP '(\w+)(?==)|(?<==\[).*?(?=\])' <<< "$line" |
              tr -d , |
              paste - - |
              sed 's/^/[/; s/\t/]="/; s/$/"/'
            echo ")"
        )"

        for key in "${!x[@]}"; do           # need quotes here
            for element in ${x[$key]}; do   # no quotes here
                printf "%s\t%s\n" "$key" "$element"
            done
        done    
    done < primary.txt
share|improve this question

4 Answers 4

bash data structures are too rudimentary for even 2-dimensional arrays. Are you sure this is the tool you want to use?

With bash and GNU grep, you can do this distasteful mess:

line='{machineA=[0, 1024, 4, 1028], machineB=[1, 1025, 5, 1029]}'
declare -A "$(
    echo "x=("
    grep -oP '(\w+)(?==)|(?<==\[).*?(?=\])' <<< "$line" |
      tr -d , |
      paste - - |
      sed 's/^/[/; s/\t/]="/; s/$/"/'
    echo ")"
)"

for key in "${!x[@]}"; do           # need quotes here
    for element in ${x[$key]}; do   # no quotes here
        printf "%s\t%s\n" "$key" "$element"
    done
done
machineA    0
machineA    1024
machineA    4
machineA    1028
machineB    1
machineB    1025
machineB    5
machineB    1029

This is pretty fragile. I'd use Perl for something like this: still ugly but more concise

echo "$line" | perl -MData::Dumper -ne '
    s/=\[/=>[/g; 
    eval "\$x=$_";
    # do something with your data structure (a hash of arrays) 
    print Dumper($x)
'
$VAR1 = {
          'machineB' => [
                          1,
                          1025,
                          5,
                          1029
                        ],
          'machineA' => [
                          0,
                          1024,
                          4,
                          1028
                        ]
        };
share|improve this answer
    
Thanks for suggestion. I might go with shell script option as finally I need to use scp so I believe doing scp in shell script will be easy. But anyways, let see how this goes out. I have updated my question with the actual shell script I might be using after incorporating your suggestion. Please take a look and let me know if it looks correct and if there are anything you would like to modify then let me know as well. –  SSH Dec 17 '13 at 1:49
    
+1 Pretty slick move with the eval, there. –  Joseph R. Dec 17 '13 at 8:38

The shell text processing utilities are primarily designed to manipulate data represented with one record per line and fields separated by either whitespace, or a fixed character. This format is completely different and you won't be able to process it in a straightforward way.

One approach is to preprocess the file to fit the type of format that can be processed easily. I assume that brackets and braces aren't used in any way other than portrayed here (braces around the whole text, brackets around machine value lists).

<data.txt sed -e 's/^{//' -e 's/}$//' -e 's/ *= *\[/,/g' -e 's/, */,/g' -e 's/\] *$//' -e 's/] *, */\n/g'

The result has one machine per line and commas to separate records. The following snippet parses out the machine name on each line and leaves a comma-separated list of values in values.

… | while IFS=, read -r machine values; do …

The following bash-specific snippet puts the values in an array.

… | while IFS=, read -r -a values; do
  machine=${values[0]}; shift values
  echo "There are ${#values[@]} on machine $machine"
done
share|improve this answer
    
@Giles: Thanks for suggestion. Is it also possible to get the total number of files for each machines? meaning the total count using the same above command? Like, for above example, machineA has four files and machineB has also four files –  SSH Dec 17 '13 at 8:50
    
@SSH See my edit. –  Gilles Dec 17 '13 at 13:56

You can use awk to complete the task.

awk -F "], " '/[a-zA-Z]=\[[0-9]/ {gsub(/{|}/,""); for(i=1; i<=NF; i++) if($i !~ /\]$/) print $i"]"; else print $i}' data.txt

machineA=[0, 1024, 4, 1028]
machineB=[1, 1025, 5, 1029]
share|improve this answer
    
Thanks John. Is it possible to get the total number of files as well for each machine. Like, for above example, machineA has four files and machineB has also four files. Is it possible to get that as well? –  SSH Dec 17 '13 at 8:32

This looks a bit like JSON. You can fix it to be proper JSON and use JSON tools:

$ echo '{machineA=[0, 1024, 4, 1028], machineB=[1, 1025, 5, 1029]}' |  perl -pe 's!\b!"!g; s/=/:/g' | json_pp
{
   "machineB" : [
      "1",
      "1025",
      "5",
      "1029"
   ],
   "machineA" : [
      "0",
      "1024",
      "4",
      "1028"
   ]
}
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.