0

I have a text like this pattern on many lines. Can I group only the values and the name into a single line?

ABCDEFG_10_node10:2154  ABCDEFG_10_node10:54
ABCDEFG_10_node10:2254  ABCDEFG_10_node10:64
ABCDEFG_10_node10:410 ABCDEFG_10_node10:10
ABCDEFG_10_node10:210 ABCDEFG_10_node10:10 
ABCDEFG_10_node10:365
ABCDEFG_10_node10:890
ABCDEFG_10_node10:741
XXYZZ_71_node2:24: XXYZZ_71_node2:504:
X3y5z_53_node1:664: X3y5z_53_node1:990:
RCTY_11_node2:224: RCTY_11_node2:234:

Output expected:

ABCDEFG_10_node10: 2154,2254,410,210,365,890,741,54,64,10,10
XXYZZ_71_node2: 24,504
X3y5z_53_node1: 664,990
RCTY_11_node2: 224,234

I’m on AIX. How can I do this?

5
  • 2
    Do you specifically insist on that order,  or would 2154,54,2254,64,410,10,210,10,365,890,741 be acceptable?
    – Scott
    Oct 6, 2017 at 10:47
  • any order is acceptable scott.
    – satsensort
    Oct 6, 2017 at 11:05
  • 1
    @satsensort, you have added some content to your input. Does it really contain trailing semicolons after some values XXYZZ_71_node2:24: , X3y5z_53_node1:990:? or it's just a typo? Oct 6, 2017 at 11:16
  • 1
    Please make sure your examples accurately represent your file. Your new data have the extra : and also extra whitespace. Do we need to deal with that as well?
    – terdon
    Oct 6, 2017 at 11:20
  • If you're happy with one or several of the answers, upvote them. If one is solving your issue, accepting it would be the best way of saying "Thank You!" :-)
    – Kusalananda
    Oct 12, 2017 at 19:43

4 Answers 4

5

A Perl approach (assuming the order isn't important):

$ perl -lne 'while(/(\w+):(\d+)/g){
                push @{$k{$1}},$2
             } 
             END{
                print "$_: " . join ",", @{$k{$_}} for keys %k
             }' file 
ABCDEFG_10_node10: 2154,54,2254,64,410,10,210,10,365,890,741

That reads the input file line by line (-ln) and runs the script given by -e on it. The while(/(\w+):(\d+)/g) will collect all instances of non-whitespace, followed by a : and then more non-whitespace. Since they are being captured in parentheses, the $1 will be the name and the $2 the value. These are then pushed into a hash of arrays (hash %k, whose values are arrays). Finally, we print each key of the hash (the names) along with the array of its values, joined by ,.

If you value conciseness, you can do write the above as a one-liner:

perl -lne 'while(/(\S+):(\S+)/g){push @{$k{$1}},$2}}{$"=",";print"$_: @{$k{$_}}" for keys%k' file

pushed past legibility:

perl -nE'push@{$k{$1}},$2while/(\w+):(\d+)/g}{$"=",";say"$_: @{$k{$_}}"for keys%k' file
3

Awk solution:

awk -F':|[[:space:]]+' '{ 
         a[$1]=a[$1]? a[$1]","$2:$2; if(NF==4) b[$3]=b[$3]? b[$3]","$4:$4 
     }
     END{ for(i in a) printf "%s: %s%s\n",i,a[i],(i in b)? ","b[i]:"" }' file

  • -F':|[[:space:]]+' - complex field separator

  • a[$1]=a[$1]? a[$1]","$2:$2 - grouping values for each unique name ABCDEFG...

  • if(NF==4) b[$3]=b[$3]? b[$3]","$4:$4 - if there is an additional right-side section - group values into additional array b


The output:

ABCDEFG_10_node10: 2154,2254,410,210,365,890,741,54,64,10

----------

If the order of values isn't important the above approach could be slightly simplified:

awk -F':|[[:space:]]+' '{ 
        a[$1]=a[$1]? a[$1]","$2:$2; if(NF==4) a[$3]=a[$3]? a[$3]","$4:$4 
     }
     END{ for(i in a) print i":",a[i] }' file
2
  • @satsensort, you're welcome Oct 6, 2017 at 12:00
  • You may be able to simplify the awk by formatting the input to one column: fmt -w 1 input | sed 's/:$//' | awk ... (I also stripped off the trailing colon characters)
    – Jeff Schaller
    Oct 11, 2017 at 1:46
1
awk '{ for (i=1;i<=NF;i++) { split($NF,arr,":");if (dat[arr[1]]=="") { dat[arr[1]]=arr[2] } else { dat[arr[1]]=dat[arr[1]]","arr[2] } } } END { for ( i in dat ) { print i": "dat[i] } }' filename

An alternative awk solution to Roman's where we take each space delimited piece of data in turn and then further split the data using the split function in the array arr based on the character : We then build an array keyed on the ABC etc string with the string of numbers to be printed. We then loop through this array (dat) building a string starting with the key along with : and the string. This is then printed.

0
0

Using Raku (formerly known as Perl_6)

raku -e 'say .key, " => ", .value.words[1,3,5...*] for lines.split(/<[:\s]>/, :skip-empty).rotor(2).classify( *.[0]);'

Sample Input:

ABCDEFG_10_node10:2154  ABCDEFG_10_node10:54
ABCDEFG_10_node10:2254  ABCDEFG_10_node10:64
ABCDEFG_10_node10:410 ABCDEFG_10_node10:10
ABCDEFG_10_node10:210 ABCDEFG_10_node10:10 
ABCDEFG_10_node10:365
ABCDEFG_10_node10:890
ABCDEFG_10_node10:741
XXYZZ_71_node2:24: XXYZZ_71_node2:504:
X3y5z_53_node1:664: X3y5z_53_node1:990:
RCTY_11_node2:224: RCTY_11_node2:234:

Sample Output:

XXYZZ_71_node2 => (24 504)
RCTY_11_node2 => (224 234)
ABCDEFG_10_node10 => (2154 54 2254 64 410 10 210 10 365 890 741)
X3y5z_53_node1 => (664 990)

Briefly, lines are read in, destructively split on : and \s (omitting any empty elements via :skip-empty), rotor (join) every 2 elements, and classify-ing by the first of each pair. [I could make the call .classify( *.[0].unique) to clarify the intent, but Raku does the right thing either way].

If the OP really wants the format specified in his/her post, then substitute => with : and add a call to join at the end of the .value call, so the whole section prior to for reads: say .key, ": ", .value.words[1,3,5...*].join(",")

Sample Output (2):

X3y5z_53_node1: 664,990
RCTY_11_node2: 224,234
ABCDEFG_10_node10: 2154,54,2254,64,410,10,210,10,365,890,741
XXYZZ_71_node2: 24,504

https://docs.raku.org/routine/classify
https://raku.org

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.