1

I am using unix shell script and have an Input File with data as:

3:abc
1:xyz
1:abc
2:def
10:xyz

My expected output is:

4:abc
11:xyz
2:def

i.e. Find unique string on each line after delimiter and add up the numbers before that. How to do this?

3

Here is a solution using awk. It accumulates the values into an array.

awk -F ":" '{count[$2]+=$1} END {for (key in count) print key, count[key]}' awk_data.txt

And here is a version, using a bash script:

#!/usr/bin/env bash
declare -A count
while read line; do
    key=${line##*:}
    cnt=${line%%:*}
    count[$key]=$(($cnt + ${count[$key]=0}))
done < "$1"
for K in "${!count[@]}"; do echo $K ${count[$K]}; done

And another bash version from the comments, using IFS=:

#!/usr/bin/env bash
declare -A count
while IFS=: read -r cnt key; do
    count[$key]=$(($cnt + ${count[$key]=0}))
done < "$1"
for K in "${!count[@]}"; do echo $K ${count[$K]}; done
  • This works like charm. lovely ! thank you Stephen. – user1582625 Feb 11 '17 at 5:38
  • Why not IFS=: read -r CNT key – iruvar Feb 11 '17 at 17:25
  • @iruvar, I sadly (and likely in-advisedly) tend to avoid IFS because of the side effects, that can cause weirdness if you forget the unset. But a fair comment, I added version using IFS=: – Stephen Rauch Feb 11 '17 at 17:39
  • You can scope IFS to just the read statement and avoid the need to unset using while IFS=: read -r cnt key – iruvar Feb 11 '17 at 18:52
  • @iruvar, It is always a good day when you learn something new. Thanks! – Stephen Rauch Feb 11 '17 at 19:12
3

You may also want to take a loot at GNU datamash utility. Example (table.txt from PO):

$ sort -t: -k 2 table.txt | datamash -t: -g 2 sum 1 collapse 1
abc:4:1,3
def:2:2
xyz:11:10,1
  • -t: field delimiter is :
  • -g 2 group by the (previously sorted) second column
  • sum 1 sum the correspondent values in column 1
  • collapse 1 and also collapse them in a "," separate list

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.