0

data.now:

blah1,dah,blaha,sweet,games.log,5297484456,nagios-toin,529748456,on__host=92 SERVICE__ALERT_=51 Warning___The__results__of__service=16 Warning___on__host=92 Auto_save__of__retention__data__completed=1 Warning___Return=68 PASSIVE__SERVICE__CHECK_=53 ,1026--1313,1
blah1,dah,blaha,sweet,games.log,5297484456,nagios-toin,529748456,on__host=14 SERVICE__ALERT_=51 Warning___The__results__of__service=16 Warning___on__host=93 Auto_save__of__retention__data__completed=1 Warning___Return=68 PASSIVE__SERVICE__CHECK_=53 ,1026--1313,1
blah1,dah,blaha,sweet,games.log,5297484456,nagios-toin,529748456,on__host=93 SERVICE__ALERT_=51 Warning___The__results__of__service=16 Warning___on__host=92 Auto_save__of__retention__data__completed=1 Warning___Return=8 PASSIVE__SERVICE__CHECK_=53 ,1026--1313,1
blah1,dah,blaha,sweet,games.log,5297484456,nagios-toin,529748456,on__host=73 SERVICE__ALERT_=51 Warning___The__results__of__service=16 Warning___on__host=99 Auto_save__of__retention__data__completed=1 Warning___Return=68 PASSIVE__SERVICE__CHECK_=53 ,1026--1313,1
blah1,dah,blaha,sweet,games.log,5297484456,nagios-toin,529748456,on__host=93 SERVICE__ALERT_=51 Warning___The__results__of__service=16 Warning___on__host=75 Auto_save__of__retention__data__completed=1 Warning___Return=38 PASSIVE__SERVICE__CHECK_=53 ,1026--1313,1
blah1,dah,blaha,sweet,games.log,5297484456,nagios-toin,529748456,on__host=21 SERVICE__ALERT_=51 Warning___The__results__of__service=16 Warning___on__host=12 Auto_save__of__retention__data__completed=1 Warning___Return=28 PASSIVE__SERVICE__CHECK_=23 ,1026--1313,1[/CODE]

The fields in each line above are separated by "comma". What I'm interested in is field 9.

Field 9 has a number of values.

What i want to do is two parts:

  1. Be able to add up all the values of a specific pattern in all the lines in the datafile. For instance, if i want to know the total value "on__host" on all lines in the data file.

  2. Be able to add up all the values from all the patterns on each line, on all the lines in the log.

There was an old data i was working with which only had a single value in the 9th field. that was easy to handle. The data looked like this:

data.prev

blah1,dah,blaha,sweet,games.log,5297484456,nagios-toin,529748456,53,1026--1313,1

And all I needed to do to add up the values in the 9th field of all the lines was:

awk -F, 'BEGIN{sum=0} {sum+=$9} END {print sum}' data.prev

The kind of script that would solve this problem for me would look like this:

#!/bin/sh
Pattern=$1
if [ "${Pattern}" = "allpatterns" ] ; then
   awk should add up all the values in the 9th field of data in data.now
else
   if the user did not specify "allpatterns", then, awk should take the pattern name specified by the user and use that to decide which pattern to add up in the 9th field of all the lines.
    awk -F"," '$9 ~ /'${Pattern}'/ '{do awk magic}'  -- this is just an idea.
fi

this script is expected to run on all unix systems. The shell i will be using is either /bin/sh or /bin/bash.

For the first part of my request, if all the values of all the patterns on all the lines in the data file are added up, the output should be just the resulting sum of the numbers..i.e 504 (just an arbitrary number i picked).

For the second part of my request, if the values of a specific pattern are added up, the output should be just the resulting sum of the values for that specific pattern...i.e. "on__host=400".

0

For the first part, I think something like this would work:

cat data.now | awk -F, '{print $9}' | sed 's/=/ /g' | awk '
    { 
        for (i = 1; i <= NF; i += 2) {
            a[$i]+=$(i+1); 
        }
    } 
    END {
        for (i in a) print i, "=", a[i]
    }'

This code produces the following output:

PASSIVE__SERVICE__CHECK_ = 288
Warning___Return = 278
Auto_save__of__retention__data__completed = 6
SERVICE__ALERT_ = 306
Warning___The__results__of__service = 96
on__host = 386
Warning___on__host = 463

The only potential problem here is that the order is not the same as the one of the original file. If the order matters to you, you can use:

cat data.now | awk -F, '{print $9}' | sed 's/=/ /g' | awk '
    { 
        for (i = 1; i <= NF; i += 2) {
            a[$i]+=$(i+1); 
            b[i]=$i;
        }
    } 
    END {
        for (i in b) print b[i], "=", a[b[i]]
    }'

but it's not as easy to read.

For your second question, I think that this works to calculate the sum, although it might not be the shortest or most efficient way to do it.

cat data.now | awk -F, '{print $9}' | 
    sed "s/.*\<${Pattern}=\([0-9]\+\).*/\1/g" | 
    awk '{s += $1} END {print s}'
0

You can split filed $9 on space, then iterate over the result to select the pattern, and split the found element on = to extract the value:

awk -F, -v pat='^on__host=' '{
  n = split($9,a," "); 
  for(i=1;i<=n;i++) {
    if(a[i] ~ pat) {
      split(a[i],b,"="); 
      sum+=b[2]
    } 
  }
} 
END{print sum}
' data.now 

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.