Is there a more elegant way to count words and assign that count to variables?

Question

I have a script:

#!/bin/bash

/root/xiotech status > xiostatus.tmp
SyncCount=$(grep -c Sync xiostatus.tmp)
PauseCount=$(grep -c paused xiostatus.tmp)
CopyingCount=$(grep -c Copying xiostatus.tmp)

if [ "$SyncCount" -eq "11" ]
then echo All 11 mirrors are in sync.

else echo $PauseCount mirrors are paused and $CopyingCount mirrors are syncing.
fi

rm -f xiostatus.tmp

Is there a more elegant way to count and "variable-ize" those counts using something like awk? In this case the file is tiny so it's not a big deal, but if the file were 900mb, it would take a lot of extra cycles to go through it 3 times...

Answer 1

awk can replace the entire script pretty easily:

#!/usr/bin/awk -f

/Sync/ {SyncCount++}
/paused/ {PauseCount++}
/Copying/ {CopyingCount++}

END {
    if(SyncCount == 11)
        print "All 11 mirrors are in sync."
    else
        print (+PauseCount) " mirrors are paused and " (+CopyingCount) " mirrors are syncing."
}

The (+var) is to force awk to treat the variable as a number (so it will output 0 if the variable was unset). You can also use a BEGIN block to set all the variables to 0 initially:

BEGIN {
    SyncCount = PauseCount = CopyingCount = 0
}

---

Stick that in a file and run awk -f /path/to/the/script.awk xiostatus.tmp. If you don't need the temporary file, you can even do /root/xiotech status | awk -f /path/to/the/script.awk.

If you set the execution bit on the awk script, you can call it as a standalone executable: /path/to/the/script.awk xiostatus.tmp, or /root/xiotech status | /path/to/the/script.awk.

Answer 2

For those who want to count all instances, here is an awk version which will count multiple non-overlapping instances when there is more than one on the same line

UPDATE: I have now included another method which uses split(.... It is much faster than the match( substr(... method which is now listed below the faster one. The split(... method is more than 4 times faster than the other... (tested on 87 files for a total of 407,612 lines.
For further comparison, Michael Mrozek's method, using /Sync/ range-selection (which counts lines containg each pattern vs. counting all instances of the pattern) is twice as fast as this new method (for the same data).

Another fringe(?) benefit of this faster split(methos) is that it is quite tolerant to invalid UTF-8 chars in the file (so long as they are not in the delimiter pattern)... The delimiters are themselves the actual string patterns being counted... Several of my test files had invalid UTF-8 in them, and it took me quite a while to discover why I got different results from the two methods.
Once the problem files were re-encoded to valid UTF-8, both methods produce identical results.

Here is the new faster method (4+ times faster)... using split(...

#!/bin/bash
pat='xx|yy|zz'
awk -v vpat="$pat" 'BEGIN { 
  split(vpat, pat, "|"); for(i in pat) pz++ 
} 
{ if (NF) { for( p in pat ) { ct[p]+=(split( $0, A, pat[p] ) -1) }}
}
END { print " count   pattern"
      for (p=1; p<=pz; p++) { printf "%6d   %s\n", +ct[p], pat[p] } 
}' file

---

Here is the Slower method. using match( substr(...

#!/bin/bash
# Count occurrences of multiple non-overlapping string patterns
awk 'BEGIN {
  pattern[1]="xx"
  pattern[2]="yy"
  pattern[3]="zz"
}
{ for( p in pattern ) {
    LHB=0; RSTART=RLENGTH=1
    while( match( substr( $0, LHB+=(RSTART+RLENGTH-1)), pattern[p] )){
      count[p]++ 
    }
  }
} END {
  print "occurs  pattern"
  for (p in pattern) {
    printf "%6d  %s\n", +count[p], pattern[p] 
  } 
}' file

Here is the input file

xx xx  xx
xx             yy           xx

The output is as follows:

occurs  pattern
     5  xx
     1  yy
     0  zz

Answer 3

How about:

eval `/root/xiotech status | grep -Eo 'Sync|paused|Copying' | sort | uniq -c | 
    awk '{print "count_" $2 "=" $1}'`
if [ "$count_Sync" -eq 11 ]; then
    echo All 11 mirrors are in sync.
else
    echo $count_paused mirrors are paused and $count_Copying mirrors are syncing.
fi

The grep -Eo allows for searching multiple patterns (separated by "|") and returns only the string that matched. The sort | uniq -c shows the count of the words found. The awk script formats new shell commands to create variables starting with "count_". And finally, the eval will take the created shell commands and evaluate them within the shell.