For those who want to count all instances, here is an awk version which will count multiple non-overlapping instances when there is more than one on the same line
UPDATE: I have now included another method which uses split(...
. It is much faster than the match( substr(...
method which is now listed below the faster one.
The split(...
method is more than 4 times faster than the other... (tested on 87 files for a total of 407,612 lines.
For further comparison, Michael Mrozek's method, using /Sync/
range-selection (which counts lines containg each pattern vs. counting all instances of the pattern) is twice as fast as this new method (for the same data).
Another fringe(?) benefit of this faster split(methos)
is that it is quite tolerant to invalid UTF-8 chars in the file (so long as they are not in the delimiter pattern)... The delimiters are themselves the actual string patterns being counted... Several of my test files had invalid UTF-8 in them, and it took me quite a while to discover why I got different results from the two methods.
Once the problem files were re-encoded to valid UTF-8, both methods produce identical results.
Here is the new faster method (4+ times faster)... using split(...
#!/bin/bash
pat='xx|yy|zz'
awk -v vpat="$pat" 'BEGIN {
split(vpat, pat, "|"); for(i in pat) pz++
}
{ if (NF) { for( p in pat ) { ct[p]+=(split( $0, A, pat[p] ) -1) }}
}
END { print " count pattern"
for (p=1; p<=pz; p++) { printf "%6d %s\n", +ct[p], pat[p] }
}' file
Here is the Slower method. using match( substr(...
#!/bin/bash
# Count occurrences of multiple non-overlapping string patterns
awk 'BEGIN {
pattern[1]="xx"
pattern[2]="yy"
pattern[3]="zz"
}
{ for( p in pattern ) {
LHB=0; RSTART=RLENGTH=1
while( match( substr( $0, LHB+=(RSTART+RLENGTH-1)), pattern[p] )){
count[p]++
}
}
} END {
print "occurs pattern"
for (p in pattern) {
printf "%6d %s\n", +count[p], pattern[p]
}
}' file
Here is the input file
xx xx xx
xx yy xx
The output is as follows:
occurs pattern
5 xx
1 yy
0 zz