Detect pattern repetition in a file

Question

I want to make sure a file follows the following pattern:

... ... ... foo ... foo_KO ... ... 
... bar ... ... ... ... bar_KO ...

The file is extremely long, so, in other words, I want to make sure of the following rule, in sed terms:

Every time ([a-z]*)_KO is matched, there is another field on the same line whose value is \1.

I hoope I'm clear enough.

I'm on Solaris 10, using ksh.

Example:

intput

... ... ... foo ... foo_KO ... ... 
... bar ... ... ... ... bar_KO ...

output

valid

---

input

... ... ... foo ... foo_KO ... ... 
... bar ... ... ... ... bar_KO ...
... fubar_KO ... ... ... ... ... 

output

invalid line 3 

or

unmatched pattern fubar_KO

Answer 1

This will output the line numbers of invalid lines:

sed -n '/\([a-z]\+\).*\1_KO/b;/\([a-z]\+\)_KO.*\1/b;=' input_file

But not deals with multiple “_KO”'s on one line.

---

This should be more reliable and portable:

awk '{for(i=1;i<=NF;i++)if((s=$i)~/^([a-z]+)_KO/){sub(/_KO$/,"",s);o=0;for(j=1;j<=NF;j++)if($j==s)o=1;if(!o)printf"line %d unmatched %s\n",NR,s}}' input_file

---

This is trickier, more efficient and supports multiple patterns, but as depends on sorting, locale settings may affect its success:

ruby -nae '$F.sort!.select!{|v|v=~/^[a-z]+(_KO)?$/};$F.each_with_index{|v,k|puts"line #{$.} unmatched #{v}"if v[/^([a-z]+)_KO$/]and$F[k-1]!=$1}' input_file

---

Basically the same as the above awk solution, but looks less verbose in perl:

perl -nae 'for$k(grep/^[a-z]+_KO$/,@F){print"line $. unmatched $k\n"unless grep{$_ eq substr$k,0,-3}@F}' input_file

Answer 2

Some cryptic perl:

perl -ne '/(\w+)_KO\b/ && !/\b$1(?!_KO)\b/ && print "invalid line $.: missing $1\n"' filename

/(\w+)_KO\b/ find the "_KO" word and capture the non-KO part into variable $1
!/\b$1(?!_KO)\b/ is true if there is NO "standalone" word without _KO (\b is a word boundary)
If both those conditions are true, print the error message.

To determine if the file is "valid", either

output=$(perl -ne '/(\w+)_KO\b/ && !/\b$1(?!_KO)\b/ && print "invalid line $.: missing $!\n"' filename)
[ -z "$output" ] && echo valid

or

perl -ne '
    BEGIN {$count = 0}
    if (/(\w+)_KO\b/ && !/\b$1(?!_KO)\b/) {print "invalid line $.: missing $!\n"; $count++}
    END {$count == 0 && print "valid\n"}
' filename