1

I have a small problem with awk multiple pattern matching which I cannot figure out. I have the following awk line:

awk '/pat1/{v1=$4; next} /pat2/{v2=$5; next} /pat3/{v3=$6;next} /pat4/{v4=$5; print v1,"    ",v2,"    ",v3"    ",v4}' myfile.out

This gives the result I want (have the mathed results printed on a line every time they match) given that ALL of them match. If one of the patterns is not present then nothing will match.

So if all match I get what I expect:

pat1    pat2    pat3    pat4
pat1    pat2    pat3    pat4
pat1    pat2    pat3    pat4
pat1    pat2    pat3    pat4
.
.
.

The patX values are different in each row!

Is there a way to tell awk to look for these patterns and if they do not appear to leave the place empty?

So for example if in the first instance pat3 and pat4 do not yet appear in the document that is updating, then I should get:

pat1    pat2    
pat1    pat2   pat3    ------> (here let's assume that pat3 has made an appearange)
pat1    pat2   pat3    pat4 ------> (here pat4 started to appear too)
pat1    pat2   pat3    pat4
pat1    pat2   pat3    pat4
.
.
.

Can this be done with awk?

Edit: Here is the two example scenarios I am facing. My files start off as empty and then they fill with data and I need to filter some patterns from them. Not all the patterns appear from the beginning. So the file will start off as:

some text here pat1
some more text here

some more text here pat2

some more text here and pat3


If I use the awk command above it will give an empty result because pat4 is not present yet! As time goes by it will eventually appear.

some text here pat1
some more text here

some more text here pat2

some more text here and pat3

some more text here pat4

some text here pat1
some more text here

some more text here pat2

some more text here and pat3

some more text here pat4

some text here pat1
some more text here

some more text here pat2

some more text here and pat3

some more text here pat4

The result of the awk command looks as expected:

pat1      pat2      pat3     pat4
pat1      pat2      pat3     pat4
pat1      pat2      pat3     pat4

However, at the beginning I would like to obtain the result:

pat1    pat2    pat3   

I hope this is clearer now ( I have rewritten and tested the awk command above to make it simpler for this example).

2
  • 2
    Please edit your question to provide sample input and get rid of the .s, just give us some concise, testable sample input and expected output that we can copy/paste to test with. The output you provided is only half of an example, we need the input too. Btw your code uses patt3 (2 ts) but output pat3 (1 t), please make sure the code you post matches the sample input/output you post.
    – Ed Morton
    Sep 28 at 13:46
  • @EdMorton sorry for not including an example. I have edited the question for better clarity (I hope!). I have also edited the awk command to make it clearer too.
    – lucian
    Sep 28 at 17:16
1

It seems like you'd need something like:

$ cat tst.awk
BEGIN { OFS="     " }

{ sub(/\r$/,"") }

( ($NF ~ /pat1/) && (state == 0) ) ||
( ($NF ~ /pat2/) && (state == 1) ) ||
( ($NF ~ /pat3/) && (state == 2) ) ||
( ($NF ~ /pat4/) && (state == 3) ) {
    v[++state] = $NF
}

state == 4 {
    print v[1], v[2], v[3], v[4]
    state = 0
}

$ awk -f tst.awk file
pat1     pat2     pat3     pat4
pat1     pat2     pat3     pat4
pat1     pat2     pat3     pat4
6
  • thank you for your answer. I am trying to adapt it to my particular case but I am having trouble understanding it. What is the purpose of state and where does it increment? also what does this $NF ~ /pat1/ do?
    – lucian
    Sep 28 at 19:17
  • 1) state is the state variable in the Finite State Machine I implemented for you to keep track of which pats have matched and the next one expected to match. 2) It gets incremented in v[++state] = $NF which gets executed every time the next expected pat* is matched. 3) $NF ~ /pat1/ does a partial regexp comparison of pat1 against the last space-separated field on each line.
    – Ed Morton
    Sep 28 at 19:21
  • thank you! I'll keep trying to see if I get it working!
    – lucian
    Sep 28 at 19:27
  • Obviously it works for the input you posted so if you've now realized that the input you posted isn't truly representative of your real input then you can always accept and answer to this question and then ask a new question with better input.
    – Ed Morton
    Sep 28 at 20:00
  • found what I needed in one of your other answers! cheers
    – lucian
    Sep 28 at 20:13
1

Using Raku (formerly known as Perl_6)

raku -e 'my @a; my @pat = <<pat1 pat2 pat3 pat4>>; for lines() { for @pat -> $i { @a.push( m[$i] // " __ " ) };}; .put unless $_ eq " __   __   __   __ " for @a.rotor(4);'

Answering in Raku because of a niggling question: what if a line contains more-than-one desired pattern? Or any patterns appear out of order? The code below handles those cases (also removing lines where no patterns are found).

See the last few lines of the Sample Input, and the resulting extracted patterns in the last few lines of the Sample Output. (Note, I've removed blank lines from samples, below):

Sample Input:

some text here pat1
some more text here
some more text here pat2
some more text here and pat3
some more text here
some text here pat1
some more text here
some more text here pat2
some more text here and pat3
some more text here pat4
some text here pat1
some more text here
some more text here pat2
some more text here and pat3
some more text here pat4
some more text here pat1 pat2
some more text here pat1 pat2 pat3 pat4

Sample Output:

pat1  __   __   __ 
 __  pat2  __   __ 
 __   __  pat3  __ 
pat1  __   __   __ 
 __  pat2  __   __ 
 __   __  pat3  __ 
 __   __   __  pat4
pat1  __   __   __ 
 __  pat2  __   __ 
 __   __  pat3  __ 
 __   __   __  pat4
pat1 pat2  __   __ 
pat1 pat2 pat3 pat4

Note: a common case that isn't handled by the code above is the case where a single pattern is found in multiple copies on a line. The code above has no mechanism to count occurrences, it will just tell you if a pattern is "seen" or not. Example below:

echo "text here pat2 pat2 pat2 pat4" | raku -e 'my @a; my @pat = <<pat1 pat2 pat3 pat4>>; for lines() { for @pat -> $i { @a.push( m[$i] // " __ " ) };}; .put unless $_ eq " __   __   __   __ " for @a.rotor(4);'

Duplicated Pattern Output:

__  pat2  __  pat4

http://raku.org

0

Perhaps just use the END clause to print out your results.

    awk '/pat1/{v1=$4; next} /pat2/{v2=$5; next} /pat3/{v3=$6; next} /pat4/{v4=$5;} END{ print v1,"    ",v2,"    ",v3"    ",v4 }' myfile.out
1
  • 1
    I just tried it but it only gives me the last instance. I need them all! As the file updates with new data this awk should catch all the patXs.
    – lucian
    Sep 28 at 17:48
0

While trying to adapt @EdMorton's answer to my problem I found the info that I needed from an older thread where he contributed with an answer and managed to solve the problem entirely. here is my solution:

awk '/pat1/{v1=$4; next}{v1="xxx"} /pat2/{v2=$5; next}{v2="xxx"} /pat3/{v3=$6;next}{v3="xxx"} /pat4/{v4=$5}{v4="xxx"} {print v1,"    ",v2,"    ",v3"    ",v4}' myfile.out

Now when one of the patterns doesn't exist yet I will get an xxx in its place. I can replace the xxx with a single space character or any other value.

Thank you for all your suggestions and help!

P.S.: I find that sometimes it is not so easy to provide a sample input to explain the problem. sorry if that lead to frustration!

0

Here's a solution using bash arrays and grep:

#!/usr/bin/env bash

declare -a pats
pats=('pat1' 'pat2' 'pat3' 'pat4')

# find the width of the widest pattern:

wid=0
for pat in "${pats[@]}"
do
  if [[ ${#pat} > $wid ]]; then wid=${#pat}; fi
done

# The above might give misleading widths if your patterns are 
# regexes that are signifcantly shorter or longer than the text
# they might match, such as "[a-z][^[[:blank:]]]*[a-z]"

# If you prefer, specify a constant width for all columns here:

wid=10

pad() {
  # right-pad string $1 to length $2
  # if $1 is overlength, do nothing
  local fmt="$(printf "%%-%ds" $2)"
  printf "$fmt" "$1"
}

while IFS= read ln
do
  for pat in "${pats[@]}"
  do
    txt="$(grep -o "$pat" <<< "$ln" || echo '----')"
    printf "%s" "$(pad "$txt" $wid)"
    printf "  "
  done
  printf "\n"
done

Input:

some text here pat1
some more text here
some more text here pat2
some more text here and pat3
some more text here
some text here pat1
some more text here
some more text here pat2
some more text here and pat3
some more text here pat4
some text here pat1
some more text here
some more text here pat2
some more text here and pat3
some more text here pat4
some more text here pat1 pat2
some more text here pat1 pat2 pat3 pat4
some more text here pat4 pat3 pat2 pat1

Output:

pat1        ----        ----        ----        
----        ----        ----        ----        
----        pat2        ----        ----        
----        ----        pat3        ----        
----        ----        ----        ----        
pat1        ----        ----        ----        
----        ----        ----        ----        
----        pat2        ----        ----        
----        ----        pat3        ----        
----        ----        ----        pat4        
pat1        ----        ----        ----        
----        ----        ----        ----        
----        pat2        ----        ----        
----        ----        pat3        ----        
----        ----        ----        pat4        
pat1        pat2        ----        ----        
pat1        pat2        pat3        pat4        
pat1        pat2        pat3        pat4        

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.