-3

How to find or search word PATTERN1 and PATTERN2 and print them both later i want to search PATTERN3 and Print the whole content or section between the search PATTERN2/PATTERN3 (Note Pattern1/2/3 occurs many time within the file)

Input file

other lines
...
####Pattern 1####
...
other lines
...
####Pattern 2####
line 1
line 2
line 3
line 4
line 5
line ...
####Pattern 3####
...
other lines
####Pattern 1####
...
other lines
####Pattern 2####
line 1
line 2
line 3
line 4
####Pattern 3####
...
other lines

Output l need is

####Pattern 1####
####Pattern 2####
line 1
line 2
line 3
line 4...etc
####Pattern 3####

closed as unclear what you're asking by RobertL, RalfFriedl, Thomas, G-Man, user88036 Sep 28 '18 at 0:02

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • If PATTERN1 occurs between PATTERN2 and PATTERN3, does PATTERN1 get printed twice? I think not but the specification is not clear, and this case is not in the example. – RobertL Sep 25 '18 at 16:43
  • It's impossible to generate the example output from the example input because this data: "line 4...etc" is nonexistent in the input. Please show the actual output that you need, without annotations in the data. – RobertL Sep 25 '18 at 17:05
  • Hi RobertL and DouglasDD, If Pattern1 exist between Pattern2/3 it shouldn't get printed Also if Pattern3 comes between Pattern1/2 it should not print. So the search should be like it should search Pattern1 first then Pattern2 and Pattern 3 and then print it out. – James bond Sep 25 '18 at 17:41
  • Sorry but I cannot understand your specification. Can you update your example to cover all of these cases? – RobertL Sep 25 '18 at 17:58
1

Now that you've specified that we need to skip all the spurious patterns that can appear out of the expected order, we require something more complicated like a finite state machine.

So here's a much more complicated sample input file that includes all the weird edge cases, and comments:

a
#### Pattern 2 #### Ignore because we have not yet seen Pattern 1
z
#### Pattern 3 #### Ignore because we have not yet seen Pattern 1, 2
b
#### Pattern 1 #### (!!)
#### Pattern 1 #### Don't print 1 in between 1-2
c
d
#### Pattern 2 #### (!!)
e
#### Pattern 1 #### Don't print 1 in between 2-3
#### Pattern 2 #### ?? Don't print 2 in between 2-3 ??
f
#### Pattern 3 #### (!!)
?? Now reset and accept look for the start of a NEW 1,2,3 cycle. Right ??
g
#### Pattern 3 #### Ignore
#### Pattern 2 #### Ignore
#### Pattern 3 #### Ignore
h
#### Pattern 1 #### (!!)
i
#### Pattern 3 #### Don't print 3 in between 1-2
j
#### Pattern 2 #### (!!)
k
l
#### Pattern 3 #### (!!)
m
n

Expected output:

#### Pattern 1 #### (!!)
#### Pattern 2 #### (!!)
e
f
#### Pattern 3 #### (!!)
#### Pattern 1 #### (!!)
#### Pattern 2 #### (!!)
k
l
#### Pattern 3 #### (!!)

So it looks like we need to buid a finite state machine with 3 states. So now we're writting a full fledged script....

#!/usr/bin/perl
use warnings;
use strict;

## Linear state machine 0 --> 1 --> 2 --> 0
my @patterns = (
    qr(Pattern 1),  # state 0: noprint;              match this: noprint && state = 1
    qr(Pattern 2),  # state 1: noprint;              match this:   print && state = 2
    qr(Pattern 3)   # state 2: print (NOT patterns); match this: print   && stage = 0
    );
my $state = 0;

while (<>) {
    if (0 == $state) {
        if (m/$patterns[0]/) {
            ++$state;
        }
    } elsif (1 == $state) {
        if (m/$patterns[1]/) {
            print;
            ++$state;
        }
    } elsif (2 == $state) {
        if (m/$patterns[0]/ || m/$patterns[1]/) {
            # Ignore
        } elsif (m/$patterns[2]/) {
            print;
            $state = 0;
        } else {
            print;
        }
    } else {
        die "Bad programmer! ($state)";
    }
}

Kinda ugly. A more flexible state machine could be implemented with a hash of [$state_num, $pattern_num] => sub { ...action... } where skip/ignore is the default action for any [state, pattern] combination that doesn't appear in the hash. But that's left as an exercise to the enthusiastic reader ;-)

  • Hi DouglasDD, If Pattern1 exist between Pattern2/3 it shouldn't get printed Also if Pattern3 comes between Pattern1/2 it should not print. So the search should be it should search Pattern1 first then Pattern2 and 3 and then print it out. – James bond Sep 25 '18 at 17:43
  • The above Perl you shared does do the job but there are Pattern 3 in between Pattern 1/ 2 which should not get printed. in your case it does get printed which is not needed – James bond Sep 25 '18 at 17:44
  • Hi Douglas in the above sample from you and the output, This is wrong as after Pattern2 Pattern 1 is coming So this output is not expected #### Pattern 1 #### (!!) #### Pattern 2 #### (!!) e f #### Pattern 3 #### (!!) Print should happened only if the sequence is found from Pattern1/2/3 – James bond Sep 26 '18 at 18:49
  • Hmm, I'm getting more confused not less! We're really going to need a complete example full of edge cases, and the correct output. What's confusing me now is when which other patterns can be ignored, f/ex: """In mode 'looking for 3 how do we handle 2? 1?""" And the same for all 9 possible combinations of in mode $x and found pattern $y (for $x in 1..3 and $y in 1..3). – DouglasDD Sep 27 '18 at 8:15
0

Pretty straightforward with sed:

sed -n '/Pattern 1/p; /Pattern 2/,/Pattern 3/p' file
####Pattern 1####
####Pattern 2####
line 1
line 2
line 3
line 4
line 5
line ...
####Pattern 3####
####Pattern 1####
####Pattern 2####
line 1
line 2
line 3
line 4
####Pattern 3####

Given Douglas's input file, we can have this awk that produces his expected output. Like his answer, this is a state machine that uses a couple of logical variables to determine the state:

awk -v p1="#### Pattern 1 ####" \
    -v p2="#### Pattern 2 ####" \
    -v p3="#### Pattern 3 ####" '
        $0 ~ p1 && !have_p1 && !in_p2p3             {have_p1 = $0}
        $0 ~ p2 &&  have_p1 && !in_p2p3             {in_p2p3 = 1; print have_p1; print}
        have_p1 &&  in_p2p3 && $0 !~ p1 && $0 !~ p2 {print}
        $0 ~ p3 &&  in_p2p3                         {in_p2p3 = 0; have_p1 = ""}
' file
#### Pattern 1 #### (!!)
#### Pattern 2 #### (!!)
e
f
#### Pattern 3 #### (!!)
#### Pattern 1 #### (!!)
#### Pattern 2 #### (!!)
k
l
#### Pattern 3 #### (!!)
  • Hi Glenn, it works fine but if Pattern 1 is not there it shouldn't print the next Pattern2/3, In your case it does prints them – James bond Sep 25 '18 at 21:22
  • Why is this requirement not in your question? – glenn jackman Sep 25 '18 at 21:30
  • 1
    I see other comments as well. You need to give us sample input that can encapsulate all your requirements and edge cases. – glenn jackman Sep 25 '18 at 21:36
  • Sorry My bad couldnt capture the cases inthe sample, anyway in the above comments Dogulas has captured the sample the output which is what i want, if that can be done in a more simple way in sed would be of great help to me – James bond Sep 26 '18 at 17:26
  • I feel confident saying this cannot be done in a simple way with sed – glenn jackman Sep 26 '18 at 19:11
0

Here is one way to do this , which goes over the input file twice. First time it goes over and looks at the pattern lines only and from these we determine the state transitions and note the required : 1->2->3.

code=$(< INPUT_FILE \
    sed -e '/Pattern [1-3]/!d;=' |\
    sed -e 'N;s/\n/:/'                     |\
    sed -e '
        /Pattern 1/!d;$d;N
        /\n.*Pattern 1/D
        /Pattern 2/!d;$d;N
        /Pattern 3/!d
    '                                      |\
    sed -e '
        s/:.*//;N;s///;N;s///
        s/\n.*\n/,/;s/$/p/
    '
)

# and having computed the right ranges to print, we now enter the 2nd pass
sed -ne "$code" inp |\
sed -e '/Pattern 1/,/Pattern 2/!b' -e '//!d'

and if you want just one invocation of sed , you can do that too as shown below:

sed -e '
    /Pattern 1/,/Pattern 2/!d    ;# reject non-interesting range
    /Pattern 1/h                 ;# store in hold beginning of range
    /Pattern [23]/H              ;# store pattern 2 and 3 lines in hold too
    /Pattern 2/!d                ;# not at end of range ... delete
    g;/Pattern 3/d               ;# range seen completed, now check whether pattern 3 came
                                 ;# during the 1->2 search, and delete everything & restart
                                 ;# afresh if it did. otherwise, empty the pattern space in
                                 ;# preparation for reading the 2->3
    s/.*//

    :loop                        ;# setup a while(1) loop to read 2->3 range
        $d;N                     ;# read the next line into the pattern space provided
                                 ;# it isnt the last
        /Pattern [12]/d          ;# if we encounter pattern 1/2 then drop everything & start afresh
        /Pattern 3/{             ;# we checked 1/2 didnot come and 3 came
            s/^\n//;H;g;b
        }
    bloop
' input-file.txt

Here is the FSM-based approach using hashes to encode the present-state,next-state relation ship (Mealy-machine formulation):

perl -lne '
    BEGIN {
        sub getLinetype {
            local $_ = @_ ? shift : $_;
            return
                /Pattern 1/ ? "PATT_1" :
                /Pattern 2/ ? "PATT_2" :
                /Pattern 3/ ? "PATT_3" :
                "NON_PATT_123";
        }

        # ---------------------------------------------------------------------
        #     PS      line_type             NS             action
        # ---------------------------------------------------------------------
        $h{ RESET  }{ PATT_1       } = [ "STATE1", sub { @A = ( $_ ) } ];
        $h{ RESET  }{ PATT_2       } = [ "RESET",  sub { @A = ()     } ];
        $h{ RESET  }{ PATT_3       } = [ "RESET",  sub { @A = ()     } ];
        $h{ RESET  }{ NON_PATT_123 } = [ "RESET",  sub { @A = ()     } ];
        # ---------------------------------------------------------------------
        $h{ STATE1 }{ PATT_1       } = [ "STATE1", sub { @A = ( $_ ) } ];
        $h{ STATE1 }{ PATT_2       } = [ "STATE2", sub { push @A, $_ } ];
        $h{ STATE1 }{ PATT_3       } = [ "RESET",  sub { @A = ()     } ];
        $h{ STATE1 }{ NON_PATT_123 } = [ "STATE1", sub {     ;       } ];
        # ---------------------------------------------------------------------
        $h{ STATE2 }{ PATT_1       } = [ "STATE1", sub { @A = ( $_ ) } ];
        $h{ STATE2 }{ PATT_2       } = [ "RESET",  sub { @A = ()     } ];
        $h{ STATE2 }{ PATT_3       } = [ "STATE3", sub { print for splice(@A), $_ } ];
        $h{ STATE2 }{ NON_PATT_123 } = [ "STATE2", sub { push @A, $_ } ];
        # ---------------------------------------------------------------------
        $h{ STATE3 }{ PATT_1       } = [ "STATE1", sub { @A = ( $_ ) } ];
        $h{ STATE3 }{ PATT_2       } = [ "RESET",  sub { @A = ()     } ];
        $h{ STATE3 }{ PATT_3       } = [ "RESET",  sub { @A = ()     } ];
        $h{ STATE3 }{ NON_PATT_123 } = [ "RESET",  sub { @A = ()     } ];
        # ---------------------------------------------------------------------

        $present_state = "RESET";
    }

    my $line_type = getLinetype();

    my $next_state = $h{$present_state}{$line_type}->[0];
    my $action_ref = $h{$present_state}{$line_type}->[1];

    $action_ref->();

    $present_state = $next_state;
' input-file.txt

Not the answer you're looking for? Browse other questions tagged or ask your own question.