22

How to filter out 2 lines for each line matching the grep regex?
this is my minimal test:

SomeTestAAAA
EndTest
SomeTestABCD
EndTest
SomeTestDEFG
EndTest
SomeTestAABC
EndTest
SomeTestACDF
EndTest

And obviously I tried e.g. grep -vA 1 SomeTestAA which doesn't work.

desired output is:

SomeTestABCD
EndTest
SomeTestDEFG
EndTest
SomeTestACDF
EndTest
1
  • grep -v 'SomeTextAA' | uniq ?
    – user14755
    Commented Aug 30, 2015 at 0:37

7 Answers 7

14

You can use grep with -P (PCRE) :

grep -P -A 1 'SomeTest(?!AA)' file.txt

(?!AA) is the zero width negative lookahead pattern ensuring that there is no AA after SomeTest.

Test :

$ grep -P -A 1 'SomeTest(?!AA)' file.txt 
SomeTestABCD
EndTest
SomeTestDEFG
EndTest
SomeTestACDF
EndTest
6
  • what's the escape character for dots? like Some.Test.AA?
    – Behrooz
    Commented Aug 29, 2015 at 23:08
  • 1
    @Behrooz Escape dots by \. so grep -P -A 1 'SomeTest\.(?!AA)' file.txt or grep -P -A 1 'SomeTest(?!\.AA)' file.txt
    – heemayl
    Commented Aug 29, 2015 at 23:10
  • This works in this particular case because in OPs sample lines come in pairs of SomeTest*\nEndTest so you are actuallly grepping all lines matching SomeTest* but not SomeTestAA + one line of context after the match. Add some more lines to the input (e.g add a line foobar after each EndTest line) then try it again. Commented Aug 29, 2015 at 23:17
  • 1
    @don_crissti that's true, I already worked around that.
    – Behrooz
    Commented Aug 29, 2015 at 23:20
  • @Behrooz - care to share with us how you worked around that and maybe answer my comment under your question ? Commented Aug 29, 2015 at 23:32
9

You can use GNU sed's d command to delete a line, and prefix it with /pat/,+N to select lines matching the pattern and the subsequent N lines. In your case, N = 1 since you only want to delete the single subsequent line after a matching line:

sed -e '/SomeTestAAAA/,+1d'
6

Tried with below GNU sed command and it worked fine

command

sed  '/SomeTestAA/,+1d' filename

output

SomeTestABCD
EndTest
SomeTestDEFG
EndTest
SomeTestACDF
EndTest
1
  • This worked beautifully! I needed to make +1d into +2d to also get rid of annoying whitespace, but otherwise 10/10
    – MartinJH
    Commented Oct 5, 2021 at 9:22
5

Here's a sed solution (with -n i.e. no auto-printing) that works with arbitrary input:

sed -n '/SomeTestAA/!p          # if line doesn't match, print it
: m                             # label m
//{                             # if line matches
$!{                             # and if it's not the last line
n                               # empty pattern space and read in the next line
b m                             # branch to label m (so n is repeated until a
}                               # line that's read in no longer matches) but
}                               # nothing is printed
' infile

so with an input like

SomeTestAAXX
SomeTestAAYY
+ one line
SomeTestONE
Message body
EndTest
########
SomeTestTWO
something here
EndTest
SomeTestAABC
+ another line
SomeTestTHREE
EndTest
SomeTestAA
+ yet another line

running

sed -n -e '/SomeTestAA/!p;: m' -e '//{' -e '$!{' -e 'n;b m' -e '}' -e'}' infile

outputs

SomeTestONE
Message body
EndTest
########
SomeTestTWO
something here
EndTest
SomeTestTHREE
EndTest

that is, it removes exactly the lines that grep -A1 SomeTestAA infile would select:

SomeTestAAXX
SomeTestAAYY
+ one line
--
SomeTestAABC
+ another line
--
SomeTestAA
+ yet another line
5
  • Interesting. I didn't realize that // matched /SomeTestAA/. I thought, in this case, it would have matched the negated expression: /SomeTestAA/!. (+1)
    – Peter.O
    Commented Aug 30, 2015 at 7:04
  • 1
    @Peter.O - thanks ! No, per the specs, an empty RE should always match the last RE used in the last command; the ! is not part of the RE, it's a sed thing. Commented Aug 30, 2015 at 8:16
  • @Peter.O What do you mean by RE? Commented Jul 7, 2020 at 3:55
  • 1
    @ConstantFun - RE is short for Regular Expression Commented Jul 14, 2020 at 17:12
  • 1
    @Peter.O Thanks for clarifying. I feel kind of stupid now. xD Commented Jul 16, 2020 at 15:50
4

One option is to use perl compatible regular expression grep:

pcregrep -Mv 'SomeTestAA.*\n' file

The option -M allows pattern to match more then one line.

4
  • 1
    @don_crissti Both lines will be removed. OP's specification doesn't cover this case.
    – jimmij
    Commented Aug 29, 2015 at 23:35
  • It's quite obvious the OPs sample & question don't cover such cases, I'm just curious to know how this works (I'm not familiar with pcre) because with an odd number of consecutive lines that match, this works (it removes the context line too) and with an even number of consecutive lines that match, it fails (it doesn't remove the context line after). Commented Aug 30, 2015 at 8:22
  • Given that (GNU) grep already supports PCRE (via the -P option), what is the advantage of using pcregrep?
    – arielf
    Commented Sep 11, 2015 at 4:44
  • @arielf grep doesn't support -M option.
    – jimmij
    Commented Sep 11, 2015 at 19:39
3

Using standard sed:

$ sed '/SomeTestAA/{ N; d; }' file
SomeTestABCD
EndTest
SomeTestDEFG
EndTest
SomeTestACDF
EndTest

The sed script parses the input file line by line, and when a line matches the pattern SomeTestAA, the two sed editing commands N and d are executed. The N command appends the next line of input to the pattern space (the buffer that sed can edit), and the d deletes the pattern space and starts the next cycle.

2

You might have better luck with something that looks at multi-line regions as single records. There's an sgrep which I haven't used much.

There's also awk, where you can set the input record separator, and output record separator, to whatever you like.

pat="^SomeTestAA"
awk  'BEGIN{ RS=ORS="\nEndTest\n"} !/'"$pat/" foo

Most of the awk program is single-quoted, but I change to double quotes at the end so the $pat shell variable can be expanded.

1
  • awk -vpat="^SomeTestAA" -vRS="\nEndTest\n" 'BEGIN{ ORS=RS } $0 !~ pat' file
    – Peter.O
    Commented Aug 30, 2015 at 0:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .