Remove lines starting with pattern A or B, except when they contain pattern C or D

Question

I have a file like this:

1 foo
1 bar
1 someOtherString
2 contains bar and more
2 test
3 contains a random string
4 some other string
2 don't remove this line bar
2 but remove this line
14 keep this line
21 and also this line
7 bar

From this file, i want to get this file:

1 foo
1 bar
2 contains bar and more
3 contains a random string
4 some other string
2 don't remove this line bar
14 keep this line
21 and also this line
7 bar

Basically:

• keep all lines that do not start with "1 " nor "2 "
• keep all lines that contain either "foo" or "bar"
• remove all other lines
• keep the order intact

Answer 1

Consider the data in a file called file. The requirements to achieve your example results are slightly different to the statements you posted:

• Keep all lines in file that match either of these criteria:
  • do not start with 1 nor 2 (that's a digit followed by a space)
  • contain foo or bar
• Remove all other lines
• Keep the order intact

This can be expressed in perl by matching two expressions and printing the line (in sequence) if either matched:

perl -ne '( !/^(1 |2 )/ or /foo|bar/ ) and print' file

As an afterthought, this entire requirement can be expressed slightly differently:

• For each line in turn
  • Print the line if it doesn't start with 1 or 2
  • Print the line if it contains foo or bar

And this maps to awk really conveniently:

awk '!/^(1 |2 )/ || /foo/ || /bar/' file

In both cases (perl and awk) the RE ^(1 |2 ) can be simplified by bringing out the common factor and rewriting it as ^[12] :

perl -ne '( !/^[12] / or /foo|bar/ ) and print' file
awk '!/^[12] / || /foo/ || /bar/' file

Answer 2

Using sed:

$ sed '/^[12]\>/ { /foo/ !{ /bar/ !d; }; }' file
1 foo
1 bar
2 contains bar and more
3 contains a random string
4 some other string
2 don't remove this line bar
14 keep this line
21 and also this line
7 bar

The above uses sed to test whether each line starts with either 1 or 2 followed by a non-word character (you may replace \> by a single space if you wish). If it does not, then the line is printed. If it does, the line is tested for the substring foo. If that substring exists, the line is printed. If it does not exist, a similar test is made for bar, and the line is printed if it matches and deleted if it doesn't.

Reading the logic from the d backwards: The line is deleted if it doesn't match bar and does not match foo but starts with 1 or 2.

Answer 3

Using Raku (formerly known as Perl_6)

raku -ne '.put if .grep( !/^ [ 1 | 2 ] \s / | / foo | bar / );'  

OR

raku -ne '.put if .grep( { !/^ [ 1 | 2 ] \s /} | / foo | bar / );'  

OR

raku -ne '.put if .grep( (none /^ [ 1 | 2 ] \s /) | / foo | bar / );'  

Sample Input:

1 foo
1 bar
1 someOtherString
2 contains bar and more
2 test
3 contains a random string
4 some other string
2 don't remove this line bar
2 but remove this line
14 keep this line
21 and also this line
7 bar

Sample Output:

1 foo
1 bar
2 contains bar and more
3 contains a random string
4 some other string
2 don't remove this line bar
14 keep this line
21 and also this line
7 bar

Above is a rough translation of @roaima's Perl5 code, although in Raku this is properly a grep operation. In Raku, you can test for the absence of a pattern using either the ! negation operator (shown above with/without a surrounding {...} codeblock), or using a none junction (related to Sets in Raku).

You may note the inclusion of a \s character in the regex. That's because undeclared (\s) whitespace and/or "...unquoted whitespace in a regex is generally ignored..." by default in Raku, making for more readable code. See links below.

https://docs.raku.org/language/regexes#Sigspace
https://raku.org

Answer 4

We can use lookarounds profitably for such kind of multi-condition searches:

$ grep -vP '^(?=[12]\h)(?!.*(?:foo|bar))' file

---

Python can be used with it's re module to perform matching and filtering based on the search result status.

python3 -c 'import re, sys
with open(sys.argv[1]) as f:
  print(*filter(lambda x: re.match(r"(?![12]\s)",x) or re.search(r"foo|bar",x),f),sep="",end="")
' file