0

I have a file like this:

1 foo
1 bar
1 someOtherString
2 contains bar and more
2 test
3 contains a random string
4 some other string
2 don't remove this line bar
2 but remove this line
14 keep this line
21 and also this line
7 bar

From this file, i want to get this file:

1 foo
1 bar
2 contains bar and more
3 contains a random string
4 some other string
2 don't remove this line bar
14 keep this line
21 and also this line
7 bar

Basically:

  • keep all lines that do not start with "1 " nor "2 "
  • keep all lines that contain either "foo" or "bar"
  • remove all other lines
  • keep the order intact
3
  • 1
    Welcome to the site. Please indicate what you already tried and where you faced problems. That way you can avoid receiving solutions that you already know won't work.
    – AdminBee
    Oct 15, 2021 at 13:40
  • Do you need to do this with grep or are you open to other standard tools? Also, in the actual use case: (1) Is the "exclusion" pattern also actually the first "word" and not simply the start of the line (otherwise, 14 and 21 would be removed in your example), and (2) can the pattern that would mandate keeping the line be part of the first "word" (e.g. is a line starting with 1foo possible)?
    – AdminBee
    Oct 15, 2021 at 14:32
  • @AdminBee I don't care which tool, as long as it works. The exclusion pattern is not "1" but "1 " (that is why "14" should not match). To (2): No. Oct 15, 2021 at 14:48

4 Answers 4

4

Consider the data in a file called file. The requirements to achieve your example results are slightly different to the statements you posted:

  • Keep all lines in file that match either of these criteria:
    • do not start with 1 nor 2 (that's a digit followed by a space)
    • contain foo or bar
  • Remove all other lines
  • Keep the order intact

This can be expressed in perl by matching two expressions and printing the line (in sequence) if either matched:

perl -ne '( !/^(1 |2 )/ or /foo|bar/ ) and print' file

As an afterthought, this entire requirement can be expressed slightly differently:

  • For each line in turn
    • Print the line if it doesn't start with 1 or 2
    • Print the line if it contains foo or bar

And this maps to awk really conveniently:

awk '!/^(1 |2 )/ || /foo/ || /bar/' file

In both cases (perl and awk) the RE ^(1 |2 ) can be simplified by bringing out the common factor and rewriting it as ^[12] :

perl -ne '( !/^[12] / or /foo|bar/ ) and print' file
awk '!/^[12] / || /foo/ || /bar/' file
5
  • Maybe /^(1|2) / for a more concise version?
    – terdon
    Oct 15, 2021 at 16:43
  • @terdon I preferred keeping the pattern described by the OP, but yes in my real code I'd have simplified the common element
    – roaima
    Oct 15, 2021 at 16:44
  • What is different in my listed requirements and the ones you listed? Oct 15, 2021 at 18:48
  • @12431234123412341234123 the way your requirements are read is with an implied and between them: "keep all lines that do not start with "1 " nor "2 "" and then "keep all lines that contain either "foo" or "bar"" and then "remove all other lines". The example (thank you for that) shows that your first two requirements are actually to be joined with or.
    – roaima
    Oct 15, 2021 at 20:16
  • @roaima I still don't see a difference, of course you can formulate it differently but for me this two requirements are the same. Anyway, it works. Oct 18, 2021 at 10:53
2

Using sed:

$ sed '/^[12]\>/ { /foo/ !{ /bar/ !d; }; }' file
1 foo
1 bar
2 contains bar and more
3 contains a random string
4 some other string
2 don't remove this line bar
14 keep this line
21 and also this line
7 bar

The above uses sed to test whether each line starts with either 1 or 2 followed by a non-word character (you may replace \> by a single space if you wish). If it does not, then the line is printed. If it does, the line is tested for the substring foo. If that substring exists, the line is printed. If it does not exist, a similar test is made for bar, and the line is printed if it matches and deleted if it doesn't.

Reading the logic from the d backwards: The line is deleted if it doesn't match bar and does not match foo but starts with 1 or 2.

0

Using Raku (formerly known as Perl_6)

raku -ne '.put if .grep( !/^ [ 1 | 2 ] \s / | / foo | bar / );'  

OR

raku -ne '.put if .grep( { !/^ [ 1 | 2 ] \s /} | / foo | bar / );'  

OR

raku -ne '.put if .grep( (none /^ [ 1 | 2 ] \s /) | / foo | bar / );'  

Sample Input:

1 foo
1 bar
1 someOtherString
2 contains bar and more
2 test
3 contains a random string
4 some other string
2 don't remove this line bar
2 but remove this line
14 keep this line
21 and also this line
7 bar

Sample Output:

1 foo
1 bar
2 contains bar and more
3 contains a random string
4 some other string
2 don't remove this line bar
14 keep this line
21 and also this line
7 bar

Above is a rough translation of @roaima's Perl5 code, although in Raku this is properly a grep operation. In Raku, you can test for the absence of a pattern using either the ! negation operator (shown above with/without a surrounding {...} codeblock), or using a none junction (related to Sets in Raku).

You may note the inclusion of a \s character in the regex. That's because undeclared (\s) whitespace and/or "...unquoted whitespace in a regex is generally ignored..." by default in Raku, making for more readable code. See links below.

https://docs.raku.org/language/regexes#Sigspace
https://raku.org

0

We can use lookarounds profitably for such kind of multi-condition searches:

$ grep -vP '^(?=[12]\h)(?!.*(?:foo|bar))' file

Python can be used with it's re module to perform matching and filtering based on the search result status.

python3 -c 'import re, sys
with open(sys.argv[1]) as f:
  print(*filter(lambda x: re.match(r"(?![12]\s)",x) or re.search(r"foo|bar",x),f),sep="",end="")
' file

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .