85

I would like to get the multi pattern match with implicit AND between patterns, i.e. equivalent to running several greps in a sequence:

grep pattern1 | grep pattern2 | ...

So how to convert it to something like?

grep pattern1 & pattern2 & pattern3

I would like to use single grep because I am building arguments dynamically, so everything has to fit in one string. Using filter is system feature, not grep, so it is not an argument for it.


Don't confuse this question with:

grep "pattern1\|pattern2\|..."

This is an OR multi pattern match.

75

agrep can do it with this syntax:

agrep 'pattern1;pattern2'

With GNU grep, when built with PCRE support, you can do:

grep -P '^(?=.*pattern1)(?=.*pattern2)'

With ast grep:

grep -X '.*pattern1.*&.*pattern2.*'

(adding .*s as <x>&<y> matches strings that match both <x> and <y> exactly, a&b would never match as there's no such string that can be both a and b at the same time).

If the patterns don't overlap, you may also be able to do:

grep -e 'pattern1.*pattern2' -e 'pattern2.*pattern1'

The best portable way is probably with awk as already mentioned:

awk '/pattern1/ && /pattern2/'

With sed:

sed -e '/pattern1/!d' -e '/pattern2/!d'

Please beware that all those will have different regular expression syntax.

  • 1
    The agrep syntax is not working for me... which version was it introduced in? – Raman Sep 5 '16 at 22:15
  • @Raman 2.04 from 1992 already had it. I've no reason to believe it wasn't there from the start. Newer (after 1992) versions of agrep can be found included with glimpse/webglimpse. Possibly you have a different implementation. I had a mistake for the ast-grep version though, the option for augmented regexps is -X, not -A. – Stéphane Chazelas Sep 6 '16 at 5:55
  • @StéphaneChazelas Thanks, I have agrep 0.8.0 on Fedora 23. This appears to be a different agrep than the one you reference. – Raman Sep 6 '16 at 6:37
  • 1
    @Raman, yours sounds like TRE agrep. – Stéphane Chazelas Sep 6 '16 at 7:01
  • 2
    @Techiee, or just awk '/p1/ && /p2/ {n++}; END {print 0+n}' – Stéphane Chazelas Jun 28 '17 at 20:23
19

You didn't specify grep version, this is important. Some regexp engines allow multiple matching groupped by AND using '&' but this is non-standard and non-portable feature. But, at least GNU grep doesn't support this.

OTOH you can simply replace grep with sed, awk, perl, etc. (listed in order of weight increasing). With awk, the command would look like

awk '/regexp1/ && /regexp2/ && /regexp3/ { print; }'

and it can be constructed to be specified in command line in easy way.

  • 3
    Just remember that awk uses ERE's, e.g. the equivalent of grep -E, as opposed to the BRE's that plain grep uses. – jw013 Nov 10 '12 at 9:42
  • 3
    awk's regexes are called EREs, but in fact they're a bit idiosyncratic. Here are probably more details than anyone cares for: wiki.alpinelinux.org/wiki/Regex – dubiousjim Nov 10 '12 at 15:35
  • Thank you, grep 2.7.3 (openSUSE). I upvoted you, but I will keep question open for a while, maybe there is some trick for grep (not that I dislike awk -- simply knowing more is better). – greenoldman Nov 10 '12 at 15:42
  • 2
    The default action is to print the matching line so the { print; } part isn't really necessary or useful here. – tripleee Apr 20 '17 at 11:58
7
+50

If patterns contains one pattern per line, you can do something like this:

awk 'NR==FNR{a[$0];next}{for(i in a)if($0!~i)next}1' patterns -

Or this matches substrings instead of regular expressions:

awk 'NR==FNR{a[$0];next}{for(i in a)if(!index($0,i))next}1' patterns -

To print all instead of no lines of the input in the case that patterns is empty, replace NR==FNR with FILENAME==ARGV[1], or with ARGIND==1 in gawk.

These functions print the lines of STDIN which contain each string specified as an argument as a substring. ga stands for grep all and gai ignores case.

ga(){ awk 'FILENAME==ARGV[1]{a[$0];next}{for(i in a)if(!index($0,i))next}1' <(printf %s\\n "$@") -; }
gai(){ awk 'FILENAME==ARGV[1]{a[tolower($0)];next}{for(i in a)if(!index(tolower($0),i))next}1' <(printf %s\\n "$@") -; }
7

This is not a very good solution but illustrates a somewhat cool "trick"

function chained-grep {
    local pattern="$1"
    if [[ -z "$pattern" ]]; then
        cat
        return
    fi    

    shift
    grep -- "$pattern" | chained-grep "$@"
}

cat something | chained-grep all patterns must match order but matter dont
3

git grep

Here is the syntax using git grep combining multiple patterns using Boolean expressions:

git grep --no-index -e pattern1 --and -e pattern2 --and -e pattern3

The above command will print lines matching all the patterns at once.

--no-index Search files in the current directory that is not managed by Git.

Check man git-grep for help.

See also:

For OR operation, see:

1

Here's my take, and this works for words in multiple lines:

Use find . -type f followed by as many
-exec grep -q 'first_word' {} \;
and the last keyword with
-exec grep -l 'nth_word' {} \;

-q quiet / silent
-l show files with matches

The following returns list of filenames with words 'rabbit' and 'hole' in them:
find . -type f -exec grep -q 'rabbit' {} \; -exec grep -l 'hole' {} \;

0

ripgrep

Here is the example using rg:

rg -N '(?P<p1>.*pattern1.*)(?P<p2>.*pattern2.*)(?P<p3>.*pattern3.*)' file.txt

It's one of the quickest grepping tools, since it's built on top of Rust's regex engine which uses finite automata, SIMD and aggressive literal optimizations to make searching very fast.

See also related feature request at GH-875.

-2

To find ALL of the words (or patterns), you can run grep in FOR loop. The main advantage here, is searching from a list of regexs.

EDIT my answer with a real example:

# search_all_regex_and_error_if_missing.sh 

find_list="\
^a+$ \
^b+$ \
^h+$ \
^d+$ \
"

for item in $find_list; do
   if grep -E "$item" file_to_search_within.txt 
   then
       echo "$item found in file."
   else
       echo "Error: $item not found in file. Exiting!"
       exit 1
   fi
done

Now let's run it on this file:

hhhhhhhhhh

aaaaaaa

bbbbbbbbb

ababbabaabbaaa

ccccccc

dsfsdf

bbbb

cccdd

aa

caa

# ./search_all_regex_and_error_if_missing.sh

aaaaaaa aa

^a+$ found in file.

bbbbbbbbb bbbb

^b+$ found in file.

hhhhhhhhhh

^h+$ found in file.

Error: ^d+$ not found in file. Exiting!

  • 1
    Your logic is faulty -- I asked for ALL operator, your code works as OR operator, not AND. And btw. for that (OR) is much easier solution given right in the question. – greenoldman Aug 14 '18 at 22:18
  • @greenoldman The logic is simple: The for will loop on ALL of the words/patterns in the list, and if it is found in file - will print it. So just remove the else if you don't need action in case word was not found. – Noam Manos Aug 16 '18 at 15:07
  • I understand your logic as well as my question -- I was asking about AND operator, meaning the file is only a positive hit if it matches pattern A and pattern B and pattern C and... AND In you case file is positive hit if it matches pattern A or pattern B or... Do you see the difference now? – greenoldman Aug 17 '18 at 6:19
  • @greenoldman not sure why you think this loop does not check AND condition for all patterns? So I've edited my answer with a real example: It will search in file for all regex of list, and on the first one which is missing - will exit with error. – Noam Manos Aug 19 '18 at 15:04
  • You have it right in front of your eyes, you have positive match just after first match is executed. You should have "collect" all outcomes and compute AND on them. Then you should rewrite the script to run on multiple files -- then maybe you realize that the question is already answered and your attempt does not bring anything to the table, sorry. – greenoldman Aug 20 '18 at 5:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.