5

Usually grep outputs any lines that match a pattern. I want to be able to find lines that match the pattern multiple times. For example, if my search pattern was "foo", then:

foo bar      # Would not be matched
foo foo bar  # Would be matched
bar foofoo   # Would be matched
foobarfoo    # Would be matched

Is there a way I can tell grep to find only lines that contain multiple matches of my search pattern?

5
  • Everything I read when I was looking this up was explaining how to use multiple patterns at the same time. I guess I just am looking up the wrong terms... Jul 1, 2020 at 14:30
  • grep has no such option. grep "${pattern}.*${pattern}" does what you want, though. Jul 1, 2020 at 14:31
  • @HaukeLaging I didn't think about putting it all in one pattern! That makes perfect sense. Do you want to make that an answer? Jul 1, 2020 at 14:33
  • If the pattern is bob, would you say it matches twice on bobob? In another words, are the matches required to be disjoint or not? Jul 2, 2020 at 14:20
  • Same for <.*> on an input like <a<b>c> where you could say there's only one or there are 4 matches (<a<b>, <a<b>c>, <b>, <b>c>) Jul 2, 2020 at 14:22

3 Answers 3

12
grep -E "(foo.*){2}" file

This matches at least 2 times on each line of file or output, you can give minimum number of matches.

3

If you want to match all lines that matches any string twice:

grep '\(.\{1,\}\).*\1'

You can change the length to match by changing 1,:

seq 10000 | grep '\(.\{2,\}\).*\1'

This uses Basic Regular Expression (BRE) and should therefore work on any POSIX compliant grep.

If you convert the regexp to use non-greedy regexp (not supported everywhere) it does not seem to speed up matching:

grep -E '(..*?).*?\1'

The graph shows runtime in seconds of runs with and without non-greedy on 100 lines of each n numbers (~ line length).

greedy() {
  a=`seq $1`;
  yes $a | head -n 100 | grep '\(.\{1,\}\).*\1' | LC_ALL=C wc;
}
nongreedy() {
  a=`seq $1`;
  yes $a | head -n 100 | grep -E '(..*?).*?\1' | LC_ALL=C wc;
}
export -f greedy
export -f nongreedy
parallel --jl my.log {2} {1}000 {2} ::: {1..100} ::: greedy nongreedy

enter image description here

1
1
grep "foo.*foo" file.txt

That will only return lines where foo appears two or more times. It won't return lines where it only appears once.

The above will work without quotes most of the time but in other cases where there are filenames in the directory that match the glob such as foo.barfoo, for example, it is necessary to quote the regex which is why I have edited my answer to include double quotes.

9
  • 4
    You must quote the regex. Otherwise very unpleasant things can happen. Jul 1, 2020 at 14:34
  • @HaukeLaging That depends on the context. In this case, it works whether it is quoted or not. Whitespace and special characters aren't an issue here. Jul 1, 2020 at 14:36
  • 1
    @NasirRiley Yes, it works just fine in this case, but other people who stumble across this question might be in a scenario where it would cause an issue... Jul 1, 2020 at 14:37
  • 3
    @NasirRiley, the unquoted version falls apart the moment someone has files that happen to match it as a filename. Or if you run zsh, since it complains about unmatched globs by default (zsh: no matches found: foo.*foo). And that's a good feature. Better to just quote anything that looks like a glob if it isn't meant as one.
    – ilkkachu
    Jul 1, 2020 at 15:11
  • 8
    No, it must be quoted because * is a special character in the shell. foo.*foo will be expanded by the shell to the list of filenames in the current directory that start with foo. and end in foo and if there's no matching file, it will either trigger an error (csh, tcsh, zsh, fish, bash -O failglob) or expand to nothing (older fish, zsh -o nullglob, bash -O nullglob) or be left as-is. Jul 1, 2020 at 15:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .