6

I have many files I need to search for some string. I'm using grep -rl 'pattern' * to find files that contain the pattern. However, I'm only interested in file count - if string occurs in more than N files, I want grep to stop immediately after hitting Nth match(as searching through the whole file hierarchy is long operation). It would be nice if it returned some meaningful exit code, but if that's impossible then I can just pipe it to wc without a problem.

How can I tell grep to stop searching other files after matching Nth file?

3 Answers 3

10

You can pipe grep result to head.

Note, that in order to ensure stopping after Nth match, you need to using stdbuf to make sure grep don't buffer its output:

stdbuf -oL grep -rl 'pattern' * | head -n10

As soon as head consumed 10 lines, it terminated and grep will receive SIGPIPE because it still output something to pipe while head was gone.

This assumed that no file names contain newline.

8
  • Will that do what OP expects if there are N (e.g. 10) matches in the first file? Just curious if OP is wanting N files as opposed to the first N matches ?
    – Jeff Schaller
    Aug 24, 2015 at 14:12
  • @JeffSchaller -l flag makes grep stop searching file after first match. Yes, I want first N matched files. However, I'm trying out head now and it looks like it doesn't stop execution when encountering Nth result. However, it may only be coincidence, as this grep is generally lengthy. Going to check and see if it really works for me. Aug 24, 2015 at 14:14
  • Ah - missed the -l flag; it does the right thing for me -- sigchld after N lines of output from grep.
    – Jeff Schaller
    Aug 24, 2015 at 14:20
  • @JeffSchaller: Ah, well, a little confusing between the title and the question from OP. -l option work well in this case, stop scanning file after first match.
    – cuonglm
    Aug 24, 2015 at 14:30
  • 3
    You'd want to run grep with stdbuf -oL (if on GNU or FreeBSD) so that it doesn't buffer its output. Otherwise, it's only going to be killed after it writes 2 full buffers, 8k on GNU/Linux which is a very large file list. Aug 24, 2015 at 14:43
1

While this isn't exactly what you asked, I think this may suit your needs considering that using the count of the files can make your script time vary depending on the size of the files, etc, and seeing as how you are trying to limit processor time, you could

timeout -k 1m grep -rl 'pattern' *

This, rather than count how many files are processed to keep the script running fast, will actually shut the command down after an alloted period of time has passed. In my code example, it is 1 minute but that can be changed to seconds (s) hours (h) or even days (d) by replacing the 1m with the correct respective suffex. For example, let's let it run for an hour...

timeout -k 1h grep -rl 'pattern' *

I hope this helps someone!

0

Not pure grep, but with bash:

i=0
grep -rl 'pattern' * | while read l ; do
  i=$(($i+1))
  echo $l
  if [ $i -ge N ] ; then 
    echo "at least N matches"
    break
  fi
done

Will be faster if you have far more files matching than your threshold of N matches.

2
  • oh - copy and paste error - corrected.
    – FelixJN
    Aug 24, 2015 at 13:32
  • For each line there was an error(or so it seems), but that means that script still waited for all lines before processing lines, so no interruption of command I wanted to achieve. Aug 26, 2015 at 12:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .