0

I got a command foo that output a list of files separated by \n line.

I am using the below command to filter the results by regex content of the files.

foo | xargs grep -l regex

The problem is that some files are very large and the content that I am searching can be found only at the first 10 lines. How can I tell grep to only process the first 10 lines in order to speed this execution?

7
  • I'd suggest using the other answer to your earlier question, which can be straightforwardly modified to include a head command – steeldriver Jul 10 '20 at 17:01
  • 2
    You can use grep option -m 1 which stops reading the file after the first match. Doesn't help with big files if the pattern is not found. – glenn jackman Jul 10 '20 at 17:40
  • 2
    @glennjackman doesn't the -l option stop after the first match anyhow? – steeldriver Jul 10 '20 at 18:04
  • @steeldriver yes, but I didn't want to assume the OP only wanted a list of matching filenames. – glenn jackman Jul 10 '20 at 19:30
  • @glennjackman yeah, I agree. -m 1 is a good solution, but it only solves half of the problem. – Ilya Gazman Jul 10 '20 at 19:51
1

As commented, to solve half the problem:

foo | xargs grep -m 1 regex

To solve the rest requires a bit more scripting:

foo | xargs sh -c 'for file; do head "$file"; done | grep regex' sh

But that doesn't give you the filenames because grep is reading a stream.

If you have GNU awk:

foo | xargs gawk -v pattern='regex' -v lines=10 -v OFS=':' '
  $0 ~ pattern {print FILENAME, FNR, $0}
  FNR == lines {nextfile}
'

To just get the filename

  $0 ~ pattern {print FILENAME; nextfile}
2
  • I see you've tagged macos: you can get GNU awk with brew install gawk – glenn jackman Jul 10 '20 at 20:14
  • You should probably add a nextfile after the print too in case the "pattern" appears multiple times in the first 10 lines since the OP seems to just want the file names and even if not then that'd mimic what grep -m 1 does. – Ed Morton Jul 12 '20 at 21:53
0
for i in $(foo); do echo -e "$i \c"; head -n10 $i | grep -c regex; done

will print the filenames from the foo script followed by a number. If the number is zero, then no regex match, and these zero entries in the result can easily be filtered out if not needed.

0

A modified version of glenn jackman's answer that outputs the filenames:

foo | xargs sh -c 'for file; do head "$file" | grep -l --label="$file" regex; done' sh

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.