Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.
Sign up to join this community
I got a command foo that output a list of files separated by \n line.
I am using the below command to filter the results by regex content of the files.
foo | xargs grep -l regex
The problem is that some files are very large and the content that I am searching can be found only at the first 10 lines. How can I tell grep to only process the first 10 lines in order to speed this execution?
As commented, to solve half the problem:
foo | xargs grep -m 1 regex
To solve the rest requires a bit more scripting:
foo | xargs sh -c 'for file; do head "$file"; done | grep regex' sh
But that doesn't give you the filenames because grep is reading a stream.
If you have GNU awk:
foo | xargs gawk -v pattern='regex' -v lines=10 -v OFS=':' '
$0 ~ pattern {print FILENAME, FNR, $0}
FNR == lines {nextfile}
'
To just get the filename
$0 ~ pattern {print FILENAME; nextfile}
brew install gawk
– glenn jackman
Jul 10 '20 at 20:14
nextfile after the print too in case the "pattern" appears multiple times in the first 10 lines since the OP seems to just want the file names and even if not then that'd mimic what grep -m 1 does.
– Ed Morton
Jul 12 '20 at 21:53
for i in $(foo); do echo -e "$i \c"; head -n10 $i | grep -c regex; done
will print the filenames from the foo script followed by a number. If the number is zero, then no regex match, and these zero entries in the result can easily be filtered out if not needed.
A modified version of glenn jackman's answer that outputs the filenames:
foo | xargs sh -c 'for file; do head "$file" | grep -l --label="$file" regex; done' sh
headcommand – steeldriver Jul 10 '20 at 17:01-m 1which stops reading the file after the first match. Doesn't help with big files if the pattern is not found. – glenn jackman Jul 10 '20 at 17:40-loption stop after the first match anyhow? – steeldriver Jul 10 '20 at 18:04-m 1is a good solution, but it only solves half of the problem. – Ilya Gazman Jul 10 '20 at 19:51