Grep a directory and return list with line numbers

Question

I'm currently trying to learn more about bash scripting and all of that fun stuff, and I pieced together this little command:

find $path | xargs grep -n $pattern | awk '{print $1}'

While this DOES work, I was wondering if I was reinventing the wheel. Is there a better way to search through a directory, grep the files for a pattern, and return a list with line numbers?

Sorry in advance if this is an inappropriate/off-topic place to post this question.

Answer 1

Many grep variants implement a recursive option. E.g., GNU grep

-R, -r, --recursive
          Read all files under each directory, recursively; this is equivalent to the -d recurse option.

You can then remove find:

grep -n -r $pattern $path | awk '{ print $1 }'

but this keeps more than the line number. awk is printing the first column. This example

src/main/package/A.java:3:import java.util.Map;
src/main/package/A.java:5:import javax.security.auth.Subject;
src/main/package/A.java:6:import javax.security.auth.callback.CallbackHandler;

will be printed as

src/main/package/A.java:3:import
src/main/package/A.java:5:import
src/main/package/A.java:6:import

notice the :import in each line. You might want to use sed to filter the output.

Since a : could be present in the file name you can use the -Z option of grep to output a nul character (\0) after the file name.

grep -rZn $pattern $path | sed -e "s/[[:cntrl:]]\([0-9][0-9]*\).*/:\1/" 

with the same example as before will produce

src/main/package/A.java:3
src/main/package/A.java:5
src/main/package/A.java:6

Answer 2

For the first part, note that xargs only works if there are no whitespace characters or \'" in your file names. See How to search for a word in entire content of a directory in linux for an explanation and an alternative.

Also, always put double quotes around variable substitutions: "$path". Without the double quotes, the shell expands whitespace and wildcards in the value of $path, so using it unquoted breaks if you have whitespace or wildcards in that file name. The same goes for $pattern (just for laughs, try leaving the quotes out and searching for h* in a directory containing files called hi and hello).

If your version of grep has the -r option to traverse directories recursively, you don't need find here. The -r option is present on Linux, FreeBSD, Mac OS X and Cygwin among others. Otherwise:

find "$path" -type f -exec grep -Hn "$pattern" {} + | awk -F: '{print $1 ":" $2}'

I fixed your awk call above, as well, so that it prints only the file name and the line numbers. I also pass the -H option to grep, to ensure that it always prints the file name, even if there happens to be a single file. This code assumes that your file names don't contain : or newlines; if they might, things get complicated, and you'd better either rely on GNU grep's -Z option or process the files individually:

find "$path" -type f -exec sh -c 'for x; do grep -n "$0" <"$x" | awk -v fn="$x" -F: 'print fn ":" $1'; done' "$pattern" {} +

Answer 3

Not sure what exactly you're trying to do here.

find $path | xargs grep -n $pattern | awk '{print $1}'

To me, this translates to find all the files in $path, and search them with numbered lines for pattern $pattern and print the line number and the first word of the line that matches $pattern. (possibly not including $pattern itself)

If that's the case, then you are slightly re-inventing the wheel. You can do all of that directly from the find command, without the extra penalty of the xargs pipe.

find $path -exec grep -n $pattern {} \; -print | awk '{print $1}'

or remove the awk pipe for the entire contents of the line.

Using find's own -exec has the added benefit of gracefully handling whitespace in filenames.

Answer 4

I'd get rid of the grep and use awk:

find $path -type f -print0 | xargs -0 awk "/$pattern/{print FILENAME,FNR}"

But using grep and cut:

find $path -type f -print0 | xargs -0 grep -nH "$pattern" | cut -d: -f1,2

Include the -type f clause so you don't get errors trying to search (in either grep or awk) on non-regular file types (symlinks, directories, sockets). If you read from a pipe or a socket when another program is supposed to be, then you might mess up that program.

The find ... -print0 | xargs -0 gets around having whitespace in the filenames. It is not available on every UNIX system, but is on most.

Answer 5

check the -c and -n useful options too.

Answer 6

Here's what I would do:

• avoid using so many pipes. Whenever possible, use a workaround. Instead of find . | grep -n <> why not use -exec?

  • You can also take advantage of Process Substitution.

Try doing the following:

awk '{print $1}' <(find $path -exec grep -n $pattern {} \;)

NB: This may work as is, or with a slight variation, depending on the shell and the version of find you're using.