3

I'm writing a command to find broken Cygwin symlinks.

Right now, I'm using:

egrep -rnw . -e "^\!<symlink>"

But this isn't very efficient. I only care if the file starts with !<symlink>, but grep will keep searching the rest of the file.

Is there any way I can efficiently search files by content prefix?

1

3 Answers 3

3

Building on the other answer and the comment on it,

find . -type f -exec awk '/^!<symlink>/ {print FILENAME} {nextfile}' {} +

Because it uses -exec … +, it will execute awk as few times as possible (based on the maximum argument list size).  If awk finds a line that matches the regular expression (i.e., the search string), it will print the filename.  (You don't need the backslash (\) if you put the string in single quotes.)  Obviously, if you also want to see the content of the line, you can say

{print FILENAME; print}

or

{print FILENAME ": " $0}

And then, unconditionally, it advances to the next file; so it effectively looks at only the first line of each file.  Note: nextfile does not seem to be defined by POSIX; it is probably only in GNU awk.

3
2

How about:

find . -type f -exec head -1v {} + | egrep -B 2 -nw -e "^\!<symlink>"
4
  • 1
    Well first, you're going to need -type f at the very least. Unfortunately, this is extremely inefficient, as a new process is created for each file.
    – PythonNut
    Commented Sep 3, 2015 at 3:39
  • 1
    It also won't report the name of the file the string is found in. Commented Sep 3, 2015 at 8:51
  • @StéphaneChazelas grep -B2 takes care of that (you can further postprocess to get rid of the lines that don't contain file names). Commented Sep 4, 2015 at 21:17
  • @PythonNut This inefficiency is easily solved by switching to -exec … {} + Commented Sep 4, 2015 at 21:18
0

Also building on others ideas.

Have you thought of xargs? It can do some parallelization.

find . -type f -print0 | xargs -0 -P4 head -1 | egrep -wn "^\!<symlink>"

What it does: looks for all regular file and separate the list of files with '\0' instead of white space.
We instruct xargs to understand that '\0' are the separator and to run 4 concurrent processes (adjust that to your number of cores and disk speed). Xargs will run the head command on each file.
Finally grepping the results with your pattern.

Head will still be run plenty of time. But concurrently! Unless you need to do periodic search of this kind, I would not bother trying to optimize it further. The time it's taking to optimize more will not be gained by the faster runtime, if you run it only once anyway.

1
  • 1
    This will display all the first lines that match the pattern, but not the names of the files.  You can get them, on separate lines, by adding a -B1 specification to egrep — but, even then, you need to add -v to head to be safe.  Passing -n to egrep doesn't serve any purpose that I can see. Commented Sep 3, 2015 at 10:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .