I have this command which gets generated by a little script i wrote called "scan", which I use all the time to look for specific content (variable names etc) inside text files:

find "." \
  | awk '{print "\""$0"\""}' \
  | xargs grep -n --exclude-dir=.git "<iframe" \
  | awk 'length($0) < 1000'

here I'm using it to search for iframe start tags, for example.

The last line is telling it to only consider lines under 1000 characters long, since files with lines that long are generally either binaries or concatenated javascript, which I'm generally not interested in. (I can turn that condition off with an option if I do want to see long lines).

Is there a nicer way to modify this to exclude binary files? Or, would I be better off explicitly listing filetypes I want it to ignore, like .mp4, .mp3, .swf, .jpg, .png etc etc?

  • 1
    You could simplify the whole thing to grep -nrI --exclude-dir=git -F '<iframe' . | grep -Ex '.{,1000}' I think (omit the pipe to the second grep if you don't want the length filter; or add suitable --exclude=<pattern> filename conditions instead) – steeldriver Jul 5 '17 at 11:52

man grep tells me:


If the first few bytes of a file indicate that the file contains binary data, assume that the file is of type
TYPE. By default, TYPE is binary, and grep normally outputs either a one-line message saying that a binary file matches, or no message if there is no match. If TYPE is without-match, grep assumes that a >binary file does not match; this is equivalent to the -I option. If TYPE is text, grep processes a binary file as if it were text; this is equivalent to the -a option. Warning: grep --binary-files=text might output binary garbage, which can have nasty side effects if the output is a terminal and if the terminal driver interprets some of it as commands.

so it looks like -I might work for you? (Mind you, it's entirely possible that grep will be confused by any non-ASCII-character early in the file, though.)

Oh, and I think you really don't need that first gawk, find … -exec should work with that, possibly with the -0 option to terminate found file names with a null byte.

  • That's great, thanks! Just adding the -I option to the grep line reduced my example search from 98 seconds to 22 seconds! (They don't normally take that long, but i picked a very large folder to search for benchmarking purposes). cheers – Max Williams Jul 5 '17 at 10:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.