0

I have an input file we'll call scan.data. It contains a list of words that I'm going to grep. I've found two different ways of doing this, but I get drastically different results:

IFS=$' '    
find / -type f -exec grep -a -D skip -wo $(<scan.data) '{}' \;

This nets me 3,799 matches.

find / -type f -exec grep -a -D skip -of scan.data '{}' \;

This nets me 18,167 matches.

As far as I understand, setting the IFS allows me to output the contents of the subshell as a list, essentially mirroring what the second case does. So what is the difference between these two that causes such a major difference in matches?

  • your two grep commands aren't similar -- the first is using the -w flag, which will cause it to wrap the pattern(s) in \<...\>. Also, settings IFS isn't enough -- you'll have to also guard against globs with set -f. – mosvy Jun 11 at 19:26
  • Using grep ... -o "$(<scan.data)" (with quotes!) in the 1st command should be no different from the 2nd, though an absolutely pointless and uportable compliction. – mosvy Jun 11 at 19:33
  • @mosvy can you give me some more information? I understand that the commands are different, I don't understand why they are leading to drastically different results. Also, why would I need to worry about globs when using a -w flag, but not an -f flag reading contents of a file? – Y K Jun 11 at 19:45
  • Not only about globs, if your patterns contain {}, they will be interpreted by find ;-) – mosvy Jun 11 at 19:46
  • Try echo 'foobar' | grep -ow foo with and without -w. – mosvy Jun 11 at 19:47
2

Assuming that your command substitution is quoted (if it wasn't, most of the words in scan.data would be taken as filenames to run grep on) and that the first word in the file does not start with a dash (if it did, it may be interpreted as an option), the two grep invocations are mostly equivalent, except for the options used. The first variation may additionally provoke an "argument list too long" error if scan.data contains much data.

  1. The first grep uses -w. This would use the patterns read from scan.data as regular expressions and would try to match complete words on each line in the files that find provides to the utility (the would not match at the start of theodore).
  2. The second grep does not use -w. This would use the patterns read from scan.data as regular expressions and would try to match substrings on each line in the files that find provides to the utility (the would match at the start of theodore).

This may explain the difference in number of matches that you get.

You may possibly want to use -F with grep as well, if you don't want the utility to use the patterns in your scan.data file as regular expressions but as strings (this would be preferable if your pattern file contains characters that are special in regular expressions but that you need to match literally, like ., [, *, etc.).

If you have patterns in a file, consider using -f to read them. Using a command substitution is possible but only in a limited number of cases (no initial dash in first string, and only a moderate amount of strings, as mentioned at the top of this answer already). The existence of the -f option makes the use of a command substitution completely unnecessary.

  • I understand that using the -w flag in the first example would match complete words. But I notice that the -o flag in the second example also appears to limit the output to exact matches. – Y K Jun 12 at 8:40
  • @YK The -o option is used in both examples. That option does not influence matching. It affects the result of the match. Without -o, the whole matching line is returned. With -o, only the match itself is returned (e.g. the in theodore if -w was not used). – Kusalananda Jun 12 at 10:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.