3

I would like to write a simple script that would use grep to search through a list of files. The current code looks something similar to this:

a= file1.txt 

b= file2.txt

for

do

grep '$a.*$b' /root/listoffiles/*php >> found.txt

done

file1 and file2 are both word lists. So this would search through the destination having multiple criteria. For example $a could be "hello" and $b "world" or the second line could be "red" and "tshirt". If it finds anything with two matching criteria then it would just save it into the file. The loop that I am using is not working and not efficient enough. I would like to get only the results for meeting criteria. Any suggestions?

  • is order important in matching? for ex: world should always be after hello or can it occur before also? – Sundeep Mar 13 '17 at 15:11
  • also, hello and tshirt (i.e word from file1 line1 and word from file2 line2) should also be matched? or is it only same line numbers from both files – Sundeep Mar 13 '17 at 15:14
  • file1 should be first then file2. My main problem is the loop itself because I have to read 2 files. for example I was able to create a loop with a single file where it greps through the list and trying to find matching words in the destination folder. Thanks – Stephen Mar 13 '17 at 15:17
  • line 1 in file1 and line 1 in file2 should match. – Stephen Mar 13 '17 at 15:18
  • 1
    ok, try this single command instead of script, grep -f <(pr -mts'.*' file1.txt file2.txt) /root/listoffiles/*php >> found.txt – Sundeep Mar 13 '17 at 15:20
2

You would first have to escape the special characters in those two files (if any) and then join them via e.g. paste to end up with a list of patterns that you would then use via grep's -f ... and unless you have too many files you don't need a loop as grep accepts multiple arguments:

paste -d'.*' <(sed 's/[.[\*^$]/\\&/g' file1.txt) /dev/null \
<(sed 's/[.[\*^$]/\\&/g' file2.txt) | grep -f - /root/listoffiles/*php >> found.txt
  • hi, This seems to be a working solution, but. What if I got in file1 something like yellow-stone and 2.1.2. in file2. How can I ignore the special characters? So it would search yellow stone 2 1 2 in this order – Stephen Mar 14 '17 at 9:27
  • @Stephen - I'm afraid you'll have to elaborate on that (and not only that because your post is already unclear - there are two interpretations of your question and, as a result, two different answers...) Anyway, my current solution will search for for a string matching yellow-stone.*2\.1\.2 - I don't quite get what you mean by "ignore special characters" and "search yellow stone 2 1 2 in this order" (why spaces there ?) How do you define "special characters" and what do you mean by "ignoring them" ? – don_crissti Mar 14 '17 at 12:07
  • for example ignore the "-" and "." between the strings – Stephen Mar 14 '17 at 12:53
  • @Stephen - let me put it this way: given two strings (from the same line in each of those two files) yellow-stone and 2.1.2 what is the actual regex that those two strings are expected to produce ? – don_crissti Mar 14 '17 at 14:06
0

Build a pattern from the word lists. From file1.txt containing e.g. one, un, eins and file2.txt containing two, deux, zwei, you need to get the regular expression (in ERE syntax)

(one|un|eins).*(two|deux|zwei)

If the word lists don't contain any characters that are special in a regex then all you need to do is replace newlines by | (the “or” operator) except the final one.

words1=$(tr '\n' '|' <file1.txt); words1=${words1%\|}
words2=$(tr '\n' '|' <file2.txt); words2=${words2%\|}
grep -E "($words1).*($words2)"

If the word lists might contain special characters then you need to quote them.

words1=$(<file1.txt tr '\n' '|' | sed 's/[][\\().*?+|]/\\&/g'); words1=${words1%\|}
words2=$(<file2.txt tr '\n' '|' | sed 's/[][\\().*?+|]/\\&/g'); words2=${words2%\|}
grep -E "($words1).*($words2)"

Note that if you don't care about the order of the words and you can accept overlapping matches (e.g. accept twone as containing both one and two) then all you need is two grep invocations:

grep -f file1.txt | grep -f file2.txt

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.