I have FILE_A which has over 300K lines and FILE_B which has over 30M lines. I created a bash script that greps each line in FILE_A over in FILE_B and writes the result of the grep to a new file.
This whole process is taking over 5+ hours.
I'm looking for suggestions on whether you see any way of improving the performance of my script.
I'm using grep -F -m 1 as the grep command. FILE_A looks like this:
and FILE_B is like this:
So with bash I have a while loop that picks the next line in FILE_A and greps it over in FILE_B. When the pattern is found in FILE_B i write it to result.txt.
while read -r line; do grep -F -m1 $line 30MFile done < 300KFile
Thanks a lot in advance for your help.