grep - how many patterns at maximum

Question

I want to use grep in PHP in an anagram search. I have a wordlist with 600.000 lines of single words (German words).

Now I want to input 7 letters (e.g. ABCDEFG), make all 5040 permutations with them (its 7!) and feed each mostly senseless anagram possibility (e.g. ABCDEFG, ABCDEGF, ABCDGEF, ...) into grep, to check if that word is in my list.

I run it on a Ubuntu 22.04 server with 16 GB RAM, PHP 8.0 and Apache 2.4.

Is it the best way to add all 5040 anagrams as patterns? E.g.:

shell_exec( 'grep "^ABCDEFG$\|^ABCDEGF$\|..." /path/to/wordlist.txt');

Is that good practice? Or would it be better to use a for next loop with 5040 single grep calls and always only one pattern?

I read that grep could take as many patterns as I want but I didn't find any code where there would be as many patterns as in my approach. I guess the maximum is limited by server capabilities but I didn't test that yet. I don't want to slow down the server too much.

Or is my approach not that good at all and best practice would be totally different?

I tried to use sqlite and used the wordlist in a sqlite DB (with OR for each word) but this takes much too long, it's not possible.

Thanks for hints.

Answer 1

Since anagrams don't care about the order of the letters, it might make sense for us to also ignore that. Instead, count the number of each letter in the word to be tested, or even more simply, sort the letters of the word. If the counts or the sorted string match the counts or the sorted version of the key string, then you have an anagram.

A simple one-liner solution can be written in Perl. Take the following input file:

% cat test.txt
manbat
bantam
hello

Then this should print the lines which when sorted contain "aabmnt":

% perl -lne 'chomp; print if join("", sort split //, $_) eq "aabmnt"' test.txt
manbat
bantam

The join("", sort split //, $_) takes the current line $_, separates the characters, sorts them and joins them up again.

Or a bit more usefully, sort the key word within the script too, this should give the anagrams of batman, the key word to be looked for is passed through the environment:

% key=batman perl -lne 'chomp; print if join("", sort split //, $_) eq join("", sort split //, $ENV{key})' test.txt
manbat
bantam