Over a decade ago I imagined an improved version of grep that would use indexes to make recursive searching faster. I never got around to writing it, but I wonder if somebody else did!
My original idea was:
- Keep a cached index of
word -> [line-numbers]
for each file, which would automatically update when the file size or timestamp changes. - Use the cache for lightning-fast searches over any files with up-to-date indexes, if the expression can make use of it.
- For search expressions where the index cannot help, default to grep's usual behaviour.
Although this optimization would be limited to whole-word-searching, I thought it should be grep-compatible, so that it could be dropped in to replace /usr/bin/grep
and help speed up various existing scripts that use grep.
There are limitations to this idea:
- Only word-searches or searches including certain
\\<...\\>
terms would be able to benefit from the indexes, and the latter case could be complex to implement. - The time taken to check file stats to ensure the caches are up-to-date might make the time saved on searches minimal.
So do you think it would be worthwhile, or does point 2 make the idea worthless?
And has someone already tried this? If so, do please share.