Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

This question already has an answer here:

Over a decade ago I imagined an improved version of grep that would use indexes to make recursive searching faster. I never got around to writing it, but I wonder if somebody else did!

My original idea was:

  • Keep a cached index of word -> [line-numbers] for each file, which would automatically update when the file size or timestamp changes.
  • Use the cache for lightning-fast searches over any files with up-to-date indexes, if the expression can make use of it.
  • For search expressions where the index cannot help, default to grep's usual behaviour.

Although this optimization would be limited to whole-word-searching, I thought it should be grep-compatible, so that it could be dropped in to replace /usr/bin/grep and help speed up various existing scripts that use grep.

There are limitations to this idea:

  1. Only word-searches or searches including certain \\<...\\> terms would be able to benefit from the indexes, and the latter case could be complex to implement.
  2. The time taken to check file stats to ensure the caches are up-to-date might make the time saved on searches minimal.

So do you think it would be worthwhile, or does point 2 make the idea worthless?

And has someone already tried this? If so, do please share.

share|improve this question

marked as duplicate by Gilles, slm, Braiam, strugee, jasonwryan Mar 15 at 2:32

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

1  
The related question unix.stackexchange.com/questions/40206/grep-second-time-faster asks the general question "how to speed up searches" but I am disappointed that the only answer appears to be "pre-cache the files". Should I accept that the bottleneck is disk-access over multiple-files and not actually the cost of regexp processing, or is there a performance gain available at the CPU stage? –  joeytwiddle Mar 14 at 7:07
2  
unix.stackexchange.com/questions/3086/… and unix.stackexchange.com/questions/69299/… address indexed file searches without providing grep compatibility. –  joeytwiddle Mar 14 at 7:13
1  
There's also this one: Does grep use a cache to speed up the searches? –  slm Mar 14 at 7:25

1 Answer 1

This looks promising.

https://code.google.com/p/codesearch/

Code Search is a tool for indexing and then performing regular expression searches over large bodies of source code.

It does not accept the same arguments are grep, notably it does not accept a file list and -r is assumed. However it does support a few of grep's favourite flags.

usage: csearch [-c] [-f fileregexp] [-h] [-i] [-l] [-n] regexp

So it can not act as a drop-in replacement for grep, but under certain conditions a call to grep could be converted into a call to csearch.

The regexp format (RE2, nearly PCRE) also differs from GNU grep. For example the patterns \< and \> cannot be used for start/end word; but instead we can use \b for word-boundary.

To install

On Debian and Debian clones:

apt-get install codesearch

For Ubuntu 12.04 (and other versions before saucy), download it by selecting your arch (i386/amd64) here and then install it with:

$ sudo dpkg -i ~/Downloads/codesearch_*.deb
share|improve this answer
    
Thanks @bersch I didn't realise it was packaged, since I didn't find it in the repository for Ubuntu 12.04. But the saucy version installs fine, so I have noted that in the answer. I have also noted some similarities/differences to the standard grep. –  joeytwiddle Mar 15 at 19:37

Not the answer you're looking for? Browse other questions tagged or ask your own question.