Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Is there such a thing as a full-text indexing engine, that can be queried from the command line and ideally wouldn't require using a gui at all ?

I'm especially interested in indexing my ebooks and papers, so that's a mixture of pdf, epub and a few djvu. (Open)Office docs would be nice, but much lower on my list.

share|improve this question
1  
can you be more specific about what format the index takes? –  ixtmixilix Oct 13 '10 at 23:49
    
Good point, edited. –  julien Oct 14 '10 at 0:02
    
-1 As pointed out below Lucene or Tracker are good choices, but what's your problem with "use gtk, or even worse qt" ? Using just the command line is fine, but I don't see the point of criticizing a set of libraries... –  tmow Jan 27 '11 at 8:16
1  
@tmow: point taken, question edited. Didn't mean to undermine anyone's hard work, sorry if that's how it came through... –  julien Jan 27 '11 at 11:24

7 Answers 7

This answer recommends using Google's codesearch,

Code Search is a tool for indexing and then performing regular expression searches over large bodies of source code.

It is written in the Go language, but binaries are available if Go is not installed. Unfortunately I did not find any Debian package for it in the main repository.

share|improve this answer

Recoll can be built with no GUI and will search your document types from the command line.

share|improve this answer

check out xapian. It has command line interface and can index a lot of formats.

share|improve this answer
    
xapian is a great index, my personal favorite, and it's written in native C++!. Something which uses xapian would be most ideal, for many projects requiring indexing +1. –  TechZilla Oct 14 '12 at 22:08

Have you looked at Lucene or Sphinx? While you will need to initially parse the documents you want to index, once that's done, either one can search from the cli.

For Lucene, there is some info on doing this available.

Sphinx, is a bit more vague, but there is also some documentation available. You can pass structured XML data of your choice to sphinx via the xmlpipe2 data source.

Lucene relies on Java, while Sphinx is built in C++ with no needed outside dependencies.

Either one is going to require a bit of work to do what you want, but, seems like a totally workable solution.

share|improve this answer
1  
As an aside, if you want to index data that's in a DB (postrgres, mysql) then either of these also work incredibly well. –  gabe. Oct 14 '10 at 3:14
    
hmm, I had dismissed sphinx for being too low-level, but looking at xmlpipe2 it seems a wrapper script for pdf2txt or the like would be pretty easy... –  julien Oct 14 '10 at 13:02
    
+1 for mentioning in Lucene. I had good times with Lucene, it is a killer stuff!! –  Nikhil Mulley Jan 7 '12 at 13:08

Tracker can be invoked from the command line and gtk+ is not a hard dependency for a project (but may be for packages).

share|improve this answer
    
Well if I'm not mistaken, it'll still try to pull (large parts of) gnome as a dependency. –  julien Oct 13 '10 at 23:05
1  
As I said - project (in version 0.9.x at least the only GNOME hard dependency is glib). Packages may compiled the sane-default staff like GUI so you may need to compile it by hand. –  Maciej Piechotka Oct 13 '10 at 23:09

I worked on writing a full text search tool (a new apropos) for indexing and searching man pages for NetBSD this summer using Sqlite3. It consists of two command line tools:

  • makemandb: Which parses and builds an index of the content of man pages.
  • apropos: The tool for querying this index.

You could easily write a similar tool for yourself, for pdf's you will need a library for parsing pdf documents and similarly a utility to parse the open office documents.

You can read more about the project here

The code is here

share|improve this answer

There currently are two streams of Tracker, stable (0.8) and unstable (0.9). Your OS likely has the 0.8 version, so if you can afford it (it has some bleeding edge software dependencies), go grab the latest tarfile (0.9.x). It has lots of improvements over 0.8, and is currently being stabilized further in order to be 0.10 (even numbers represent stability). If you choose to go this route, use this command to configure:

./configure --disable-tracker-needle --disable-tracker-preferences --disable-tracker-explorer --disable-tracker-status-icon

You are likely not going to have the dependencies installed, so it should be esier to simply install 0.8 from your distro, and just avoid the GUI bits. On Debian Squeeze, Ubuntu 10.10, and Ubuntu 11.04, these are nicely split. So (as root) run:

apt-get install --no-install-recommends tracker-utils tracker-miner-fs

The CLI tool for this is tracker-search, so run it with the --help option to see how to take advantage of it :-)

notes:

  • On Fedora 14, the Tracker package has dependencies on GTK+. I guess it's because it includes things like tracker-applet and tracker-preferences. They do however have a separate package for tracker-search-tool, the GUI search interface.
  • DjVu and ePUB aren't (yet) supported. Here's a list of of what is.
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.