0

I know elasticsearch is good for full text search but when it comes to analyze logs, say nginx logs what is the advantage of using it over grep (with tail, awk, etc.) ?

Quite of often I dig through logs try to find some keywords, with grep it is simple. If I use elasticsearch, the investment of setting it up to work and the learning curve are huge compared to grep. So I am wondering is it worth the "trouble" ?

In what scenario grep just can't handle and elasticsearch will shine ?

closed as primarily opinion-based by muru, Rui F Ribeiro, msp9011, Kiwy, meuh May 21 at 17:55

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. If this question can be reworded to fit the rules in the help center, please edit the question.

1

With grep, there is no indexing, so each time you want to search for something, the command needs to read through all the logs. When you have a non-trivial amount of logs, this can take a long time and a lot of CPU power.

With elasticsearch, the messages are indexed as they arrive, so the actual searching can happen faster and with less CPU power. Elasticsearch also includes various analytic functions grep cannot do alone.

It's mostly a matter of scale. Basically, if you only have megabytes of logs, dealing with them using grep and other command line tools is trivial. If you have gigabytes of logs, searching through them with grep is starting to get slow and inefficient, but doable if you plan ahead. With terabytes of logs, you'll definitely need something like Elasticsearch to deal with them in a reasonable way.

Not the answer you're looking for? Browse other questions tagged or ask your own question.