Questions tagged [elasticsearch]
ElasticSearch is a JSON-based search and analytics engine, often associated with Logstash, which collects data, and Kibana, which visualizes the results. There are separate tags for these.
57
questions
0
votes
1
answer
25
views
I want remove repeated records and remove those lines in awk
I want check repeated records in column 2 and remove those lines in awk
create a
delete a
create b
create c
delete c
create d
delete f
create f
create g
create h
Expected Output
create b
create d
...
- 15
0
votes
0
answers
5
views
Elasticsearch in Docker max file descriptors issue
I'm trying to run a container from a custom Docker image of Elasticsearch. It's crashing with a
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
How ...
- 101
0
votes
0
answers
75
views
No type specified for field [settings] elasticsaearch
I am getting below error
curl -X PUT http://localhost:9200/graylog_new -H 'Content-Type: application/json' -d @log.json
{"error":{"root_cause":[{"type":"...
- 51
0
votes
0
answers
128
views
Timeout error in Elastic search
I am working on the below URL
https://v2.whoisdatacenter.com
When I hit query like Neha in the "name" tab, it works well.
But when I hit query like Neha Jain in the name tab, It takes very ...
0
votes
0
answers
10
views
403 Error in schema testing tool
Problem :
My URLs are
https://v2.domainsanalytics.com/allheartweb.com
https://v2.domainsanalytics.com/name/komal
https://v2.domainsanalytics.com/name/komal.com
I am only getting an error when a dot . ...
0
votes
0
answers
131
views
Allow a Python program run without root privileges to restart Elasticsearch
There are a few suggestions out there with clues about how this might be achieved. It seems to me that the restart command of Elasticsearch shouldn't need root privileges, at least in my use case.
...
- 1,052
0
votes
1
answer
82
views
elasticsearch: On premise restore snapshot from aws s3
This could sound pretty straightforward. However, I've spent days looking in the web for a method where I can migrate snapshot from aws s3 to on premise elasticsearch cluster.
All the docs I've found ...
- 1,631
0
votes
1
answer
69
views
HTTPD Redirect Rule as Proxy is giving me file not found error. How do I proxy to an external url?
I have a reverse proxy to AWS Elasticsearch. I am having issues with using RedirectRule: no matter what I try, my URL is being interpreted as a file.
SSLProxyEngine On
ProxyRequests On
...
- 11
0
votes
1
answer
367
views
Auditbeat exclude /usr/sbin/cron
I'll tried to exclude event from cron jobs running that can be found with the KQL request : auditd.summary.how :"/usr/sbin/cron"
My host does not running SE Linux, so the rules i found (put ...
- 101
0
votes
1
answer
2k
views
No valid OpenPGP data found - Elasticsearch wget
I am trying to install elasticsearch on Ubuntu 20.04, but I am getting the following error:
home@VirtualBox$ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
gpg: no ...
- 101
0
votes
0
answers
246
views
Curl argument list too long error in elastic search mapping and setting
mapping=(</index_automation/{product}/mapping/Mapping/"$tempmappingfile")
settings=(</index_automation/{product}/mapping/Mapping/"$tempsettingsfile")
dataraw="{{...
0
votes
1
answer
2k
views
Kibana service won't start
I am using Manjaro and installed elasticsearch and kibana with
yay -S elasticsearch kibana
Starting the elasticsearch service works well
sudo systemctl start elasticsearch
I've configured kibana ...
- 103
0
votes
0
answers
36
views
hearthbeat can't run
I have ELK server, and sending Heartbeats to server. I added an monitor in monitors.d directory, added setup.dashboards.enabled: true line in heartbeat.yml file, but after restarting heartbeat service,...
- 95
0
votes
1
answer
869
views
What is fields.yml file
Here I have installed packetbeat. So in the time learning the configuration for packetbeat I saw a file fields.yml in the directory /etc/packetbeat. I tried to get any information about that file but ...
- 95
0
votes
1
answer
723
views
Gmake on FreeBSD: exec: “clang”: executable file not found in $PATH [closed]
I'm trying to install Filebeat (elasticsearch log shipping) on PfSense 2.4.5. I follow the instruction from: https://blog.securitybits.io/2019/12/beats-7.5.0-on-pfsense-2.4.4/ At the make step, I got ...
- 19
0
votes
1
answer
420
views
Elasticsearch curator not deleting old indices
I am using Elasticsearch + Kibana + File-/Metric-/Winlogbeat to summarize my logfiles.
To not fill up my HDD with tons of logs I wanted to just keep them 14 days - so I installed Curator to do this ...
0
votes
1
answer
6k
views
what is GREEDYDATA in elasticsearch
reading the conf files of logstash i found in filter conf
grok {
match => { "message" => "Put\s*command\s*:\s+%{GREEDYDATA:command}" }
}
How does this filter work , i tried to ...
- 13
0
votes
1
answer
50
views
Unable to remove the unwanted messages(especially Auditbeat) in my centos server
I need your help in solving the below mentioned Issue.
I have Installed Auditbeat-7.3.1 as a part of ELK setup in my centos server. Everything was working well, But i was facing problem while ...
1
vote
1
answer
272
views
rsyslog error in logstash
I have installed ELK and tried to configure rsyslog server with logstash but i am getting lot of \ while executing curl -XGET 'http://172.22.63.61:9200/logstash-*/_search?q=*&pretty'
Rsyslog ...
- 11
3
votes
2
answers
9k
views
Change filesystem without losing data
I've got 16 nodes of elasticsearch (RHEL 7) - 18TB each, every node has a ext4 filesystem. For better efficency I need to change to XFS filesystem.
Is there any tool / way which help me change ...
- 33
4
votes
1
answer
1k
views
Yum install: unexpected nothing to do while upgrading elasticsearch
I'm experimenting something estrange with yum in centos7, lets see If someone understands what's happening:
Usually, I upgrade packages using sudo yum install <package_name>-<version_info>...
- 41
0
votes
1
answer
196
views
unable to connect with localhost (11.213.234.24) on port 9200 from vagrant
I am running vagrant from a virtual box with centos. Trying to connect my localhost on port 9200 from vagrant by using (curl -X GET 'http://11.213.234.24:9200'). Getting below error
curl: (7) couldn'...
0
votes
1
answer
645
views
Elasticsearch fails to start with 'permissions read' error even though read permissions exist
I have a symlink at /etc/elasticsearch/analysis that points to a mounted drive at /elasticdata/fileshare/analysis on Ubuntu 16.04 LTS Azure VM. When I try and start elasticsearch I am getting the ...
- 103
0
votes
0
answers
31
views
Elasticsearch process run under weird username
I have Elasticsearch 1.7.3 running successfully on CentOS7. However, I observed:
%ps -aux | grep elastic
**209** 10504 85.0 29.2 36122740 19283600 ? SLl 04:02 0:05
/usr/bin/java -Xms30g -...
- 1
0
votes
1
answer
878
views
Advantage of elasticsearch over grep when trying to search keyword in logs files [closed]
I know elasticsearch is good for full text search but when it comes to analyze logs, say nginx logs what is the advantage of using it over grep (with tail, awk, etc.) ?
Quite of often I dig through ...
- 177
0
votes
1
answer
32
views
How to operate on a directory with different owner from a script?
Few days ago i have installed elasticsearch,
For diagnosing some problems on my elasticsearch setup, i have checked the logs files located in /var/log/elasticsearch and tailed them but apparently the ...
- 406
3
votes
1
answer
2k
views
Why does a systemd unit file at `/usr/lib/systemd/system/` still works for Ubuntu?
I understand that systemd stores unit files at different locations for different versions of Linux. On RHEL, it's at /usr/lib/systemd/system/, whereas on Debian-based machines it's at /lib/systemd/...
- 505
1
vote
2
answers
13k
views
Elasticsearch cannot open log file: Permission denied
The error
I installed Elasticsearch on CentOS using rpm and am encountering this error when I try to start it via systemctl start elasticsearch:
Feb 20 21:08:34 server.cberdata.org systemd[1]: ...
- 121
0
votes
1
answer
315
views
How can I set the Java heap size for elasticsearch on Nixos?
It seems these should go in the ${configDir}/jvm.options"} (/var/lib/elasticsearch/config/jvm.options ?) looking at the package at https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/...
- 4,347
0
votes
1
answer
762
views
One port is listening with an * and the other is listening with 127.0.0.1 and only the first is accepting remote connections
Please see the attached picture below, and then I will explain:
So, I have Solr running which uses port 8983. And as you can see in the screen shot, it's listed as "TCP *:8983 (LISTEN)". And, it's ...
- 123
0
votes
1
answer
1k
views
Can I lock down access to a TCP port in Linux
I'm trying to secure a single-node Elasticsearch machine. Basically, I want to put Elasticsearch behind a reverse proxy (nginx) and make sure that the only way any user or process can communicate with ...
- 105
1
vote
1
answer
3k
views
How is JVM memory usage calculated?
I am tuning my Elasticsearch cluster, and see that the memory usage doesn't line up. Running "free" on my instance:
free
total used free shared buff/cache ...
- 121
0
votes
1
answer
817
views
problem re-loading index from filebeat in elasticsearch
I am using the ELK stack (more ELG stack as I am using Grafana as the front end instead of kibana for personal reasons). I am using Filebeat to send the logs file to Logstash which are then stored in ...
6
votes
1
answer
12k
views
How to give RAM to the filesystem cache
I am tuning my linux machine running Elasticsearch. It says that I should give at least half the memory of the machine running elasticsearch to the filesystem cache. But I don't know how much of it is ...
- 305
-1
votes
2
answers
23k
views
How do i completely uninstall ELK (Elasticsearch, Logstash, Kibana)?
I search on internet that we have to unistall each of the ELK part one by one like unistall stand-alone kibana, elastic search, and logstash. Is there any command which no need to unistall all of them ...
- 215
3
votes
2
answers
484
views
Timeouts between an API and an elasticsearch server every 2 hours
We are experiencing a weird issue on our servers. (Debian 8.9) We have an API which is a PHP application. It requests an elasticsearch which instance is on a separate server.
Every 2 hours, we are ...
- 301
1
vote
1
answer
3k
views
How to start elasticsearch on a CentOS?
I have downloaded the latest elasticsearch version 5.5.1 on my Centos 7 server from https://www.elastic.co/downloads/elasticsearch. I have unzipped the file and store it in elasticsearch directory.
...
- 143
2
votes
1
answer
5k
views
"unable to load jna" error on Elasticsearch on Centos
What is the solution to this?
New install of elasticsearch 5.4.1 on Centos 6.8
[2017-06-15T13:54:19,140][WARN ][o.e.b.Natives ] unable to load JNA n
ative support library, native methods ...
3
votes
2
answers
383
views
Can not run Elasticsearch on Linux Mint 18
I installed Elasticsearch on Linux Mint without any trouble.
Everything according this tutorial Easy install for elasticsearch on Ubuntu 14.04.
All installation steps went well.
However, when I want ...
- 209
0
votes
1
answer
492
views
Kibana- Want to split vertical bars based on my log fields
I have an application log file consists of following log levels: INFO, WARN, ERROR, DEBUG. Following filter criteria works fine in logstash config file:
filter {
grok {
match => { "...
- 335
0
votes
1
answer
241
views
Send Apache Catalina Logs to Logstash
Dependencies :
OS Version : CentOS 5.0
Server Environment : Production
Logstash Version : 5.0
Elasticsearch Version : 5.0
I have several app servers running on CentOS 5.0. All these are on-...
- 31
0
votes
2
answers
916
views
How can I create a menu item for terminal-applications starting in background mode
I want to start e.g. my ElasticSearch instance which is installed manually via one-click in the cinnamon menu.
What I can do is /opt/elasticsearch-1.6.2/bin/elasticsearch & and the application is ...
- 193
0
votes
1
answer
199
views
logstash - take 2 - filter to send messages from IntelMQ/python/redis to ELK
Following up on the heels of this question, https://stackoverflow.com/questions/40768603/logstash-trying-to-make-sense-of-strings-passed-by-intelmq-in-elasticsearch I am trying to create a refine/...
- 53.9k
1
vote
1
answer
255
views
Tile/Greographic Map in Kibana not working
I am trying to create a geographical map of my data in Kibana 5.01, and it does not work. The fact is that I do not even have the geoip.field that is required in the menu.
I am sending data from ...
- 53.9k
3
votes
2
answers
3k
views
Unable to start elasticsearch
Configuration :
Elasticsearch Version = 5.0.1
OS = Amazon Linux AMI release 2014.03
I am using AWS m3.large instance. I have two elasticsearch nodes. For my data node(node-2) , When I started ...
- 31
4
votes
3
answers
8k
views
How to raise max threads for elasticsearch?
I have a fresh installation of Ubuntu 16.04 and I installed elasticsearch 5 on it. Now when I try to start elasticsearch using
/etc/init.d/elasticsearch start
After a few seconds in the log file I ...
1
vote
1
answer
2k
views
send json via rsyslog to elasticsearch
in our company we need aggregate services statistics
for some reason we decide to use rsyslogd to send applications statistics (json format) to elasticsearch system directly (this tutorial).
but we ...
2
votes
2
answers
302
views
Malicious processes open connections to China
I recently started migrating some of my company's servers to Google's compute engine. Among other things I set up a 2-node elasticsearch cluster.
Today I run top command on one of the nodes and I ...
- 123
3
votes
0
answers
5k
views
Able to ping my host but getting no route to host
I have a elasticsearch service running on MY_HOST_IP(10.x.x.xx) and port 9200. I am able to ping 10.x.x.xx from my VM, but when I try to connect elasticsearch it gives me :
no route to host ...
- 385
0
votes
1
answer
386
views
Install ELK in centos
I install ELK on centos following below link.
https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-1-7-logstash-1-5-and-kibana-4-1-elk-stack-on-centos-7
But the problem ...
- 477