1

For various reasons, I am not running logstash (7.10.1) as a service, but rather invoking it on-demand, in a bash script:

/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/my_ls.conf &
echo ""
echo "#########################################################"
read -p "Press enter to continue "

It works well, but when finishing to create an elasticsearch index successfully, it always pauses with:

[INFO ]  [[main]-pipeline-manager] javapipeline - Pipeline Java execution initialization time {"seconds"=>1.3}
[INFO ]  [[main]-pipeline-manager] javapipeline - Pipeline started {"pipeline.id"=>"main"}
[INFO ]  [[main]<file] observingtail - START, creating Discoverer, Watch with file and sincedb collections
[INFO ]  [Agent thread] agent - Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[INFO ]  [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}

In order to make it return to the CLI prompt, I need to run from another SSH terminal the following:

pkill -f logstash 

This is of course inconvenient and I am looking for a way to make the bash script prompt with "Press any key to exit".

My problem is that the statements after the logstash invocation (with the & appended), including the prompt read -p "Press enter to continue" are displayed before logstash actually starts doing its work and the bash script never exits to the CLI prompt.

What is the proper way to make the bash script prompt with "Press any key to exit" when logstash finishes the index creation?

6
  • 2
    "once logstash is started (even with the & appended), bash never gets to the next statement in the script" – In case of & I really doubt it. What is the next line of the script? How do you know Bash never gets to it? Commented Jan 4, 2022 at 16:14
  • @KamilMaciorowski You are correct. Thanks to your comment, I just noticed the echo "##########" line after the the call to logstash is actually being executed, but displayed before the entire logstash output. This confused me. I am going to amend my post to more accurately reflect what actually happens.
    – datsb
    Commented Jan 4, 2022 at 16:25
  • The behaviour described in this comment ↑ is pretty much what I would expect. Logstash runs in the background and writes its output in parallel with your foreground process. The timing here is such that the foreground echo writes before logstash does. Commented Jan 4, 2022 at 16:34
  • 2
    This off-site question and answer from a few years ago seems to indicate that you cannot run logstash just to build the indexes and then exit. I found no newer article suggesting that this had changed. Commented Jan 4, 2022 at 17:00
  • 2
    It's apparent that this is kind of an intended behaviour when running in the foreground. So I'll propose a workaround for OP, and it is to use expect to capture the output of the script when it has ended. Based on the output, make expect send a pkill or whatever, to have a prompt again. This might be a little tricky but it's not impossible
    – Alex
    Commented Jan 4, 2022 at 17:22

1 Answer 1

0

Solved!

Here is how:

First, it is important to acknowledge, as @roaima indicated, that it is true that logstash was not designed to build an index and then exit. Having this knowledge helps focus on a workaround as @Alex suggested. Moreover, this known limitation led to logstash-plugin based workarounds as described here:

  1. exec plugin
  2. exit plugin

However, a plugin requires its installation on every logstash instance and its own statement in every .conf file.

So, I found this UL answer, which spoon-fed me with the solution:

    LS_OUTPUT=$(mktemp "${TMPDIR:-/tmp/}$(basename 0).XXX")
    /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/my_ls.conf &> $LS_OUTPUT &
    LS_PID=$!

    echo "Logstash pid: ${LS_PID}"
    echo "Logstash output: $LS_OUTPUT"
    echo "Wait:"

    until grep -q -i 'agent - Successfully started Logstash API endpoint' $LS_OUTPUT
    do
      if ! ps $server_pid > /dev/null
      then
        echo "Logstash died. Exiting." >&2
        exit 1
      fi
      echo -n "."
      sleep 1
    done
    echo
    echo "Logstash is running!"

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .