Find a pattern and replace it's value in shell script

Question

I have been trying to do following changes to my configuration file using shell script. This is my following elastic search configuration file.

Need to comment out below line and replace its value.

#network.host: 192.168.0.1

and add these lines in network section

transport.host: localhost
transport.tcp.port: 9300

How can I achieve this?

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#network.host: 192.168.0.1
#
# Set a custom port for HTTP:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.zen.ping.unicast.hosts: ["host1", "host2"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes: 
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#

Answer 1

You can use sed to do it.

The first part is quite simple, finding a commented network.host and replacing it with an uncommented one with a different value can be done with:

sed -i -e 's/^#network\.host: .*/network.host: 1.2.3.4/' "${ES_HOME}/config/elasticsearch.yml"

The -i option does an in-place modification, so it will replace your current elasticsearch.yml file. (You can save a backup of it, for instance elasticsearch.yml.bak by using -i.bak instead.)

The argument to -e is a sed script, in this case a regular expression with a search/replace command. It matches a commented line, starting with #network.host and replaces it with an uncommented line including an IP.

If you want to get the IP or host name from an environment variable, you can do so by breaking the '...' string in two and inserting the external variable there:

sed -i -e 's/^#network\.host: .*/network.host: '"${ip_address}"'/' "${ES_HOME}/config/elasticsearch.yml"

But note that this can be brittle... If the contents of ${ip_address} include a / character, this will break the sed command...

For the second part, inserting the transport.host lines, you can use sed's i\ command to insert a line before the one you match. For instance, you can match the last comment in the Network section ("... consult the network module documentation") and insert there. As you're inserting multiple lines, you'll want to start a new block { so you can run multiple commands.

This should do it (note that this is a command that spans multiple lines!):

sed -i -e '
/consult the network module documentation/{
i\
# Set custom transport settings:
i\
#
i\
transport.host: localhost
i\
transport.tcp.port: 9300
i\
#
}' "${ES_HOME}/config/elasticsearch.yml"

Now we can put it all together, and also add a check to skip the insertion if it was already done before. We can do so by looking for the comment we inserted ("Set custom transport setting") and using the b command to skip to the end of the script, skipping the following edit in that case.

The final script is:

# Set your own IP into ${ip_address} however you have to.
ip_address=1.2.3.4
sed -i -e '
s/^#network\.host: .*/network.host: '"${ip_address}"'/
/^# Set custom transport settings/,$b
/consult the network module documentation/{
i\
# Set custom transport settings:
i\
#
i\
transport.host: localhost
i\
transport.tcp.port: 9300
i\
#
}' "${ES_HOME}/config/elasticsearch.yml"