104

I'm trying to perform environment variable replacement through envsubst, but I want to only replace specific variables.

From the docs I should be able to tell envsubst to only replace certain variables but I'm failing to be able to do that.

For example, if I have a file containing:

VAR_1=${VAR_1}
VAR_2=${VAR_2}

how should I execute envsubst so that it only replaces the reference to ${VAR_1}?

0

4 Answers 4

290

Per the man page:

envsubst [OPTION] [SHELL-FORMAT]

If a SHELL-FORMAT is given, only those environment variables that are referenced in SHELL-FORMAT are substituted; otherwise all environment variables references occurring in standard input are substituted.

Where SHELL-FORMAT strings are "strings with references to shell variables in the form $variable or ${variable}[...] The variable names must consist solely of alphanumeric or underscore ASCII characters, not start with a digit and be nonempty; otherwise such a variable reference is ignored.".


Note that the format ${VAR:-default} is not supported. I mentioned HERE some alternatives that support it along with other features.


Anyway, back to gettext envsubst:
So, one has to pass the respective variables names to envsubst in a shell format string (obviously, they need to be escaped/quoted so as to be passed literally to envsubst). Example:

input file e.g. infile:

VAR1=${VAR1}
VAR2=${VAR2}
VAR3=${VAR3}

and some values like

export  VAR1="one" VAR2="two" VAR3="three"

then running

envsubst '${VAR1} ${VAR3}' <infile

or

envsubst '${VAR1},${VAR3}' <infile

or

envsubst '${VAR1}
${VAR3}' <infile

outputs

VAR1=one
VAR2=${VAR2}
VAR3=three

Or, if you prefer backslash:

envsubst \$VAR1,\$VAR2 <infile

produces

VAR1=one
VAR2=two
VAR3=${VAR3}

To avoid leakage of the custom-set variables into the executing shell's environment, instead of exporting these variables, some shells such as Bash allow prefixing the command with variable assignments. The values assigned in this way will only be used for the command's execution, and don't affect the shell environment afterwards:

export VAR1="one" VAR2="two" VAR3="three"

# ...

VAR1="number 1" VAR3="numero 3" envsubst '${VAR1} ${VAR3}' <infile

# output:
# VAR1=number 1
# VAR2=${VAR2}
# VAR3=numero 3

echo "VAR1=${VAR1}"
echo "VAR2=${VAR2}"
echo "VAR3=${VAR3}"
# output shows that the original environment was preserved:
# VAR1=one
# VAR2=two
# VAR3=three

13
  • 36
    why is this not the accepted answer?
    – Qsiris
    Jan 11, 2018 at 13:39
  • 10
    Yes this is the right answer! Feb 26, 2018 at 9:31
  • 11
    If you know a variable prefix to use to limit the variables that can be substituted: envsubst "$(printf '${%s} ' ${!PREFIX*})" < infile Jan 22, 2019 at 15:13
  • 5
    Or if you have a list of them in SUBST_VARS then: envsubst "$(printf '${%s} ' $SUBST_VARS)" < infile Jan 22, 2019 at 15:15
  • 2
    This answer ALSO substitutes an empty string for variables present that are not set. As opposed to ONLY replacing the specific variables. Mar 6, 2020 at 12:45
3

Although related to docker, the utility envplate should do the job https://github.com/kreuzwerker/envplate

From the readme:

Trivial templating for configuration files using environment keys. References to such keys are declared in arbitrary config files either as:

${key} or

${key:-default value}

gnutext's envsubst only replaces ${key}; if missing is replaced by ''.

1
1

Unfortunately 'export' is mandatory, otherwise it won't work. Here is an example.

First, the input file. Note that I only want to change NGINX_HOST and NGINX_PORT and don't want to touch other variables in the input file such as '$uri'.

$ egrep 'server_name|listen|try_files' nginx-default.conf.template 
  server_name ${NGINX_HOST};
  listen ${NGINX_PORT};
    try_files $uri $uri/ /index.php?$args;

Now the empty variables:

$ echo ${NGINX_HOST}

$ echo ${NGINX_PORT}

Set default values to these variables:

$ NGINX_HOST=${NGINX_HOST:-localhost}
$ NGINX_PORT=${NGINX_PORT:-80}
$ echo ${NGINX_HOST}
localhost
$ echo ${NGINX_PORT}
80

Try to use envsubst:

$ envsubst \
    '${NGINX_HOST} ${NGINX_PORT}' \
    < nginx-default.conf.template \
    | egrep 'server_name|listen|try_files' 
    
  server_name ;
  listen ;
  try_files $uri $uri/ /index.php?$args;

As you can see, it did not work. The variables must be 'export'-ed for this to work. Here is the second run:

$ export NGINX_HOST=${NGINX_HOST:-localhost}
$ export NGINX_PORT=${NGINX_PORT:-80}

$ envsubst   '${NGINX_HOST} ${NGINX_PORT}' <  nginx-default.conf.template | egrep 'server_name|listen|try_files' 

  server_name localhost;
  listen 80;
    try_files $uri $uri/ /index.php?$args;

As you can see, now it worked. Hope it helps others.

3
  • This is like a car user’s manual saying “This car must be used on a solid surface in a gravitational field.   It doesn’t work in water, or when in free-fall or in outer space.”   In other words, everybody knows this. Jun 16, 2022 at 4:21
  • 1
    The question is rather: How to avoid replacing $uri in the file if there is an uri environment variable in the current environment.
    – Kusalananda
    Jun 19, 2022 at 6:02
  • G-Man... this is a site where people who don't understand things come to learn about them. Just because you understand unix well enough to know the export is required does not mean that everyone will understand this. People who drive all the time know that to signal left you move the lever down, and up for right... but people who are still learning to drive don't understand this, and we should be more patient, even if YOU already understand it perfectly.
    – Ajax
    Oct 20, 2023 at 22:33
0

Before calling envsubst you should use export using single quotes to get back VAR_1 modified. As in:

export VAR_1='somevalue'

For more details, please see:

How to substitute shell variables in complex text files

6
  • 53
    Correct answer is below
    – Craig
    Mar 19, 2018 at 17:40
  • 1
    @JoãoAngelo Would you clarify why this was marked has the correct one over the other answer? For me it seems more appropriate asking the original asker. Jan 22, 2020 at 11:20
  • @RuiFRibeiro your answer will replace all variables (not only specific variables). all patterns like $varname will be evaluated (not only the variables you're targetting).
    – MhdSyrwan
    Feb 26, 2020 at 12:31
  • 8
    @Craig There is no "below" on stack overflow - answers change order depending on various factors. Please link to the actual answer.
    – rjmunro
    May 25, 2022 at 14:17
  • 2
    This is not the correct answer. This will lead to any undefined vriable being replaced with an empty string. The correct answer is by @don_crissti
    – Andrey
    Jan 30 at 19:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .