3

I have a sort of json template file like this:

# INPUT file.json
{
  "set1":{
          "string1": "${VAR1}",
          "string2": "${VAR2}",
          (...)
         }
}

A script collects these VARs values. So, I need to source this json template and output a json.tmp with all variables set.

# OUTPUT file.json.tmp
{
  "set1":{
          "string1": "string1_value",
          "string2": "string2_value",
          (...)
         }
}

I thought of reading the file then echo it, but it always outputs the tmp file without setting those vars.

VAR1=$1
VAR2=$2
# (...)
JSON_IN=`cat file.json`
echo $JSON_IN > file.json.tmp

Any ideas?

Thanks

1

2 Answers 2

3

This is what envsubst can do:

The variables must be exported because envsubst is a forked process and doesn't know of non-exported variables.

export VAR1=$1
export VAR2=$2
envsubst < file.json > file.json.tmp

Note, that this will break your json validity, if the variables contain characters like double quote (") or newline.

4
  • For now that actually works. But why do I have export those variables?
    – markfree
    Commented Sep 1, 2021 at 14:34
  • 1
    envsubst is forked and cannot know of non exported variable.
    – Archemar
    Commented Sep 1, 2021 at 14:48
  • 1
    Note that this will break (as in create invalid JSON) if any of the variables contain characters that ordinarily needs to be JSON encoded.
    – Kusalananda
    Commented Sep 1, 2021 at 17:43
  • Thanks for the comments, added to the answer.
    – pLumo
    Commented Sep 2, 2021 at 5:20
0

Using envsubst would work if the values of your variables are "nice" and does not need to be JSON encoded. It also requires your variables to be exported.

However, as soon as you have special values in you variables, this falls apart:

$ jq . file
{
  "set1": {
    "string1": "${VAR1}",
    "string2": "${VAR2}"
  }
}
$ export VAR1='line 1
line 2'
$ export VAR2='"hello"'
$ envsubst <file | jq .
parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 4, column 7

Instead, trim off the ${ from the start and } from the end of the values in the JSON file and use the remaining string to look up the value in the internal $ENV variable (an object holding the environment variables as key-value pairs):

$ jq '.[][] |= ( ltrimstr("${") | rtrimstr("}") | $ENV[.] )' file
{
  "set1": {
    "string1": "line 1\nline 2",
    "string2": "\"hello\""
  }
}

The .[][] extracts the paths of the values, and |= is the update operator. So the values are updated by finding the values of the environment variables that they correspond to, and replacing them with these values.

With the code above, a string that does not correspond to an existing environment variable will be replaced by null:

$ jq . file
{
  "set1": {
    "string1": "${VAR1}",
    "string2": "${VAR2}",
    "string3": "${bumblebee}"
  }
}
$ jq '.[][] |= ( ltrimstr("${") | rtrimstr("}") | $ENV[.] )' file
{
  "set1": {
    "string1": "line 1\nline 2",
    "string2": "\"hello\"",
    "string3": null
  }
}

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .