I'm trying to write a script that finds strings in a certain pattern ({?varname}) and replaces them with a corresponding environment variable.

This is what I have so far:

function get_env() {
   echo ${!var}

sed -e 's/{?\([a-z]*}\)/'$(get_env '\1')'/g' file.txt

The function works, e.g. get_env LOGNAME --> dotan

Sed returns the value of the function, e.g. if I replace the function's content with echo __$1__ I will get {?logname} --> __LOGNAME__

However put together it doesn't work. It's like the function always returns an empty string.

I'm not sure what causes the problem here. Any ideas?

  • Probably not your issue but always quote your variables. echo "${!var}"
    – jesse_b
    Feb 25 '18 at 12:58
  • 1
    You will not be able to pass the back reference \1 out of sed, back into the shell and into your function, and then get the correct value back from there.
    – Kusalananda
    Feb 25 '18 at 12:58

That command line would run get_env '\1' in a command substitution, and paste its output on the command line of sed. You should probably get an error about that, since \1 is not a valid name for a variable.

You could use Perl and s/\{\?([a-z]+)\}/$ENV{$1}/ to do that; %ENV is a hash that contains the environment variables:

$ export name=you
$ echo "hello {?name}" | perl -pe 's/\{\?([a-z]+)\}/$ENV{$1}/'
hello you

(GNU sed has the e command to run external programs, but it uses runs the whole pattern space, not just the matching part, so it doesn't seem to fit here too well.)

Purely in Bash, if you insist (setvar.sh):

while IFS= read -r line ; do
        while [[ $line =~ ^(.*)(\{\?[a-zA-Z0-9_]+\})(.*)$ ]] ; do
                printf -v line "%s%s%s" "${BASH_REMATCH[1]}" "${!varname}" "${BASH_REMATCH[3]}"
        printf "%s\n" "$line" 
done < "${1-/dev/stdin}"

$ echo 'foo {?BASH_VERSION} {?blah} bar' |blah=ABC bash setvar.sh
foo 4.4.12(1)-release ABC bar
  • 1
    good one.. suggestion: e modifier not here.. s/\{\?([a-z]+)\}/$ENV{$1}/ would work too..
    – Sundeep
    Feb 25 '18 at 13:42
  • 1
    @Sundeep, oh right, it doesn't need that, thanks!
    – ilkkachu
    Feb 25 '18 at 13:52
  • the get env '\1' part works, actually. Like I said, if the function only echos $1, the value stays the same. Anyway, thanks, but I was looking for a solution in pure bash.
    – Dotan
    Feb 26 '18 at 18:04
  • @Dotan, well yes, it works, the shell runs the function once, and passes the output to sed's command line. If the output is foo, then sed replaces all matches with foo. If the output of the function is \1 then sed replaces all matches with the first backreference. And if the function is get_env() { echo "$1"; }, it outputs its first argument, so $(get_env '\1') outputs \1.
    – ilkkachu
    Feb 26 '18 at 19:55
  • @Dotan, but as you saw and as Kusalananda commented, you can't get the string matched by sed back to the shell for the command substitution. That's the reason you're asking this in the first place, isn't it?
    – ilkkachu
    Feb 26 '18 at 20:03

This is the solution I came up with

grep -o '{?\S*}' file.txt | sed 's/{?\(.*\)}/\1/' | xargs -I {} sh -c "sed -i -e 's/{?"{}"}/'$"{}"'/' file.txt"

To break it down:

  • grep -o '{?\S*}' file.txt - get all the variables to replace
  • sed 's/{?\(.*\)}/\1/' - strip the formatting
  • xargs -I {} sh -c "sed -i -e 's/{?"{}"}/'$"{}"'/' file.txt" - search and replace for each var seprately

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.