How to expand variables inside read?

Question

In a remote CentOS with Bash 5.0.17(1) where I am the only user via SSH I have executed read web_application_root with:

$HOME/www

or with:

${HOME}/www

or with:

"${HOME}"/www

or with:

"${HOME}/www"

Aiming to get an output with an expanded (environment) variable such as MY_USER_HOME_DIRECTORY/www.

While ls -la $HOME/www works fine, ls -la $web_application_root fails with all examples; an error example is:

ls: cannot access '$HOME/www': No such file or directory

I understand that read treats all the above $HOME variants as a string (due to the single quote marks in the error) and hence doesn't expand it.

How to expand variables inside read?

Answer 1

Variables are not expanded when passed to read. If you want to expand the $VARs or ${VAR}s where VAR denotes the name of an existing environment variable (limited to those whose name starts with an ASCII letter or underscore and followed by ASCII alnums or underscores) and leave all the other word expansions ($non_exported_shell_variable, $1, $#, ${HOME+x}, $((1 + 1)), $(cmd)...) untouched, you could use envsubst (from GNU gettext):

IFS= read -r web_application_root || exit
web_application_root=$(printf %s "$web_application_root" | envsubst)
ls -la -- "$web_application_root"

You could make it a shell function that takes a variable name as argument and does both the reading and environment variable expansion with:

read_one_line_and_expand_envvars() {
  IFS= read -r "$1"
  ret="$?"
  command eval "$1"'=$(printf %s "${'"$1"'}" | envsubst)' && return "$ret"
}

To be used for instance as:

printf >&2 'Please enter the root dir (${ENVVAR} expanded): '
read_one_line_and_expand_envvars web_application_root || exit
printf >&2 'The expanded version of your input is "%s"\n' "$web_application_root"

To limit that substitution to a limited set of environment variables, you'd pass the list as a $VAR1$VAR2... literal argument to envsubst:

web_application_root=$(
  printf %s "$web_application_root" |
    envsubst '$HOME$MYENVVAR'
)

(here tells envsubst to only substitute $HOME, ${HOME}, $MYENVVAR and ${MYENVVAR} in its input, leaving all other $VARs untouched).

If you want to allow all forms of word expansions¹ (but note that then that makes it a command injection vulnerability), you could do:

web_application_root=$(eval "cat << __EOF__
$web_application_root
__EOF__")

Or again, as a function that takes the variable name as argument:

read_one_line_and_perform_shell_word_expansions() {
  IFS= read -r "$1"
  ret=$?
  command eval '
    case "${'"$1"'}" in
      (EOF) ;;
      (*)
        '"$1"'=$(command eval "cat << EOF
${'"$1"'}
EOF")
    esac' && return "$ret"
}

printf >&2 'Please enter the root dir ($var/$((...))/$(cmd) allowed): '
read_one_line_and_perform_shell_word_expansions web_application_root || exit
printf >&2 'The expanded version of your input is "%s"\n' "$web_application_root"

The same function with detailed inline documentation:

read_one_line_and_perform_shell_word_expansions() {
  # first argument of our function is the variable name or REPLY
  # if not specified.
  varname=${1-REPLY}

  # read one line from stdin with read's unwanted default post-processing
  # (which is otherwise dependant on the current value of $IFS) disabled.
  IFS= read -r "$varname"

  # record read's exit status. If it's non zero, a full line could not be
  # read. We may still want to perform the expansions in whatever much
  # was read, and pass that exit status to the caller so they decide what
  # to do with it.
  ret=$?

  # We prefix the "eval" special builtin with "command" to make it lose
  # its "special" status (namely here, exit the script about failure,
  # something bash only does when in standard mode).
  command eval '
    # the approach we take to expand word expansions would be defeated
    # if the user entered "EOF" which is the delimiter we chose for our
    # here-document, so we need to handle it as a special case:
    case "${'"$varname"'}" in
      (EOF) ;;
      (*)
        # the idea here is to have the shell evaluate the
        # myvar=$(command eval "cat << EOF
        # ${myvar}
        # EOF")
        #
        # shell code when $1 is myvar, so that the
        #
        # cat << EOF
        # contents of $myvar with $(cmd), $ENV and all
        # EOF
        #
        # shell code be evaluated, and those $(cmd), $ENV expansions
        # performed in the process
        '"$varname"'=$(command eval "cat << EOF
${'"$varname"'}
EOF")
    esac' &&
      # unless eval itself failed, return read's exit status to the caller:
      return "$ret"
}

But your problems sounds more like an XY problem. Getting input via read is cumbersome and impractical. It's much better to get input via arguments, and then you can leave it to the caller's shell to do the expansions as they intend it.

Instead of

#! /bin/sh -
IFS= read -r var
ls -l -- "$var"

(and remember that calling read without IFS= and without -r is almost never what you want).

Make it:

#! /bin/sh -
var=${1?}
ls -l -- "$var"

And then the caller can do your-script ~/dir or your-script "$HOME/dir" or your-script '$$$weird***/dir' or even your-script $'/dir\nwith\nnewline\ncharacters' as they see fit.

---

^{¹ word expansion in this context refers to parameter expansion, arithmetic expansion and command substitution. That doesn't include filename generation (aka globbing or pathname expansion), tilde expansion nor brace expansion (itself not a standard sh feature). Using a here-document here makes sure ' and "s are left untouched, but note that there still is backslash processing.}