Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a text file that I'm using as a template, it looks like this:

Hostname     : $HOSTNAME
Host Address : $HOSTADDRESS

My bash script sets two variables, HOSTNAME and HOSTADDRESS, reads the template file then does an eval to expand $HOSTNAME and $HOSTADDRESS:

HOSTNAME="SH_SQL_0089"
HOSTADDRESS="172.16.3.44"
TEMPLATE=`cat template.txt`
MESSAGE=`eval echo $TEMPLATE`

The resultant value of MESSAGE is:

Hostname     : SH_SQL_0089
Host Address : 172.16.3.44

Is it possible to reduce the last two lines to something like:

MESSAGE=$(cat template.txt | eval echo ????)

I tried using xargs:

MESSAGE=$( cat template.txt | xargs -i bash -c "eval echo {}" )

But it only substitutes $HOSTNAME and strips the carriage returns.

The reason I want to do this is that I need to add a third processing step and pipe the eval'd output to that. I feel I'm so close.

share|improve this question
add comment

4 Answers

up vote 2 down vote accepted

First, xargs can't work here because the substitutions would be executed in a subprocess (the bash processes that xargs launches). But only environment variables are passed to subprocesses, not shell variables. Evidently HOSTNAME was already in your script's environment when it started, but HOSTADDRESS isn't. At this point you might be tempted to export all variables, but even then xargs isn't a good solution because it suffers from numerous quoting issues − if your template contains \"' or if you want to retain whitespace, you're toast.

Now, looking at your current code: MESSAGE=`eval echo $TEMPLATE` is a complex way of writing eval MESSAGE=$TEMPLATE, except for quoting issues. And you do have quoting issues; for example you'll have noticed that all your whitespace has been collapsed. You've heard of Bobby Tables, haven't you? Shell expansion rules are pretty complex, but there are some rules that'll keep you sane:

  • Always double quotes around variable substitutions "$foo" and command substitutions "$(bar)". You may violate this rule if you understand why you need to omit the quotes.
  • Use $(…) rather than `…` for command substitutions. Quoting inside the backquote form is arcane and not portable, whereas quoting inside $(…) works normally.
  • Use eval only if pressed at gunpoint. If you do, be very careful, and make it as simple as possible.

So what can go wrong with eval MESSAGE="$TEMPLATE"? It makes the shell evaluate

MESSAGE=Hostname     : $HOSTNAME
Host Address : $HOSTADDRESS

Oops, we need quotes around the part that should be the value of MESSAGE. The quotes have to get through to eval, so they need to go literally through the first stage of shell expansion: eval MESSAGE="\"$TEMPLATE\"".

MESSAGE="Hostname     : $HOSTNAME
Host Address : $HOSTADDRESS"

Better, but now you have exactly the Bobby Tables injection pattern — what if the template contains a quote? Then you need to escape the quotes. The four characters that have a special meaning between double quotes are \"$`, and you want $ to retain its meaning, so add a backslash before the other three.

TEMPLATE=$(sed -e 's/[\\"`]/\\&/g' <template.txt)
eval MESSAGE="\"$TEMPLATE\""

Now the shell will evaluate

MESSAGE="Hostname     : $HOSTNAME
Host Address : $HOSTADDRESS
Name         : Bobby \"drop\" O'Tables"

and all's good.

Note that proper quoting here is to protect against accidental parsing problems; whoever controls the template still has shell access (with $(hello)).

If you wanted the template included in your shell script, this would be naturally done with a heredoc.

MESSAGE=$(cat <<EOF)
Hostname     : $HOSTNAME
Host Address : $HOSTADDRESS
EOF

But with an external template you'd need to do two steps of evaluation, hence use eval, and bash's parsing is quite buggy when it comes to funky stuff like heredocs inside eval. There's surely a way that works, at least with bash 4, but I don't recommend risking it.

share|improve this answer
    
Excellent answer Gilles. –  Kev Jun 21 '11 at 16:42
add comment

You want the shell to apply substitutions of variables, which means that the template text must be read by the shell itself as part of a command, but the template is also multi-line text so it's a bit tricky to get into the line-oriented processing normally done by UNIX commands.

One approach would be to use bash here-documents (all commands shown as typed at the shell prompt):

  1. create the template file:

    $ cat template.txt
    Hi, $NAME
    
    Welcome to $PLACE
    
  2. export the variables that should be substituted in the template text:

    $ export NAME=Bob
    $ export PLACE='unix&linux'
    
  3. create a shell variable that holds a single "newline" \n character; in bash the easiest way is to simply open a string, type a newline and close it:

    $ newline='
    > '
    
  4. finally, call bash to do the substitution:

    $ bash -c "cat <<__EOF__${newline}$(cat template.txt)${newline}__EOF__"
    Hi, Bob
    
    Welcome to unix&linux
    

Why does this work? Let's deconstruct it from the bottom: you're asking bash to execute a command (by the -c option), but the variable substitution ${newline} and the command expansion $(cat ...) happen in the parent shell before bash -c ... is actually executed. So the result is that bash -c sees a command string that embeds newlines and the whole template text. It's as if you had typed the following at a bash prompt:

cat <<__EOF__
...contents of template.txt...
__EOF__

Also note that the substitution variables need to be exported, since the variables are assigned in the parent shell but it's the "child" bash that is doing the substitution.

Note, however, that this approach can easily lead to quoting issues: since the template text is interpreted by the bash shell, backtics are expanded, which might result in commands being executed on your system.

share|improve this answer
add comment

You might even (double) eval an embedded single-quoted string for experiment's sake:

export HOSTNAME HOSTADDRESS
(
NL='
'
DELIM=$'\177'
esc="'\''"
export IFS=""
HOSTNAME="SH_SQL_00'8'9"
HOSTADDRESS='172.16.3.44 Bobby "drop" O'\''Tables ${PWD} $(ls)'
printf '%s\n' 'Hostname : $HOSTNAME' 'Host Address : $HOSTADDRESS' > template.txt
TEMPLATE="$(<template.txt)"
TEMPLATE="${TEMPLATE//\'/${esc}}"       # escape every single quote: ' --> '\'' (which later will become ''\''' by '${TEMPLATE}')
TEMPLATE="${TEMPLATE//${NL}/${DELIM}}"  # replace every newline char with a delim char
evalstr="echo '${TEMPLATE}'"
#printf '\n%s\n\n' "${evalstr}" | LC_ALL=C vis -fotc
set -xv
eval eval "${evalstr}"
) | LC_ALL=C tr '\177' '\n' | nl
share|improve this answer
add comment
MESSAGE=$(eval $(cat template.txt))
share|improve this answer
    
I think you meant $(eval echo $(cat ... - but that would collapse the text into a single line. –  Riccardo Murri Jun 21 '11 at 8:55
    
I'm not sure what you meant, but it's not going to help Kev write working code. –  Gilles Jun 21 '11 at 9:00
    
@Riccardo: Yes, thanks. –  MSpike Jun 22 '11 at 6:47
    
Gilles: I think my solution is working, with the echo mentioned by Riccardo, and I think it was the answer for his question... Anyway she/he got a solution already :) –  MSpike Jun 22 '11 at 6:49
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.