3

I need some help debugging my tcsh script. It uses heredocs. The code:

<pre_setup> <<EOF1
<setup> <<EOF2

<command>

exit 0
EOF2
exit 0
EOF1

The <pre_setup> is some pre setup command (like wash) and setup is some setup that runs after it. It works but I noticed that if setup contains single quotes, it fails. My setup looks like:

run_setup -cmd '$SOME_ENV -o outdir'

The run_setup sets $SOME_ENV and executes $SOME_ENV -o outdir. So running:

wash -n group_name <<EOF1
run_setup -cmd '$SOME_ENV -o outdir' <<EOF2

<command>

exit 0
EOF2
exit 0
EOF1

Fails with SOME_ENV: Undefined variable.. It's not problem of wash in pre_setup because If I use setenv X 1 (or cat) as pre_setup, it still fails with the same error. It has something to do with the way heredocs works. How can I make it work? Is it problem of escaping? How to debug heredocs?

Also I will mention that running run_setup -cmd '$SOME_ENV -o outdir' in the shell, works. And even running pre_setup and then this setup works. So it hes something to do with heredocs.

EDIT: Wrapping it with quotes, break code like this:

/bin/tcsh <<EOF
$WORK_AREA/script.sh.$1
echo \$status > $WORK_AREA/.$1.result
exit 0
EOF

where $1$ is the the input of the script which has this heredocs. When wrapping with single quotes, the $1 in $WORK_AREA/script.sh.$1 is empty. But without the quotes, it works. Why? How can I solve it?

1 Answer 1

4

A regular here-doc is a bit like a double-quoted string in that it expands variables and such:

foo=there
cat <<EOF
hello $foo
EOF

outputs "hello there". In sh-likes, it's usually not an error if the variable isn't defined, you just get an empty string.

Note that the quotes are not relevant. They're not special inside a here-doc, any more than single-quotes are special inside double-quotes. E.g. this also expands $foo: echo "hello '$foo'"

It seems you'd want to prevent the expansion, and you can do that by making the delimiter quoted. Here seems to be a bit of a difference between an sh-like shell and (t)csh, though.

In sh-likes, this:

foo=there
cat <<'EOF'
hello $foo
EOF

prints hello $foo, i.e. doesn't try to expand the variable.

In (t)csh it seems you need to quote the ending delimiter in the same way, so:

foo=there
cat <<'EOF'
hello $foo
'EOF'

The tcsh man page in Debian says:

<< word
Read the shell input up to a line which is identical to word. word is not subjected to variable, filename or command substitution, and each input line is compared to word before any substitutions are done on this input line.

Unless a quoting \, ", ' or ` appears in word[, ] variable and command substitution is performed on the intervening lines, allowing \ to quote $, \ and `.

I take it that "up to a line identical" (and "compared before substitutions") implies being quote-identical. (This is unlike e.g. what how Bash's man page describes it: "If any part of word is quoted, the delimiter is the result of quote removal on word, [...]".)

The second part is just the same as in POSIX sh, a quoted delimiter inhibits expansions in the here-doc text.


If you feed a shell from the here-doc, you have two shells that can expand any variables. The outer shell that builds the here-doc, and the inner one it then runs to read it. And they may have a different idea of what the variables are, e.g. the inner one won't have the arguments to the main script. And the quoting and escaping determines which one of them expands the variable.

So if you run this:

#!/usr/bin/tcsh
set foo = outer
echo in outer shell, foo is: $foo

tcsh <<EOF
set foo = inner
echo "not escaped, outer shell expands: $foo"
echo "    escaped: inner shell expands: \$foo"
EOF

The inner shell sees

set foo = inner
echo "not escaped, outer shell expands: outer"
echo "    escaped: inner shell expands: $foo"

with the first $foo already expanded, and the backslash removed from the \$foo. If you were to use <<'EOF' instead, the outer shell would expand nothing.

So here, you need to use <<EOF without quotes, and take care to escape or not escape all the variables individually according to what you want to happen.

4
  • 1
    using 'EOF' (with quotes) actually works. Can you please point me to reliable source that explain why it works? Maybe tcsh docs?
    – vesii
    Apr 6, 2021 at 13:49
  • @vesii, the man page seemed to indeed have a relevant passage, edited. I'm not really sure what sources there are for tcsh, e.g. the website at tcsh.org is a bit bare. But the latest source code tarball is dated last November, so it's not totally dead at least.
    – ilkkachu
    Apr 6, 2021 at 14:38
  • Thanks for the detailed answers! Do you mind check my edit? I have added a corner case in which it does not work for me.
    – vesii
    Apr 6, 2021 at 19:21
  • @vesii, ok, right. You have two shells there. If you want the outer one to expand some variables, then yeah, you can't use << 'EOF', and you need to escape the ones going to the inner shell individually. see edit for a demo script. (That said, I'm not exactly sure why you're feeding another shell from a shell script, but I guess that's another question...)
    – ilkkachu
    Apr 6, 2021 at 20:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .