Why does double quoting an array expansion in here document not seem to work here?

Question

I have two arrays

$ arr1=( 1 2 2 3)
$ arr2=( 2 3 3 4)

Why does this generate strange output when I follow the good practice of double quoting an array expansion

$ tsort << EOF
> "${arr1[@]}" "${arr2[@]}"
> EOF
"1
"2
2
3
3"
4"

whereas this one generates correct results, when I do not follow the good practice? What shall I do? Thanks.

$ tsort << EOF
${arr1[@]} ${arr2[@]} 
EOF

1
2
3
4

---

I think word splitting is still performed in here document:

$ wc -l << EOF
a    
b  
1
EOF
3

Answer 1

From the bash manual:

[...] all lines of the here-document are subjected to parameter expansion, command substitution, and arithmetic expansion, the character sequence \newline is ignored, and ‘\’ must be used to quote the characters ‘\’, ‘$’, and ‘`’.

It doesn't say anything about removing quotes, or field splitting or filename expansion, so the quotes in a heredoc are taken literally, and don't perform their usual functions.

Answer 2

The other answer is basically "because the manual says so", but I think there is some rationale behind this behavior.

This type of redirection instructs the shell to read input from the current source until a line containing only word (with no trailing blanks) is seen. All of the lines read up to that point are then used as the standard input […] for a command.

[…]

If any part of word is quoted, the delimiter is the result of quote removal on word, and the lines in the here-document are not expanded. If word is unquoted, all lines of the here-document are subjected to parameter expansion, command substitution, and arithmetic expansion […].

(emphasis mine).

Consider main reasons to quote something in a command line:

1. to tell the shell how to split the line and create a list of arguments (two arguments a b vs. one argument "a b");
2. to tell the shell if expansions should be performed ('' vs. "" or unquoted).

The first reason doesn't apply while creating a standard input because standard input is a stream and there is no concept of number of arguments in it. The second reason could apply but the design choice to control expansions solely by quoting or not quoting word ("EOF" vs. EOF) invalidates it.

What shall I do?

You shall remember with here document you create a stream. The reasons to use quotes that wouldn't appear in it have been invalidated. In effect the premise is every quote you use should appear in the stream.

If this worked more as you expected then unescaped quotes would disappear without making any difference. I guess here document would then support \" and \' to denote quotes that should survive. But these would be the only quotes you would really need to use. The current situation simplifies things.

Answer 3

Let's look at what actually happens here by replacing tsort with cat:

$ arr1=( 1 2 2 3 )
$ arr2=( 2 3 3 4 )
$ cat <<END
> "${arr1[@]}" "${arr2[@]}"
> END
"1 2 2 3" "2 3 3 4"

As you can see, the here-document is nothing but a text string with the values of the arrays expanded inside it. The double quotes come from the double quotes in the document itself (the shell only cares about the ${...} bit and won't touch the quote characters).

The output when deleting the double quotes would be the same, but without the double quotes,

1 2 2 3 2 3 3 4

This would be interpreted by tsort as the pairs

1 2   <-- first two numbers from arr1
2 3   <-- last two numbers from arr1
2 3   <-- first two numbers from arr2
3 4   <-- last two numbers from arr2

It's unfortunate that you chose this particular example, as this happens to be exactly the same as

1 2   <-- first number from arr1 and arr2
2 3   <-- second number from arr1 and arr2
2 3   <-- etc.
3 4

i.e., the entries of each array running down in two columns (one column per array).

To generate this second list (correctly), you can't really use a here-document. Instead you could use a shell loop:

for (( i=0; i<${#arr1[@]}; ++i)); do
    printf '%d %d\n' "${arr1[i]}" "${arr2[i]}"
done | tsort