Why is set-o errexit breaking this read/heredoc expression?

Question

I have been using the pattern below for printing multiline messages to terminal in a bash script.

read -d '' message <<- EOF
    this is a 
    mulitline
    message
EOF
echo "$message"

This has been working - until a couple of days ago the pattern just stopped working. By stopped working, I mean when bash encountered these heredoc expressions in the script - it just seems to do nothing - no output.
The only thing that I can think of thats changed in the last few days is that the environment the scripts are run inside is a Ubuntu 14.04 live USB, versus "full" installs.
Then I discovered that when I move the heredoc before the scripts set -o errexit statement it started working again. i.e. this doesn't work

#!/bin/bash

set -o errexit

read -d '' message <<- EOF
    this is a 
    mulitline
    message
EOF

echo "$message"

result: (nothing)
But this does work

#!/bin/bash

read -d '' message <<- EOF
    this is a 
    mulitline
    message
EOF

echo "$message"

result

$ sudo ./script.sh 
this is a 
mulitline
message

• bash --version - GNU bash, version 4.3.11(1)-release (i686-pc-linux-gnu)

Answer 1

read returns a non-zero exit status if it doesn't find a delimiter. With the delimiter set to the empty string, it uses the NUL byte as a delimiter, and those usually aren't found in text files.

Answer 2

The exit code of the read command is 1 when the end of file (EOF) marker is reached. This will always happen when the delimiter -d is null '' in this special case where the source stream is a heredoc that could not contain a \0.

$ read -d '' message <<-_ThisMessageEnds_
>     this is a
>     multi line
>     message
> _ThisMessageEnds_
$ exitval=$?
$ echo "The exit val was $exitval"
The exit val was 1.

That the exit value is an error (not 0) makes the script exit could be avoided with a AND/OR construct:

read -d '' message <<-_ThisMessageEnds_ || echo "$message"
    this is a
    multi line
    message
_ThisMessageEnds_

That will send the message to the console and yet avoid exiting it with errexit.

But as we are on this path to reduce, why not use this directly:

cat <<-_ThisMessageEnds_
    this is a
    mulitline
    message
_ThisMessageEnds_

No read command executed (more speed), no variable needed, no error from the exit code, less code to maintain.

Answer 3

read -d '' message

reads stdin until the first unescaped (as you didn't add -r) NUL character or the end of the input and stores the data after $IFS and backslash character processing into $message (without the delimiter).

If no unescaped delimiter is found in the input, read's exit status is non-zero. It only returns 0 (success) if a full, terminated record is read.

It's most useful for dealing with NUL-delimited records like the output of find -print0 (though you then need a IFS= read -rd '' record syntax).

Here, you need to include a NUL delimiter in your here-document for read to return successfully. That is however not possible with bash which strips NUL characters from here-documents (that's at least better than yash that strips everything past the first NUL, or ksh93 which seems to enter an infinite loop when a here-document contains a NUL).

zsh is the only shell that can have a NUL in its here documents or store it in its variables or pass NUL characters in arguments to its builtins/functions. In zsh, you can do:

NUL=$'\0'
IFS= read -d $NUL -r var << EOF
1
2
3$NUL
EOF

(zsh also understand read -d '' as a NUL delimiter like bash. read -d $'\0' also works in bash but that does pass an empty argument to read like in read -d '' as bash doesn't support NUL bytes in its command line).

(note that there's an extra newline character after that $NUL)

In bash, you can use a different character:

ONE=$'\1'
IFS= read -d "$ONE" -r var << EOF
1
2
3$ONE
EOF

But you could also do:

var=$(cat <<EOF
message
here
EOF
)

That will still not allow NUL characters. That's however standard code, so you don't need to rely on the zsh/bash specific read -d. Also note that it removes all trailing newline characters, and except in ksh93 when the cat builtin is enabled, that means spawning an extra process and command.

Answer 4

When you use set -o errexit and your script breaks, it means there's something wrong.

Here, it's read, which cannot correctly read your input.

In bash, when you use read -d '', the read builtin will use the null character \0 as line terminator. Therefore, when there's no \0 in your input, read will read all the input into the message variable and will return a non-zero exit status to indicate there was an error:

$ while read -d '' line; do echo "$line"; done < <(printf '1')

prints nothing while:

$ while read -d '' line; do echo "$line"; done < <(printf '1\0')
1

gives you 1.

read will also return non-zero status when it reaches EOF, but that's used for indicating there's no more input to read when you use read with a while loop, so the while loop can be terminated. It's not relevant to your problem.