13

I have been using the pattern below for printing multiline messages to terminal in a bash script.

read -d '' message <<- EOF
    this is a 
    mulitline
    message
EOF
echo "$message"

This has been working - until a couple of days ago the pattern just stopped working. By stopped working, I mean when bash encountered these heredoc expressions in the script - it just seems to do nothing - no output.
The only thing that I can think of thats changed in the last few days is that the environment the scripts are run inside is a Ubuntu 14.04 live USB, versus "full" installs.
Then I discovered that when I move the heredoc before the scripts set -o errexit statement it started working again. i.e. this doesn't work

#!/bin/bash

set -o errexit

read -d '' message <<- EOF
    this is a 
    mulitline
    message
EOF

echo "$message"

result: (nothing)
But this does work

#!/bin/bash

read -d '' message <<- EOF
    this is a 
    mulitline
    message
EOF

echo "$message"

result

$ sudo ./script.sh 
this is a 
mulitline
message
  • bash --version - GNU bash, version 4.3.11(1)-release (i686-pc-linux-gnu)

4 Answers 4

18

read returns a non-zero exit status if it doesn't find a delimiter. With the delimiter set to the empty string, it uses the NUL byte as a delimiter, and those usually aren't found in text files.

4
  • 5
    Strictly speaking, it's \0, not empty string.
    – cuonglm
    Feb 23, 2016 at 11:09
  • 2
    @cuonglm, more precisely, when an empty argument is passed to -d, then the delimiter is \0 (a NUL byte). A special case mostly the result of a coding accident (though the bash maintainer won't admit it and explains why it's not documented). Feb 24, 2016 at 13:34
  • "...which is always the case when the delimiter is an empty string." As pointed out by others, it's actually \0 -- and that doesn't always fail, you can use read -d '' to process the output of find ... -print0, for example.
    – solidsnack
    Feb 28, 2016 at 23:20
  • It's documented in now, in the 5.0 man page at least.
    – ilkkachu
    Aug 10, 2021 at 12:28
12

The exit code of the read command is 1 when the end of file (EOF) marker is reached. This will always happen when the delimiter -d is null '' in this special case where the source stream is a heredoc that could not contain a \0.

$ read -d '' message <<-_ThisMessageEnds_
>     this is a
>     multi line
>     message
> _ThisMessageEnds_
$ exitval=$?
$ echo "The exit val was $exitval"
The exit val was 1.

That the exit value is an error (not 0) makes the script exit could be avoided with a AND/OR construct:

read -d '' message <<-_ThisMessageEnds_ || echo "$message"
    this is a
    multi line
    message
_ThisMessageEnds_

That will send the message to the console and yet avoid exiting it with errexit.

But as we are on this path to reduce, why not use this directly:

cat <<-_ThisMessageEnds_
    this is a
    mulitline
    message
_ThisMessageEnds_

No read command executed (more speed), no variable needed, no error from the exit code, less code to maintain.

3
  • thanks, good to know about using || to prevent the script exit. I ended up taking errexit out of the script completely, I have had problems with it not triggering exit when it should - so it seems too tricky to be useful. I have also considered the simpler pattern with cat <<- EOF message EOF too . It was just nice to have the message in a variable that could be passed to a function if necessary Feb 23, 2016 at 6:53
  • 3
    "No read command executed (more speed)" -> read is a bash-builtin, cat an external program, so I doubt cat is faster (as fork-exec is necessary).
    – spawn
    Dec 20, 2020 at 12:02
  • +1 || e.g. read -d '' -r foo <<EOF || :...
    – nymo
    Mar 3 at 20:00
8
read -d '' message

reads stdin until the first unescaped (as you didn't add -r) NUL character or the end of the input and stores the data after $IFS and backslash character processing into $message (without the delimiter).

If no unescaped delimiter is found in the input, read's exit status is non-zero. It only returns 0 (success) if a full, terminated record is read.

It's most useful for dealing with NUL-delimited records like the output of find -print0 (though you then need a IFS= read -rd '' record syntax).

Here, you need to include a NUL delimiter in your here-document for read to return successfully. That is however not possible with bash which strips NUL characters from here-documents (that's at least better than yash that strips everything past the first NUL, or ksh93 which seems to enter an infinite loop when a here-document contains a NUL).

zsh is the only shell that can have a NUL in its here documents or store it in its variables or pass NUL characters in arguments to its builtins/functions. In zsh, you can do:

NUL=$'\0'
IFS= read -d $NUL -r var << EOF
1
2
3$NUL
EOF

(zsh also understand read -d '' as a NUL delimiter like bash. read -d $'\0' also works in bash but that does pass an empty argument to read like in read -d '' as bash doesn't support NUL bytes in its command line).

(note that there's an extra newline character after that $NUL)

In bash, you can use a different character:

ONE=$'\1'
IFS= read -d "$ONE" -r var << EOF
1
2
3$ONE
EOF

But you could also do:

var=$(cat <<EOF
message
here
EOF
)

That will still not allow NUL characters. That's however standard code, so you don't need to rely on the zsh/bash specific read -d. Also note that it removes all trailing newline characters, and except in ksh93 when the cat builtin is enabled, that means spawning an extra process and command.

3
7

When you use set -o errexit and your script breaks, it means there's something wrong.

Here, it's read, which cannot correctly read your input.

In bash, when you use read -d '', the read builtin will use the null character \0 as line terminator. Therefore, when there's no \0 in your input, read will read all the input into the message variable and will return a non-zero exit status to indicate there was an error:

$ while read -d '' line; do echo "$line"; done < <(printf '1')

prints nothing while:

$ while read -d '' line; do echo "$line"; done < <(printf '1\0')
1

gives you 1.

read will also return non-zero status when it reaches EOF, but that's used for indicating there's no more input to read when you use read with a while loop, so the while loop can be terminated. It's not relevant to your problem.

7
  • thanks, the other answers explained that my read expression, was breaking the script. But thats a good point that read also exits non-zero when used with a read, while loop. For that reason I dont think set -o errexit is reliable, because sometimes commands need to return non-zero as part of normal program flow Feb 23, 2016 at 11:08
  • @the_velour_fog: It's reliable, if you want it's a part of program flow, then use the flow control, like if read ...
    – cuonglm
    Feb 23, 2016 at 11:14
  • 1
    The correct description is in searching for a NUL delimiter the EOF was found first. What should read do on finding an EOF: report an error, which it does.
    – user79743
    Feb 24, 2016 at 16:10
  • 1
    @BinaryZebra, yes, though I understand cuonglm used the while loop for illustration that read returns with failure (and exit the loop) upon a not-found delimiter (though I'd agree it probably just adds more confusion). Feb 24, 2016 at 17:13
  • 1
    In essence, we're all saying the same thing, just more or less clearly, there's not much point arguing any more about it. Feb 24, 2016 at 17:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .