Piping data to a fifo with EOFs every second

Question

I'm trying to pipe some raw binary data through to a fifo (out.pipe), and ensure that EOFs get sent at regular intervals. I do not understand why the following line will not do what I want:

while true; do read -t 1 -N 128 line; echo -n $line > out.pipe; done

It appears as if, when the timeout occurs, $line is given an empty value, and even when it does not occur, all newlines are stripped from the input. This is not my understanding of what should happen according to this document:

-N nchars

read returns after reading exactly nchars characters rather than waiting for a complete line of input, unless EOF is encountered or read times out. Delimiter characters encountered in the input are not treated specially and do not cause read to return until nchars characters are read.

-t timeout

Cause read to time out and return failure if a complete line of input (or a specified number of characters) is not read within timeout seconds. timeout may be a decimal number with a fractional portion following the decimal point. This option is only effective if read is reading input from a terminal, pipe, or other special file; it has no effect when reading from regular files. If read times out, read saves any partial input read into the specified variable name. If timeout is 0, read returns immediately, without trying to read and data. The exit status is 0 if input is available on the specified file descriptor, non-zero otherwise. The exit status is greater than 128 if the timeout is exceeded.

Without the -N 128 parameter on the read, and the -n parameter on the echo, this is very close to what I want. The problem is that newlines are treated special and are sent even when there is no data coming through:

while true; do read -t 1 line; echo $line > out.pipe; done

I've been testing what goes into my pipe with the following line:

while true; do cat out.pipe; done

I've tested sending input through stdin as well as sending HTTP data through with echo -e "GET / HTTP/1.1\r\nHost: hostname\r\n\r\n" | nc localhost 80 | myreadcommand, which was very confusing due to lonely \r characters.

What am I failing to understand? Is there a much better way to do this that I am missing?

Answer 1

read's man page also says:

The terminating <newline> (if any) shall be removed from the input and the results shall be split into fields as in the shell for the results of parameter expansion

It also documents a lot of exceptions based on which parameters you pass to it on how to handle new lines. Reading the whole man page is necessary if such exceptions matter to you.

If you want to ignore certain characters such as \r, that can easily be done with sed:

echo "hi\r\nI am a script" | sed "s/\r//g"

To make the substitution more obvious, you can replace it with something else like:

echo "hi\r\nI am a script" | sed "s/\r/,/g"

It's also important to note that using the -e parameter for echo makes it interpret the backslashed special character combinations which do not actually correspond to a real line feed character present in the input.