1

I'm trying to pipe some raw binary data through to a fifo (out.pipe), and ensure that EOFs get sent at regular intervals. I do not understand why the following line will not do what I want:

while true; do read -t 1 -N 128 line; echo -n $line > out.pipe; done

It appears as if, when the timeout occurs, $line is given an empty value, and even when it does not occur, all newlines are stripped from the input. This is not my understanding of what should happen according to this document:

-N nchars

read returns after reading exactly nchars characters rather than waiting for a complete line of input, unless EOF is encountered or read times out. Delimiter characters encountered in the input are not treated specially and do not cause read to return until nchars characters are read.

-t timeout

Cause read to time out and return failure if a complete line of input (or a specified number of characters) is not read within timeout seconds. timeout may be a decimal number with a fractional portion following the decimal point. This option is only effective if read is reading input from a terminal, pipe, or other special file; it has no effect when reading from regular files. If read times out, read saves any partial input read into the specified variable name. If timeout is 0, read returns immediately, without trying to read and data. The exit status is 0 if input is available on the specified file descriptor, non-zero otherwise. The exit status is greater than 128 if the timeout is exceeded.

Without the -N 128 parameter on the read, and the -n parameter on the echo, this is very close to what I want. The problem is that newlines are treated special and are sent even when there is no data coming through:

while true; do read -t 1 line; echo $line > out.pipe; done

I've been testing what goes into my pipe with the following line:

while true; do cat out.pipe; done

I've tested sending input through stdin as well as sending HTTP data through with echo -e "GET / HTTP/1.1\r\nHost: hostname\r\n\r\n" | nc localhost 80 | myreadcommand, which was very confusing due to lonely \r characters.

What am I failing to understand? Is there a much better way to do this that I am missing?

  • i ended up using a python script to do this. – Nacht Jun 13 '16 at 6:21
2

read's man page also says:

The terminating <newline> (if any) shall be removed from the input and the results shall be split into fields as in the shell for the results of parameter expansion

It also documents a lot of exceptions based on which parameters you pass to it on how to handle new lines. Reading the whole man page is necessary if such exceptions matter to you.

If you want to ignore certain characters such as \r, that can easily be done with sed:

echo "hi\r\nI am a script" | sed "s/\r//g"

To make the substitution more obvious, you can replace it with something else like:

echo "hi\r\nI am a script" | sed "s/\r/,/g"

It's also important to note that using the -e parameter for echo makes it interpret the backslashed special character combinations which do not actually correspond to a real line feed character present in the input.

  • I have read the entire page. Is a newline not just the default delimiter? Wouldn't that make the description of -N more specific and thus overrule the text you quoted? How should I be piping data with regular EOFs? I want my data to come through unmodified. – Nacht Jun 6 '16 at 6:02
  • It actually does keep the line feeds when you use -N, but not the `\`. – Julie Pelletier Jun 6 '16 at 6:09
  • I'm confused :/ I am not sending any backslashes to read. Maybe that's what I should be doing? – Nacht Jun 6 '16 at 6:17
  • The complexity of the task lies in the different levels of the shell's interpretation of what you're doing. – Julie Pelletier Jun 6 '16 at 6:20
  • 1
    You can always use the timeout command to impose a timeout on cat for example. – Julie Pelletier Jun 6 '16 at 7:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.