Bash Pipeline Stops Working

Question

I am writing a bash script to process output from a mosquitto_sub call:

function testPipe(){
 read foo
 IFS=' ' read -ra ARRAY <<< "$foo"
 topic=${ARRAY[0]}
 cmd=${ARRAY[1]}
 echo "topic = $topic cmd = $cmd"
 cat
}

function pipeTee(){
 tee -a mqtt_broker.log
}

mosquitto_sub -h $HOST -p $PORT -t $TOPIC \
      -u $USER -P $PASS -v | pipeTee | testPipe

I tried to follow along from Pipe demon output to a function . However, here is what I observe, the first message arrives over the wire and the function testPipe is called and I see the parsed line. All subsequent messages are printed to the console raw, that is as they come in from network.

What might I be doing wrong? Does the "read foo" not consume the input in STDIN?

Answer 1

If you start the broker (with mosquitto), then subscribe to a my/test topic with

mosquitto_sub -t my/test -v

you can then publish simple messages and see what you will receive. For example,

mosquitto_pub -t my/test -m inline

will show up in the subscription output as a single line

my/test inline

If all your messages are of this type, you should simply do a loop to call your function:

... | pipeTee | while testPipe; do : ; done

but your function cannot use cat, or it will never return. If you want to send multiline messages, you will need to do replace the cat with a read inside another while loop that somehow detects the end of the message. For example, if you publish with

echo -e 'my multi\nline test' | mosquitto_pub -t my/test -s

you will see 3 lines:

my/test my multi
line test
<there is a blank line here>

The blank line is because echo adds a \n to the string. You would then need to replace cat with something like

while read data && [ "$data" != "" ]
do    echo "Got $data"
done

Since you control what you publish, you might prefer sending single lines of data, or ending messages with a special string you could look for easily like ===eof===.