I am writing a bash script to process output from a mosquitto_sub call:

function testPipe(){
 read foo
 IFS=' ' read -ra ARRAY <<< "$foo"
 echo "topic = $topic cmd = $cmd"

function pipeTee(){
 tee -a mqtt_broker.log

mosquitto_sub -h $HOST -p $PORT -t $TOPIC \
      -u $USER -P $PASS -v | pipeTee | testPipe

I tried to follow along from Pipe demon output to a function . However, here is what I observe, the first message arrives over the wire and the function testPipe is called and I see the parsed line. All subsequent messages are printed to the console raw, that is as they come in from network.

What might I be doing wrong? Does the "read foo" not consume the input in STDIN?

  • read foo only reads one line of data. If you want to read all line into an array, look into mapfile and use a for loop to iterate over the array. – jordanm Apr 15 '18 at 5:25
  • There is only one line of text. – Greg S Apr 15 '18 at 5:40
  • One more thing to add, all of the data does successfully flow into the log file. If I send 10 commands, the pipeTee will record all 10 messages. While the other function parses only the first call. – Greg S Apr 15 '18 at 5:42
  • 2
    testPipe ends with cat which will gobble up the rest of the input. – meuh Apr 15 '18 at 6:27
  • @meuh If I eliminate the cat, the parse code is called exactly once and then neither the raw input nor the parsed output shows up again. I thought that the message of unix.stackexchange.com/questions/294402/… was that you needed the cat. – Greg S Apr 15 '18 at 13:50

If you start the broker (with mosquitto), then subscribe to a my/test topic with

mosquitto_sub -t my/test -v

you can then publish simple messages and see what you will receive. For example,

mosquitto_pub -t my/test -m inline

will show up in the subscription output as a single line

my/test inline

If all your messages are of this type, you should simply do a loop to call your function:

... | pipeTee | while testPipe; do : ; done

but your function cannot use cat, or it will never return. If you want to send multiline messages, you will need to do replace the cat with a read inside another while loop that somehow detects the end of the message. For example, if you publish with

echo -e 'my multi\nline test' | mosquitto_pub -t my/test -s

you will see 3 lines:

my/test my multi
line test
<there is a blank line here>

The blank line is because echo adds a \n to the string. You would then need to replace cat with something like

while read data && [ "$data" != "" ]
do    echo "Got $data"

Since you control what you publish, you might prefer sending single lines of data, or ending messages with a special string you could look for easily like ===eof===.

  • Yes! It was the infinite loop that I didn't have!!! while testPipe; do : ; done Thank you very much!!! – Greg S Apr 15 '18 at 17:11
  • Dumb question, why did the parsing function need the infinite loop while the other did not? – Greg S Apr 15 '18 at 17:19
  • pipeTee runs tee which never returns, so you can only call this script once. – meuh Apr 15 '18 at 17:26
  • Okay, but when I remove "cat" from testPipe it doesn't work unless I incorporate an infinite loop. – Greg S Apr 15 '18 at 17:40
  • because testPipe does a single read which will only read one line of input. The function will finish and control will return to where it was called, at the end of the pipe. I'm sure there must be some bash tutorials online that would provide better examples. – meuh Apr 15 '18 at 18:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.