Is there a way of implementing the publish / subscribe pattern from the command line without using a server process? This need only work on one machine.

The main thing I want to avoid by not having a server process is having configure a machine to use these tools. I'm also quite keen on not having to deal with the possibility of my server process dying.|

This might look something like:

# client 1
subscribe name | while read line; do echo $line; done

# client 2
subscribe name | while read line; do echo $line; done

# server
echo message | publish name

Related links

  • POSIX ipc provides a serverless message queue and there are command-line clients for it (1) (2) (3). This could be used together with some sort of state storage to implement the above.
  • ZMQ provides a protocol for pub / sub communication. There are command-line tools analogous to nc for using ZMQ, such as zmcat. These could be used to set up a minimal command-line pub/sub pattern with a server.
  • Linux provides another IPC mechanism called named pipes (c.f. mkfifo). I don't know what the intended behaviour with multiple consumers is. But some initial experimentation suggests that each message is only received by one of the consumers
  • Do named pipes do what you want? If not, edit to explain why. – Michael Homer Nov 22 '17 at 19:43
  • Hmm... as far as I'm aware named pipes only have one consumer, or at least the lines are only received by the earliest connected pipe. – Att Righ Nov 22 '17 at 19:46
  • Hmm, the O_ASYNC flag to open might cause data to be sent to all connected processes. – Att Righ Nov 22 '17 at 19:58
  • I don't think this async mode is particularly related. It just seems like a way to "wake up a process" when it should be reading or writing. – Att Righ Nov 22 '17 at 20:17
  • 1
    Maybe. I would note that language as a rule tends to be ambiguous because the world is quite big :P. I think most people would view using a pipe as quite different in nature from, say, a RabbitMQ server. I guess what I really mean is "without having to setup, manage, configure and monitor a process." I'll update the question to this effect. Things like dbus are an interesting middleg-round: always running and (I think?) configuration-free. This does make me depend on X... I think. – Att Righ Nov 22 '17 at 22:50

All subscribers need to be notified of new data in a way that doesn't affect other subscribers and the server must not have to keep track of what data subscribers have received. This makes FIFO useless for this purpose. Ironically a regular file will do exactly what you want because file descriptors on regular files keep track of file changes. You can combine this with overwrite which ensures all changes are published before a new overwrite occurs meaning you are only storing one message.

touch pubsub

tail -f pubsub | while read line; do echo $line; done
tail -f pubsub | while read line; do echo $line; done

echo "message" | cat > pubsub

You will get "file truncated" on standard error which is expected behavior but if you don't want to see it add 2> /dev/null

tail is actually doing everything read and echo do but its written like that because I assume you want to incorporate it in a script.

  • 1
    If you want to do this only in memory I'm pretty sure there's a way. Easiest would be to put pubsub in /run – jdwolf Nov 22 '17 at 23:04
  • That is ingenious and wonderfully hacky!. I guess the only downside to this is the pubsub eventually becomes huge. But this isn't a problem for me. Actually it doubles up as log file :) – Att Righ Nov 22 '17 at 23:17
  • I had a look under the hood and tail -f is using inotify rather than polling, so updates are "immediate". As an aside, one probably wants a -n 0 on the tail as well. – Att Righ Nov 22 '17 at 23:22
  • When the -n option is used with -f its basically like a chuck size for lines. Every line is still printed. – jdwolf Nov 22 '17 at 23:33
  • 1
    Good point about rejoining. Although I wasn't concerned with that considering > will always overwrite. – jdwolf Nov 23 '17 at 4:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.