1

We're implementing an embedded Linux system and have to provide a live updating graph on a website that shows data coming from a background process in the system.

The question is how to optimally share data between:

  1. The data generating process that regularly updates a ring buffer with new values, written in C.

  2. The webserver CGI function that must fetch the latest contents of the buffer. We use Python for CGI's (nginx+wsgi+flask).

I currently lean towards making a unix socket solution for this, but I gather this would require multithreading in the C program to make sure the background process isn't disturbed.

I wonder if there isn't a simpler way. Could we map the buffer memory directly to be a virtual file? What about using a normal file on a RAM disk as the buffer and just seeking and writing to it?

0

Simple solution using files and atomic rename which can be easily implemented using scripting languages.

Sender

  • write data into file A
  • rename file A to B

The rename is atomic, and it can be done while the receiver is processing the old file.

The sender will not block if the receiver does not read the data.

Receiver

  1. If it is no problem to process the same data twice:

    • open file B
    • read data
    • close file

    This option also works when there is more than one receiver, e.g. multiple CGI processes of the web server.

  2. If the receiver should not process the same data twice:

    • rename file B to C
    • if successful, read and process file C
    • optionally delete file C

    This option works with a single receiver only.

In both cases old data will be lost if the sender writes data faster than the receiver can read it.

Of course there are other options.

  • One of the other options is POSIX IPC, which offers shared memory, message queues, and semaphores. Google is you friend. – Johan Myréen Feb 22 at 16:14
  • Check out zeromq ... – Murray Jensen Feb 23 at 4:26
  • Thanks for the answer, but since the data generating process will append the ring buffer with new values multiple times a second, it doesn't seem like a good fit. I'd prefer not to have to save the whole buffer in a new file on updates. – Larsp Feb 25 at 9:23
  • Thanks for the tip about zeromq, very interesting. I'll also give POSIX IPC a look. – Larsp Feb 25 at 9:23
  • @Larsp Of course you can choose a different solution. If you think this would be a performance problem I suggest to test and measure first. You should add your timing requirements to the question. How often does the sender update the data? How much data is one set? How often is the receiver expected to display the data? – Bodo Feb 25 at 9:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.