1

I am writing a data logging app, all programs are started like:

./program > out.bin

The data collector periodically pools the stdout output files and reads the data.

The issue is that the IO streams are buffered and if some program outputs data in like 1 byte per second, it takes a lot of time (up too 4k seconds with default 4kB buffer size) before the data are actually flushed out.

My question is how to force the stdout/pipe/printf buffer to flush externally, i.e. call externally something like fflush(stdout).

I have read various sites like Turn off buffering in pipe , but I can not disable the buffers as it have huge IO performace impact (measured).

I am looking for high performace solution for production and these following conditions are always met:

  • the program (data producer) PID is always known
  • the output is always a file with known path
  • the data logging process has full root access
2
gdb -p PID -batch -ex 'p fflush(stdout)'

As with any debugging and hacking, YMMV.

2

Do you have access to the source of the running programs?

Forcing a flush of an arbitrary executable is, while not theoretically impossible, very difficult. You would need to find the fflush function in the code and the stdout argument, then interrupt the execution of the program, arrange for the call to fflush, then continue execution. If the program is using a shared library, that at least makes the first part easier, finding fflush and stdout, but you still need to simulate the call. Also, for an unknown binary, you can't know whether it used stdio or whether it implements its own buffering mechanism. Knowing the path of the output file will not help you.

You can try to use gdb, then use the attach command to attach to the process. Maybe gdb can call the fflush function.

If you have the source to the program, implement a signal handler that flushes the buffer and just send the signal when you want the buffer flushed.

You can try a pipe, maybe the programs don't buffer if the output is a pipe. Change it to

./program | your_program > out.bin

Your program can accept the input, buffer it and flush the buffer when it receives a signal. It would still add CPU overhead, but not disk overhead.

  • Unfortunately I can not modify the data producer programs in any way. – Mvorisek Jun 10 at 9:36
  • I added an idea about gdb. But I think stdout is a macro and not available as a symbol at runtime. – RalfFriedl Jun 10 at 9:39
  • 2
    Does in use dynamic libraries? May be you can change the libraries, to do the same. – ctrl-alt-delor Jun 10 at 10:24
  • 1
    I added an idea about using a pipe. @ctrl-alt-delor's idea with changes in a dynamic library might also work. – RalfFriedl Jun 10 at 10:38
  • @ctrl-alt-delor I presume most of them yes, the typical programs are java, php cli, ... I will definitely test. – Mvorisek Jun 10 at 10:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.