Let's assume you have a pipeline like the following:
$ a | b
If b
stops processing stdin, after a while the pipe fills up, and writes, from a
to its stdout, will block (until either b
starts processing again or it dies).
If I wanted to avoid this, I could be tempted to use a bigger pipe (or, more simply, buffer(1)
) like so:
$ a | buffer | b
This would simply buy me more time, but in the end a
would eventually stop.
What I would love to have (for a very specific scenario that I'm addressing) is to have a "leaky" pipe that, when full, would drop some data (ideally, line-by-line) from the buffer to let a
continue processing (as you can probably imagine, the data that flows in the pipe is expendable, i.e. having the data processed by b
is less important than having a
able to run without blocking).
To sum it up I would love to have something like a bounded, leaky buffer:
$ a | leakybuffer | b
I could probably implement it quite easily in any language, I was just wondering if there's something "ready to use" (or something like a bash one-liner) that I'm missing.
Note: in the examples I'm using regular pipes, but the question equally applies to named pipes
While I awarded the answer below, I also decided to implement the leakybuffer command because the simple solution below had some limitations: https://github.com/CAFxX/leakybuffer