In stream processing and queuing, we have this notion of backpressure, which is that if a producer process is going faster than a consumer process, we should have a mechanism for slowing down the producer to avoid exceeding available memory/storage (without dropping messages, which may or may not be acceptable).
I've been curious for a little while now about whether stdio can be used to exert backpressure in this sense on a producer Unix process (e.g.
foo | bar). It would seem that even if writes to stdout were blocked when a buffer reached capacity, it would still be necessary for the producer process to Do The Right Thing (TM), and not accumulate data in memory waiting to be written to stdout. A single threaded, blocking program would seem to pass the test, but an asynchronous program may have to have it's own internal buffering and backpressure mechanism for in order not to explode with data waiting to be written.
So; to what extent is this possible and what are the particulars?