Utility to buffer an unbounded amount of data in a pipeline?

Question

Is there a utility that I can stick in a pipeline to decouple read and write speeds?

$ producer | buf | consumer

Basically, I want a utility buf that reads its input as fast as possible, storing it in memory so consumer can take its sweet time while producer runs as fast as possible.

Answer 1

The pv (pipe viewer) utility can do this (with the -B option) and a lot more, including giving you progress reports.

Answer 2

you can use dd:

producer | dd bs=64K | consumer

It's available on every unix.

Answer 3

Take a look at mbuffer. It can buffer to memory or memory mapped file(-t/-T).