Change buffer size of named pipe

Question

I hear that for named pipes, writes that are smaller than about 512bytes are atomic (the writes won't interleave).

Is there a way to increase that amount for a specific named pipe?

something like:

mkfifo mynamedpipe --buf=2500

Supposedly this is the full documentation: http://www.gnu.org/software/coreutils/manual/html_node/mkfifo-invocation.html#mkfifo-invocation

man mkfifo takes me to that page.

Answer 1

A fifo file is just a type of file which when opened for both reading and writing instantiates a pipe like a pipe() system call would.

On Linux at least, the data that transits though that pipe is not stored on the file system at all (only in the kernel as kernel memory). And the size attribute of the fifo file is not relevant and is always 0.

On Linux, you can change the size of a pipe buffer (whether that pipe has been instantiated with pipe() or via opening a fifo file) with the F_SETPIPE_SZ fcntl(), though for unprivileged users, that's bound by /proc/sys/fs/pipe-max-size. Any of the writer or reader to the pipe can issue that fcntl() though it makes more sense for the writer to do it. In the case of a named pipe, you'd need to do that for each pipe instantiated though the fifo file.

$ mkfifo fifo
$ exec 3<> fifo # instantiate the pipe
$ seq 20000 > fifo
^C  # seq hangs because it's trying to write more than 64KiB
$ exec 3<&- 3<> fifo # close the first pipe and instantiate a new one
$ (perl -MFcntl -e 'fcntl(STDOUT, 1031, 1048576)'; seq 20000) > fifo
$ # went through this time

Above, I used perl to issue the fcntl(), harcoding the value of F_SETPIPE_SZ (1031 on my system).

Answer 2

Instead of changing F_SETPIPE_SZ, there is a bash shell implementation that have the same effect, replacing the expression fifo by a substitution of processus like this : >(stdbuf -o100M grep -a '^' > fifo) (it will be substituted at runtime by the file descriptor /dev/fd/63)

mkfifo fifo
exec 3<> fifo
seq 12000000 > >(stdbuf -o100M grep -a '^' > fifo)

Not all filters are sensitive to changing buffering mode by stdbuf, you could not simply use cat for example. For the same reason you cannot either say stdbuf -o100M seq 12000000 > fifo. As far as I know, classical sed grep or awk do it.

The drawback of this is to mobilize one more process ; the advantage is that you are free to implement a huge buffer size.