Anonymous Pipe Kernel Buffer Size

Question

I am trying to understand the structures associated with a call to the pipe syscall.

int pipe(int pipefd[2]);

From my understanding, there will be a "read" and "write" kernel buffer/structure associated with the read end pipefd[0] and write end pipefd[1] of the pipe.

I would really like some pointers (no pun intended) on structures referring to the read/write ends of this pipe, which I believe are just file descriptors. For instance, how does the computer know that there is no more bytes to be transferred "read" during a read operation such as:

char tmpBuff[15];
bytesRead = read(filedes[0], tmpBuff, 15);

What structure is defined that keeps track of the buffer, file position, etc.. associated with the filedes[0] or filedes[1] descriptors?

According to the man page for ssize_t read(int fd, void *buf, size_t count);

READ(2) Linux Programmer's Manual

NAME read - read from a file descriptor

SYNOPSIS #include

   ssize_t read(int fd, void *buf, size_t count);

RETURN VALUE On success, the number of bytes read is returned (zero indicates end of file), and the file position is advanced by this number.

There must be some kind of structure pointed to be the index referred to by a file descriptor to make up some kind of structure like below.

struct file_info
{
    char *start_buf;
    char *end_buf;
    int fileposition;    
};

Lastly, is it possible from userspace to access these values out of curiousity and make a program that would say something like, "about to read from file descriptor foo, there is currently 120 bytes in the buffer and the file position is at 0". Since the buffer isn't accessible at the user level (I think), how could we access this information, only knowing the file descriptor value?

Answer 1

First, some of your question may be answered here: How big is the pipe buffer?

A pipe is a bounded circular buffer that is maintained by the kernel. The pipe system call creates a new pipe and associated two file descriptors with the pipe, one for writing to the pipe, and the other for reading from the pipe.

Writes to the pipe, via the associated file descriptor, copy the given data to that buffer if there's space for it. If there's not space for the data, the call to write() blocks the calling application until space becomes available. If a process is blocked in a write() operation on a pipe, and some other process/thread reads enough data from the pipe that would allow the blocked writer to complete its write, then the blocked writer is awoken and allowed to proceed.

The process works similarly for readers. If a reading process tries to read from a pipe and there's nothing there to read, the read() system call will block waiting for data to become available. If an process is blocked in a read() operation on a pope, and some other process/thread writes data to the pipe that would allow the blocked reader to complete its read, then the blocked reader is awoken and allowed to proceed.

In terms of the size of a pipe, you can find some information in the following man page entry from man 7 pipe:

Pipe capacity

   A pipe has a limited capacity.  If the pipe is full, then a write(2)
   will block or fail, depending on whether the O_NONBLOCK flag is set
   (see below).  Different implementations have different limits for the
   pipe capacity.  Applications should not rely on a particular
   capacity: an application should be designed so that a reading process
   consumes data as soon as it is available, so that a writing process
   does not remain blocked.

   In Linux versions before 2.6.11, the capacity of a pipe was the same
   as the system page size (e.g., 4096 bytes on i386).  Since Linux
   2.6.11, the pipe capacity is 65536 bytes.  Since Linux 2.6.35, the
   default pipe capacity is 65536 bytes, but the capacity can be queried
   and set using the fcntl(2) F_GETPIPE_SZ and F_SETPIPE_SZ operations.
   See fcntl(2) for more information.

Here's a example program to query the pipe's size:

#define _GNU_SOURCE
#include <assert.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main(void)
{
    int pipefds[2] = { -1, -1 };

    assert(pipe(pipefds) == 0);

    printf("pipe size: %d\n", fcntl(pipefds[0], F_GETPIPE_SZ));

    // pipe will be closed on exit
    return EXIT_SUCCESS;
}

Running the program gives me:

$ ./a.out
pipe size: 65536