0

A I/O efficiency test from APUE:

Test file is said to be "98.5 MB with 3 million lines."

enter image description here

Code used in "Figure 3.6":

#include "apue.h"

#define BUFFSIZE    4096

int
main(void)
{
    int     n;
    char    buf[BUFFSIZE];

    while ((n = read(STDIN_FILENO, buf, BUFFSIZE)) > 0)
        if (write(STDOUT_FILENO, buf, n) != n)
            err_sys("write error");

    if (n < 0)
        err_sys("read error");

    exit(0);
}

best time from Figure 3.6: Use different BUFFSIZE and choose the best time

single byte time from Figure 3.6: Use BUFFSIZE=1

err_sys is merely a wrapper function for error handling.

apue.h is a also wrapper header file.


Code uses fgets:

#include "apue.h"
int
main(void)
{
    int c;
    while ((c = fgetc(stdin)) != EOF){
        if (fputc(c, stdout) == EOF)
            err_sys("output error");
    }
        if (ferror(stdin))
            err_sys("input error");
    exit(0);
}

The question comes from:

The last point of interest with these timing numbers is that the fgetc version is so much faster than the BUFFSIZE=1 version from Figure 3.6. Both involve the same number of function calls—about 200 million—yet the fgetc version is more than 16 times faster in terms of user CPU time and almost 39 times faster in terms of clock time. The difference is that the version using read executes 200 million function calls, which in turn execute 200 million system calls. With the fgetc version, we still execute 200 million function calls, but this translates into only 25,224 system calls. System calls are usually much more expensive than ordinary function calls.

The test file is 98.5 MB ≈ 100 M, so 100 million calls for each I/O function call == 200 million calls, no problem here.

But how does 200 million fgets translate into 25,224 system call?

With the fgetc version, we still execute 200 million function calls, but this translates into only 25,224 system calls.

How is 25,224 calculated?

6
  • 3
    Presumably fgetc and fputc do internal buffering of ~4k, dividing the whole count by 8k (98.5*1024*1024/4096 = 25216, which is remarkably close to the given number).
    – Olorin
    Mar 5, 2019 at 6:01
  • 1
    If so, I think internal buffering should be ~8K. Read function call == write function call = 25224 / 2 = 12612. 98.5*1024*1024/12612 = 8189 B. ~8K data transmitted for each read or write operation. Am I right here?
    – Rick
    Mar 5, 2019 at 6:37
  • 1
    Ah, yes, I forgot to double the 98.5 before.
    – Olorin
    Mar 5, 2019 at 7:21
  • 1
    You can set the stdio buffer size to whatever you want with setvbuf() (eg. to 1G), and then calculate yet another value ;-)
    – user313992
    Mar 5, 2019 at 11:02
  • 1
    at the contrary, it's functions like fgetc or fputc (which may even be inlined) which need buffering the most, as the example from that book triea to demonstrate. In any case, they use full buffering by default (when not reading from some interactive device) in any implementation I may think of.
    – user313992
    Mar 5, 2019 at 16:41

0

You must log in to answer this question.

Browse other questions tagged .