A I/O efficiency test from APUE:
Test file is said to be "98.5 MB with 3 million lines."
Code used in "Figure 3.6":
#include "apue.h"
#define BUFFSIZE 4096
int
main(void)
{
int n;
char buf[BUFFSIZE];
while ((n = read(STDIN_FILENO, buf, BUFFSIZE)) > 0)
if (write(STDOUT_FILENO, buf, n) != n)
err_sys("write error");
if (n < 0)
err_sys("read error");
exit(0);
}
best time from Figure 3.6: Use different BUFFSIZE
and choose the best time
single byte time from Figure 3.6: Use BUFFSIZE=1
err_sys
is merely a wrapper function for error handling.
apue.h
is a also wrapper header file.
Code uses fgets
:
#include "apue.h"
int
main(void)
{
int c;
while ((c = fgetc(stdin)) != EOF){
if (fputc(c, stdout) == EOF)
err_sys("output error");
}
if (ferror(stdin))
err_sys("input error");
exit(0);
}
The question comes from:
The last point of interest with these timing numbers is that the
fgetc
version is so much faster than theBUFFSIZE=1
version from Figure 3.6. Both involve the same number of function calls—about 200 million—yet thefgetc
version is more than 16 times faster in terms of user CPU time and almost 39 times faster in terms of clock time. The difference is that the version using read executes 200 million function calls, which in turn execute 200 million system calls. With thefgetc
version, we still execute 200 million function calls, but this translates into only 25,224 system calls. System calls are usually much more expensive than ordinary function calls.
The test file is 98.5 MB ≈ 100 M, so 100 million calls for each I/O function call == 200 million calls, no problem here.
But how does 200 million fgets
translate into 25,224 system call?
With the
fgetc
version, we still execute 200 million function calls, but this translates into only 25,224 system calls.
How is 25,224 calculated?
fgetc
andfputc
do internal buffering of ~4k, dividing the whole count by 8k (98.5*1024*1024/4096 = 25216, which is remarkably close to the given number).setvbuf()
(eg. to 1G), and then calculate yet another value ;-)