I have this little C89 program -- what it does, how it works, and the undefined behaviour aren't important, just that it does some long memory & math operations on just one thread:
x, c, * b, m, t, i, a;
g (n) {
for (b = malloc(0); c < n; b[c - 1] = x++, t = 1) {
char s[9];
for (i = m = 0; i < sprintf(s, "%d", x); m += a, t *= a) a = s[i++] - 48;
b = m * t ? x % m + x % t ? b : realloc(b, 4 * ++c) : b;
}
return b[c - 1];
}
main (j) {
printf("%d\n", g(--j));
}
Compile it like this: gcc -std=c89 tt.c -o tt -O3
.
Then, if I use a shell script to run it in a loop, to get an idea of its running time:
#!/bin/bash
echo "using input $1"
for _ in `seq 1 10`; do
( time ./tt $(seq 1 $1) ) 3>&1 1>/dev/null 2>&3 \
| grep real \
| cut -f2
# sleep 5
done
I see output like this:
$ ./tt.sh 50
using input 50
0m0.016s
0m0.008s
0m0.008s
0m0.007s
0m0.007s
0m0.007s
0m0.008s
0m0.008s
0m0.007s
0m0.007s
Or like this:
$ ./tt.sh 34
using input 34
0m0.007s
0m0.004s
0m0.004s
0m0.004s
0m0.005s
0m0.004s
0m0.003s
0m0.003s
0m0.003s
0m0.004s
There is an initial speedup in the real
runtime of the program after the first call, and then all subsequent calls run at this fake speedup.
If I uncomment the # sleep 5
line in the shell script, we see these results:
using input 50
0m0.008s
0m0.020s
0m0.018s
0m0.012s
0m0.009s
0m0.006s
0m0.013s
0m0.012s
0m0.009s
0m0.012s
using input 34
0m0.006s
0m0.007s
0m0.004s
0m0.007s
0m0.008s
0m0.003s
0m0.004s
0m0.004s
0m0.005s
0m0.007s
The times appear more expected and accurate, and discrepancies therein must be attributed to the random state of the processor in that moment (i.e. they are small natural variations).
If I want to get an average running time for my program, I should average these numbers, but sleep 5
between each call, while it is the only way I can find to stop this behaviour, adds up to 50 seconds for 10 tests instead of a few seconds for 20 tests.
I've seen this "replay caching" behaviour before with single-threaded programs that do long operations over and over (tight loops), and I understand it's desirable in 99.9% of all cases.
Assuming this isn't a consequence of some Intel Magic™ at the hardware level, is this something the Linux kernel or Bash are doing intentionally, and how can I make it stop?
I'd like reproducible runtimes for my programs, including library loading and paging from "cold start", without sleep 5
ing, because the times affected by caching do not represent a cold start each time.
sleep 5
, what are theperf
stats like for the first run compared to the others? This feels like caching, you should sees lots more misses for the first run than for the rest... – Stephen Kitt Dec 1 '16 at 14:22perf
enough to process the output in a way pertinent to the question. – cat Dec 1 '16 at 17:32tt
. – Mark Wagner Dec 1 '16 at 20:43