18

I am timeing some of my commands. Now to average the measures, I'd like to run and time my commands a specified number of times, and get the results with a calculated mean and standard deviation. The result would be like:

avgtime -n 100 my_command

real    0m5.388s stdev 0m0.068s
user    0m5.380s stdev 0m0.067s
sys     0m0.004s stdev 0m0.000s

Is there a UNIX tool for this? Does GNU/Linux have one?

7

You can to try use the timeit module, available in any system with Python:

$ python -m timeit "__import__('os').system('my command here')"
10 loops, best of 3: 591 msec per loop
  • using os.system(), causes the overhead of calling/creating a shell with each command. Probably better use subprocess.call() – Anthon Apr 10 '14 at 15:33
  • true, but would be likely constant-ish anyway – bhdnx Apr 10 '14 at 19:05
4

It's not exactly a UNIX or GNU/Linux tool, but you could quite comfortably use the R software environment for statistical computing for this. (I cannot find anything more specific for your task, though.)

Edit How could I doubt it, there of course is a benchmark package for R: rbenchmark. It apparently wraps system.time() which you could also just use directly. Or have a look at this, a simple stopwatch function pair. Also see "Executing a system command" @Rosetta Code (or don't, it's system("command").)

Edit2 I just saw this question, "Measuring time within a script" in the right "Related" column, this could be used, too, i.e. take time, do for-loop (N times), take time again, calculate timespan, divide by N. (Even easier, try time ( for-loop ), parse its output, divide by N).

3

You can use R to quickly compute the mean, standard-deviation and other interesting values.

For example, you can use GNU time to write several runtime measurements into a CSV file:

function measure
{
  f=$1; shift
  n=$2; shift
  echo wall,user,sys,rss > "$f"
  for i in $(seq $n); do
    /usr/bin/time --output "$f" --append --format '%e,%U,%S,%M' "$@" >/dev/null
  done
}

Then you can generate the values with R like this:

Rscript --vanilla -e "b=read.csv(file='$f');summary(b);sapply(b, sd);"

I've created a small benchmark script that also does some pretty printing of the R output, e.g.:

$ benchmark.sh 100 ./some_prog arg1 arg2
n=100 | min    | Q1     | med    | mean   | Q3     | max    | std
wall  | 0.0400 | 0.0400 | 0.0500 | 0.0471 | 0.0500 | 0.0800 | 0.00624
user  | 0.0400 | 0.0400 | 0.0400 | 0.0426 | 0.0400 | 0.0700 | 0.00562
sys   | 0      | 0      | 0      | 0      | 0      | 0      | 0.00000
rss   | 2608   | 2657   | 2704   | 2728   | 2764   | 2920   | 95.06524
1

Option 1 - sqlite:

create simple table with command and time columns, and view with proper aggregation calculations. After timing, add a row to the table.

Advantages: simpler to define a table compared to solution 2.

Disadvantages: you need (do you?) care about data retention.

Option 2 - rrdtool:

Define rrd data base file, data definition and aggregation functions. After timing, feed the database with rrdtool update ....

Advantages: you can easily generate graphs with rrdtool graph .... No data retention issue (round robin database).

Disadvantages: bit harder to define rrd database compared to simple SQL table/view

1

Hyperfine is another option.

Sample usage:

hyperfine --warmup 3 'my_command'

See related question.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.