Matthew Sottile and Ronald Minnich (2004)
Analysis of microbenchmarks for performance tuning of clusters
In: Proceedings of Cluster 2004.
Microbenchmarks, i.e. very small computational
kernels, have become commonly used for quantitative measures
of node performance in clusters. For example, a commonly used
benchmark measures the amount of time required to perform a
fixed quantum of work. Unfortunately, this benchmark is one of
many that violate well known rules from sampling theory, leading
to erroneous, contradictory or misleading results. At a minimum,
these types of benchmarks can not be used to identify time-based
activities that may interfere with and hence limit application
performance. Our original and primary goal remains to identify
noise in the system due to periodic activities that are not part of
user application code. In this paper, we discuss why the `fixed
quantum of work' benchmark provides data that is of limited use
for analysis; and we show code for, discuss, and analyze results
from a microbenchmark which follows good rules of sampling
hygiene, and hence provides useful data for analysis.