JMH configuration and result interpretation

[Aleksey Shipilev]

On 04/10/2016 06:41 PM, J Kinable wrote:
> I recently encountered JMH and I wanted to give it a try. I've a few
> specific questions about how to configure JMH and how to interpret the
> result.
> 
> 1. Through the parameter 'measurementIterations' I can specify how often a
> benchmark is repeated. But what is the function of the parameter
> 'measurementTime'? Let's say I specify measurementTime=1 second and I
> create an @benchmark function which normally takes 10 minutes to complete.
> Is this function interrupted/killed after 1 second? The javadoc description
> of measurementTime  "Time of each measurement iteration" isn't very
> descriptive".
> 
> 2. Let's assume I run a benchmark with "AverageTime" mode. For each
> benchmark I get a score expressed in ms/op. How should I interpret this?
> What is "op" in this case? I would have expected: "average time to finish
> the benchmark completely".

Answers to both questions would probably be obvious, once you understand
that "op" is one @Benchmark call. JMH is not supposed to be run with
very fat @Benchmark-s. Instead, it will run the small @Benchmark
continuously within the warmup/measurement iteration time.

-Aleksey