Multiple JVMs, different numbers of threads and BenchmarkMode

[Aleksey Shipilev]

On 08/04/2014 11:22 AM, Bernd wrote:
> Am 03.08.2014 18:30 schrieb "Aleksey Shipilev" <aleksey.shipilev at oracle.com
>> :
> 
>> AverageTime or Throughput, obviously. I'm not sure how much of a problem
>> that is, actually. We should not care how many times @Benchmark was
>> called in throughput modes, we should only care it was called *enough*
>> times.
> 
> Actually this is something, I was wondering about: when minimizing runtime
> of warmup and benchmark I typically do some test runs to get a feeling for
> the throughput and then calculate the number of repetitions for the chosen
> time in head. I wonder why the number of executions for each iteration is
> not printed.

The issue is more complicated than that: one could be happy with the low
number of times @Benchmark is being called, because, e.g. it calls
thousands of leaf methods from there, and this brings the benchmark over
the warmup edge. The repetition count is meaningless here, and JMH has
no idea how to apply this generically.

This reiterates the lesson you need to do longer warmup for the unknown
workloads, and only then figure out how many warmup iterations are
enough to get to the steady state. Pretty much the thing you are doing
already.

> Maybe if you dont want to clutter the output we can only print it if the
> number is rather low?

That will clutter the output *and* confuse users. Just warmup longer
until you think you reached the steady state.

Thanks,
-Aleksey.