JMH iteration count/time affects benchmark results in an odd way

Dávid Karnok akarnokd at gmail.com
Sat Aug 6 15:46:56 UTC 2016


Hi. I'm running a benchmark (
https://github.com/reactor/reactive-streams-commons/blob/master/src/jmh/java/rsc/scheduler/SingleSchedulerPerf.java)
and getting odd results depending on the value of the measure annotation.
My specs: i7 4770K, Windows 7 x64, Java 8u102, tried with both v1.11.3 and
v1.13

If I run the benchmark with this (rest the same):

   @Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)

   I get around 8000 +/- 2000 ops/second for both the warmup and the
measurement (in each iteration).


If I run the benchmark with this:

   @Measurement(iterations = 5, time = 2, timeUnit = TimeUnit.SECONDS)

   I get 14000 +/- 1000 ops/second for both the warmup and measurement,
even though the warmup still uses time = 1


If I run the benchmark with this:

   @Measurement(iterations = 10, time = 1, timeUnit = TimeUnit.SECONDS)

   I get 14000 +/- 1000 ops/second for both the warmup and measurement
again.


I've run the bench with each settings several times but still getting the
same pattern. Given that every other parameter and algorithm is the same,
how can the measurement count or time affect ops/s upfront?

-- 
Best regards,
David Karnok


More information about the jmh-dev mailing list