Benchmark types revisited

[Sergey Kuksenko]

Look good for me.

On 06/04/2013 03:33 PM, Aleksey Shipilev wrote:
> Hi guys,
>
> We are trying to fold the remaining clutter in the API before moving on
> to parameters. One of the culprits today is the @GMB accepting the
> BenchmarkType, which is used by generator to build the synthetic code
> for the particular mode.
>
> This has a number of drawbacks:
>    a) inability to select the BenchmarkType at runtime
>    b) changing BenchmarkType requires recompilation
>    c) you can't request the class-level BenchmarkType, which will apply
> to all @GenerateMicroBenchmark
>    d) you can only distinguish different benchmark types with the prefix
> in the name
>
> So, instead of this:
>
>   class C {
>      @GenerateMicroBenchmark(BenchmarkType.AverageTimePerOp)
>      public void m1() {}
>
>      @GenerateMicroBenchmark(BenchmarkType.AverageTimePerOp)
>      public void m2() {}
>   }
>
> ...we propose to move to plain @GMBs, letting JMH to generate the code
> for *all* the modes at once, and providing us with the way to select the
> mode at runtime:
>
>   class C {
>      @GenerateMicroBenchmark
>      public void m1() {}
>
>      @GenerateMicroBenchmark
>      public void m2() {}
>   }
>
> ...also, with the optional annotation to select the default mode:
>
>   @BenchmarkMode(Mode.AverageTime)
>   class C {
>      @GenerateMicroBenchmark
>      public void m1() {}
>
>      @GenerateMicroBenchmark
>      public void m2() {}
>   }
>
> ...which also accepts multiple values:
>
>   @BenchmarkMode({Mode.AverageTime, Mode.Throughput})
>   class C {
>      @GenerateMicroBenchmark
>      public void m1() {}
>
>      @GenerateMicroBenchmark
>      public void m2() {}
>   }
>
> Running this benchmark would yield four runs: {m1, m2} x {AverageTime,
> Throughput}. New command line option "-bm" can override the default
> benchmark mode, e.g. running with "-bm AverageTime,SampleTime" will
> yield {m1, m2} x {AverageTime, SampleTime} result.
>
> The changeset with the proposed change is here:
>    http://cr.openjdk.java.net/~shade/jmh/benchtypes-1.changeset
>
> Please take a look, try to apply (hg import) and run with it.
> Comments, feedbacks, questions are welcome.
>
> -Aleksey.
>


-- 
Best regards,
Sergey Kuksenko