Creating benchmarks "dynamically"

[Kannan Goundan]

Unfortunately, I don't think I can pick my set of arguments to @Param at
runtime (since they need to be seen by the annotation processing tool).

Another alternative would be to dynamically generate some Java code, run
'javac' on it (with annotation processing), load the class, then run JMH.
Seems a little clunky, though.

Are there lower-level JMH APIs I can use to do things more directly?

On Wed, Aug 24, 2016 at 12:57 PM, Henri Tremblay <henri.tremblay at gmail.com>
wrote:

> My quick thought is to use Params
> h
> ttp://hg.openjdk.java.net/code-tools/jmh/file/
> 2e5a7761ce12/jmh-samples/src/main/java/org/openjdk/jmh/
> samples/JMHSample_27_Params.java
>
> Something like (I haven't compiled it and I'm not sure you can setup a
> param but you should figure it out):
>
> @Param({"java.util.HashMap"})
> String implementation
>
> Map map;
>
> @Setup
> public void setup()  throws Exception {
>    map = (Map) Class.forName(implementation).newInstance();
> }
>
> @Benchmark
> public void test() {
>   // actual benchmark
> }
>
> On 24 August 2016 at 14:52, Kannan Goundan <kannan at cakoose.com> wrote:
>
>> I'm setting up a framework for students to understand different "Map"
>> implementations.
>>
>> The way it works:
>> - The student implements the Map interface in different ways (binary tree,
>> hashtable, etc)
>> - The "Impls" class has a public static mapping from implementation names
>> to implementation factory functions.
>> - There's a "Test" class with a "main" function.  You pass in a list
>> implementation names on the command line and runs correctness testing on
>> them.
>> - There's a "Bench" class with a "main" function.  You pass a list of
>> implementation names on the command line, along with the path to a file.
>> It runs benchmarks of all the implementations using the keys from that
>> file.  This allows the student to quickly try out different sets of keys
>> to
>> see how their implementation performs.
>>
>> I'd like to use JMH to implement "Bench", but I can't figure out the right
>> way to do that.  The list of implementations is dynamic (I just have a
>> list
>> of factory functions) and the list of keys is also dynamic (read in from a
>> file).
>>
>> Any ideas?
>>
>
>