Ability to add "jvm arguments" based on "@Param"

[Aleksey Shipilev]

On 28.05.2015 11:19, Vladimir Sitnikov wrote:
> Having thought a bit, I am inclined to
> OptionsBuilder.searchSpaceExplorer(Function<OptionsBuilder,
> List<OptionsBuilder>> explorer)
> 
> In other words, explorer would be able to return multiple sub-options
> (empty list meaning "stop here", and a list of length 3 meaning "try
> those 3 combinations").
> 
> It is not clear yet how that should interact with current parameter
> space examination though.

I was thinking about that in my free time. I am inclined to think that
annotations are inherently limited in expressibility when it comes to
"sane" patterns. Trying to fit the complex parameter schemes into
annotations, while being "friendly" to users, turns into a nightmare for
the harness. See e.g.:
 https://bugs.openjdk.java.net/browse/CODETOOLS-7901012

Therefore, IMO, we need to continue with "If you want something else,
use the APIs" mantra. What you are suggesting seems to fit there: what
you are building is the Supplier<Options> for Runner. The only open
question is whether it should be completely external, and spawn a Runner
for each Options, or we should internalize it within a single
Runner.run(Supplier<Options>) call.

I would guess that early prototyping should follow the "external" route.
That way, you will focus on the things that matter API-wise in the
search space walker, not the weird interop issues. Internalizing the
space walk would require, among other things, rewiring the runners to
disambiguate the benchmarks for different *options*, not only the
different *@Params*. That would extend to the runners themselves,
results and resultformats, etc.

Anyhow, to be fair, when I was dealing with the same problem of running
with different JVM options, it took me 5 minutes to construct a for loop
calling into JMH's Runners, and aggregate the results back. So we are
talking about a serious infra rewrite for a minute convenience, right?

Thanks,
-Aleksey