Invert command line overriding

[Staffan Friberg]

Hi,

I have been think a bit further about this and similar use cases where 
when you specify JVM arguments as part of your benchmark.

At least for JVM arguments there is the chance that the microbenchmark 
require specific command line options to actually execute correctly, be 
it disabling an optimization or a certain heap size.

Would it make sense to add a "force" option to annotations which tells 
the JVM to use the pre-configured options over the command line options?
If that is the case we probably should add a "-overrideForce" flag to 
the JMH command line interface so you do no have to recompile if you 
want to do some quick testing with a benchmark, but it would remove the 
risk by error overriding required options.

Another example would be a benchmark you know require longer warmup than 
others and you could force it while configuring all other via command 
line without having to resort to annotating all microbenchmarks with 
their warmup should they differ for the JMH default.

The JVM args could partly be solved by using the append args options, 
since that will override earlier command line options, but this might 
help solve it in a more generic way for all annotations.

Cheers,
Staffan

On 04/09/2014 07:38 AM, Doug Simon wrote:
> On Apr 9, 2014, at 9:02 AM, Aleksey Shipilev <aleksey.shipilev at oracle.com> wrote:
>
>> On 04/08/2014 06:14 PM, Doug Simon wrote:
>>> For some individual benchmarks in a suite of benchmarks, I need
>>> extra warmup iterations. I can specify this with an explicit @Warmup
>>> annotation. However, it appears as though the -wi command line
>>> option overrides such annotations. Is it possible to somehow make -wi
>>> only apply to benchmarks for which there is no explicit @Warmup
>>> annotation?
>> No, the options overriding follow the simple principle: anything you can
>> supply later should override anything you supplied before. In this case,
>> command line options override the annotations. In this sense, command
>> line options provide the final authority on the benchmark execution mode.
> Ok, thanks for the explanation.
>
>> I could of course achieve this via separate jmh executions but that
>>> will non-trivial changes to our benchmarking infrastructure.
>> Or, you may use Java API to drive your benchmarks if you need something
>> not in line with the default JMH behavior.
> Good suggestion - I’ll look into it.
>
> -Doug