Fixture&State settings and impact on measurement

[Aleksey Shipilev]

Hi,

On 05/17/2013 09:56 PM, Yann Le Tallec wrote:
> @State(Scope.Thread) / @Setup(Level.Iteration) => 0.6 ns
> @State(Scope.Benchmark) / @Setup(Level.Iteration) => 0.6 ns
> @State(Scope.Thread) / @Setup(Level.Invocation) => 11 ns
> @State(Scope.Benchmark) / @Setup(Level.Invocation) => 19 ns
> 
> Is this the expected behaviour?

Yes. In order to substract the @Setup cost from the measurement, we have
to do timestamps before and after the Level.Invocation code. See the
generated Java file in target/generated/ to see the exact code.
Level.Invocation is not supposed to be used in very fine benchmarks, but
it is useful in heavy-weight ones.

> Should one simply adjust actual measurements by subtracting those
> numbers(i.e. is this a fixed and stable overhead)?

If that's what you are after, it is probably the saner idea to measure
all the code within the @GMB method, and then measure the infra overhead
in another @GMB method, then subtract; that would be better without the
Level.Invocation.

-Aleksey.