@Setup(Level.Invocation)

[Sergey Kuksenko]

Had you read Level.Invocation docs?
I'll juct copy it here:

/**
  * Invocation level: to be executed for each benchmark method execution.
  *
  * WARNING: HERE BE DRAGONS!
  * MAKE SURE YOU UNDERSTAND THE REASONING AND THE IMPLICATIONS
  * OF THE WARNINGS BELOW BEFORE EVEN CONSIDERING USING THIS LEVEL.
  *
  * It is almost never a good idea to use this in nano- and 
micro-benchmarks.
  *
  * <p>WARNING #1: In order to subtract the helper time from the
  * benchmark itself, we will have to have at least two timestamps
  * per *each* benchmark invocation. If the benchmarked method is
  * small, then we saturate the system with timestamp requests, which
  * *both* make timestamp requests the critical part of the benchmark
  * time, and inhibit workload scalability, introducing the artificial
  * scalability bottleneck.</p>
  *
  * <p>Also, the hiccups in measurement can be hidden from these individual
  * timing measurement, which can introduce inconsistent results. The 
largest
  * caveat is measuring oversaturated system where the descheduling events
  * will be missed, and the benchmark will perceive the fictionally large
  * throughput.</p>
  *
  * <p>WARNING #2: In order to maintain the basic interference behavior
  * of other Levels (e.g. the State(Scope.Benchmark) should only fire the
  * helper method once per invocation, regardless of the thread count),
  * we have to arbitrate the access to the state between worker thread,
  * and do that on *critical path*, thus further offsetting the 
measurement.</p>
  *
  * <p>WARNING #3: Current implementation in JMH allows the helper method
  * execution to overlap with the benchmark method itself in order to 
simplify
  * arbitrage. (To be redefined in future).</p>
  */
Invocation,

On 07/09/2014 06:05 PM, Ivan Gerasimov wrote:
> Hello!
>
> I noticed a great influence of presence of @Setup(Level.Invocation) on
> some benchmark results.
>
> This simple test gives the following numbers.
>
> @State(Scope.Thread)
> public class MyBenchmark {
>
> //    @Setup(Level.Invocation)
> //    public void setup() {}
>
>      @Benchmark
>      public int testMethod() {
>          return 42;
>      }
> }
>
> Benchmark                      Mode   Samples        Score  Score
> error    Units
> o.s.MyBenchmark.testMethod    thrpt         5 521204947.564
> 200284466.443    ops/s
>
>
> When I uncomment the setup() function, the numbers drops to:
>
> Benchmark                      Mode   Samples        Score  Score
> error    Units
> o.s.MyBenchmark.testMethod    thrpt         5   120471.248 32030.457 ops/s
>
>
> Am I correct that it's not practical to use @Setup(Level.Invocation)
> with measuring anything that lasts less than a microsecond or something?
>
> Sincerely yours,
> Ivan
>

-- 
Best regards,
Sergey Kuksenko