Execution of fixture methods in the case of error during benchmark.

[Aleksey Shipilev]

On 6/4/21 1:41 PM, Mehdi Ennaïme wrote:
>  > ...so the subsequent iteration within the trial would not run. The next trial should run (presumably
>  > in newly forked VM), and @Setup(TRIAL) should run the.
> 
> I think this was the issue, since I was debugging other parts of the code in my IDE, the number of 
> forks was set to 0 :

Yeah, so this is a corner case in error handling in non-forked mode.

The @State initialization is handled by storing the initialized instance into static field. Normal 
path nulls that field on normal benchmark exit, so next @Param in sequence is able to reinitialize 
it with new value. But when benchmark exits with the error, it bypasses the normal exit, @State 
stays initialized, and @Setup never runs again. In forked mode, we are insulated by the fact we 
would run next @Param in the a fresh VM.

I am a bit struggling to come up with an easy fix for it, so in the mean time, do not run with zero 
forks ;) Follow the advice that Runner prints:

  *** WARNING: Non-forked runs may silently omit JVM options, mess up profilers, disable compiler 
hints, etc. ***
  *** WARNING: Use non-forked runs only for debugging purposes, not for actual performance runs. ***"

-- 
Thanks,
-Aleksey