Using JMH to test virtual thread performance

[Firouz Bharthania]

Alan and Aleksey,

Giving it a second thought, I think you are both correct. Due to heavy synchronization and scheduling among many number of threads we should expect a high deviation from norm when measuring “Average Time”.

Thanks,

Firouz Bharthania


> On Feb 8, 2022, at 10:17 AM, Aleksey Shipilev <shade at redhat.com> wrote:
> 
> On 2/8/22 15:02, Alan Bateman wrote:
>> On 08/02/2022 00:22, Firouz Bharthania wrote:
>>> Can JMH (Java Microbenchmarking Harness) be used for measuring virtual threads at this time, because I get high, unacceptable error margins when I run tests?
> 
> The "high, unacceptable error margins" might just be indicative that benchmark is very noisy. In that case, no benchmark harness could possibly help.
> 
>> Aleksey may have some advice but I would expect it should be possible to
>> create a runner that invokes org.openjdk.jmh.Main.main in the context of
>> a virtual thread. I don't know if anyone has done that.
> 
> It is probably not a great idea to execute @Benchmark methods on virtual threads, because they are using quite heavy synchronization, which AFAICS would just block the carrier thread in current implementation? Anyhow, JMH allows overriding the ExecutorService that runs the benchmark threads, not sure if it fits current Loom APIs: https://github.com/openjdk/jmh/blob/85dfd3bf334159b0bc1acf1059cb6c505d0c95e4/jmh-core/src/main/java/org/openjdk/jmh/runner/BenchmarkHandler.java#L227-L234
> 
> -- 
> Thanks,
> -Aleksey
>