Long Running JMH Benchmarks not working on micro benchmark suite?

[Paul Sandoz]

Hi Martin,

I would recommend placing default sizes such that the benchmark executes within a reasonable time e.g. a few minutes or less overall. Then the user can override with larger sizes.

Further, I suggest you only use the preferred species for the vector benchmarks. That will ensure the optimal use, otherwise, for you will be running benchmarks for say AVX512 on AVX2 systems and the performance will nosedive and will take longer to run, which may be one cause for larger matrix sizes. And, since we are talking in general O(N^3) in complexity larger sizes are gonna have quite an impact.

In your code you can avoid the explicit use of a Blackhole by returning the result from the method (JMH will consume it for you).

Paul.


> On Apr 15, 2023, at 1:19 AM, Martin Stypinski <mstypinski at gmail.com> wrote:
> 
> Hi,
> 
> I try to contribute my matrix multiplication benchmarks into the Vector API micro benchmark suite. I try to replicate some issues that I encountered with bigger matrix sizes.
> 
> Currently, my benchmarks run with array sizes up to 1024x1024 but I have no syntax error with array sizes of 32768x32768 but the benchmark just stops, halts, freezes? without results?
> 
> Is there a maximum time limit which is given by the framework?
> I tried to increase time on Warmup and Measurements to 120s, without effect. I have a feeling it stalls around the 1s mark.
> 
> My current code can be found here:
> https://gist.github.com/Styp/39be04debde25a716e824c61c3c349a2