Long Running JMH Benchmarks not working on micro benchmark suite?

[Paul Sandoz]

> On Apr 17, 2023, at 9:31 AM, Martin Stypinski <mstypinski at gmail.com> wrote:
> 
> Thanks,
> 
> I'll fix the code.
> Just for my personal benchmarking, what is the easiest way to override the parameters to run long benchmarks?

You can override using the command line “-p" argument:

"
  -p <param={v,}*>
            Benchmark parameters. This option is expected to
                              be used once per parameter. Parameter name and parameter
                              values should be separated with equals sign. Parameter
                              values should be separated with commas.
"

In general see the documentation output when you run like this:

  java -jar target/benchmarks.jar -h

Paul.


> I would like to test a few things that I was working on last year and see if the effects are still the same. This enables me to use the latest build and make everything more reproducible down the road...
> 
> Martin
> 
> Am Mo., 17. Apr. 2023 um 18:26 Uhr schrieb Paul Sandoz <paul.sandoz at oracle.com>:
> Hi Martin,
> 
> I would recommend placing default sizes such that the benchmark executes within a reasonable time e.g. a few minutes or less overall. Then the user can override with larger sizes.
> 
> Further, I suggest you only use the preferred species for the vector benchmarks. That will ensure the optimal use, otherwise, for you will be running benchmarks for say AVX512 on AVX2 systems and the performance will nosedive and will take longer to run, which may be one cause for larger matrix sizes. And, since we are talking in general O(N^3) in complexity larger sizes are gonna have quite an impact.
> 
> In your code you can avoid the explicit use of a Blackhole by returning the result from the method (JMH will consume it for you).
> 
> Paul.
> 
> 
> > On Apr 15, 2023, at 1:19 AM, Martin Stypinski <mstypinski at gmail.com> wrote:
> > 
> > Hi,
> > 
> > I try to contribute my matrix multiplication benchmarks into the Vector API micro benchmark suite. I try to replicate some issues that I encountered with bigger matrix sizes.
> > 
> > Currently, my benchmarks run with array sizes up to 1024x1024 but I have no syntax error with array sizes of 32768x32768 but the benchmark just stops, halts, freezes? without results?
> > 
> > Is there a maximum time limit which is given by the framework?
> > I tried to increase time on Warmup and Measurements to 120s, without effect. I have a feeling it stalls around the 1s mark.
> > 
> > My current code can be found here:
> > https://gist.github.com/Styp/39be04debde25a716e824c61c3c349a2
>