RFR 8221623 : Add StackWalker micro benchmarks to jdk repo

[Mandy Chung]

or make forks/iterations as required parameters when invoking the 
microbenchmark through make.

I'm doing the micro test run just for the review and I want it to finish 
in short time.  Performance runs will be a different configuration.

Mandy

On 9/19/19 1:40 PM, Claes Redestad wrote:
> Hi,
>
> FYI, you can control the number of forks etc run by make test:
>
> make test TEST="micro:java.lang.StackWalkBench" 
> MICRO="FORK=1;WARMUP_ITER=2;OPTIONS=-p depth=128"
>
> (documented in doc/testing.md|html)
>
> Still, I agree it might be a good idea to have make test TEST=... run
> fewer forks/iterations by default, since this shortcut was primarily
> intended as a means for devs to quickly verify specific microbenchmarks.
>
> However, reducing forks and iterations only cuts down one factor of the
> total runtime: others being the number of methods and the number of
> chosen parameters (as controlled by @Param-annotated fields).
>
> So IMHO, what should be thought through carefully when adding micros is
> whether all methods are necessary or at least useful and whether we can
> reduce the number of parameter chosen.
>
> For these two, I think picking 2 or 3 values for depth (lowest, highest,
> maybe a midpoint) might be sufficient. This would cut total runtime
> budget by 60% or 40%, respectively.
>
> Thanks!
>
> /Claes
>
> On 2019-09-19 20:09, Brent Christian wrote:
>> Hi, Mandy
>>
>> Yes, that 'make' job would take ~7 hours on my machine.
>> I believe this is typical for running micros using 'make'. AFAICT, 
>> the jmh defaults are used, so it forks (running all the warmups + 
>> iterations) 5 times.  Trying another (>100 loc) benchmark  at random, 
>> the java.lang.ArrayCopy one would take 25 hours.
>>
>> There might be a way to pass jmh args (e.g. '-f 1') via 'make test 
>> TEST=...', though I really only ever run benchmarks using 'java -jar 
>> path/to/benchmark.jar'.
>>
>> It may be worth looking into updating the make target to setup more 
>> practical JMH runtimes by default.  But I don't think this an issue 
>> particular the new benchmarks.
>>
>> Thanks,
>> -Brent
>> On 9/18/19 7:33 PM, Mandy Chung wrote:
>>> Hi Brent,
>>>
>>> $ make test TEST="micro:java.lang.StackWalkBench"
>>>
>>> It took very long that I killed the job.  Does this happen to you?
>>>
>>> Mandy
>>>
>>> On 9/13/19 3:07 PM, Brent Christian wrote:
>>>> Hi,
>>>>
>>>> Please review these StackWalker and Throwable benchmarks for 
>>>> addition into the JDK microbenchmarks.
>>>>
>>>> Bug:
>>>> https://bugs.openjdk.java.net/browse/JDK-8221623
>>>> Webrev:
>>>> http://cr.openjdk.java.net/~bchristi/8221623/webrev07/
>>>>
>>>> The StackWalker benchmarks use StackWalker's forEach(), walk(), and 
>>>> Options to measure retrieval of various types of information from 
>>>> the call stack.
>>>>
>>>> The Throwable benchmarks do corresponding exercises; there are also 
>>>> a couple of Logging benchmarks.
>>>>
>>>> A JMH @Param is used to test a variety of call stack depths.
>>>>
>>>>
>>>> In the future, we might consider a benchmark for 
>>>> Reflection.getCallerClass().  (It is more involved today to 
>>>> benchmark that method than at the time these benchmarks were 
>>>> originally written, so that one's commented out.) See JDK-8230976.
>>>>
>>>> Thanks,
>>>> -Brent
>>>>
>>>> 1. https://bugs.openjdk.java.net/browse/JDK-8230976
>>>
>>