RFR 8221623 : Add StackWalker micro benchmarks to jdk repo

[Claes Redestad]

Hi,

FYI, you can control the number of forks etc run by make test:

make test TEST="micro:java.lang.StackWalkBench" 
MICRO="FORK=1;WARMUP_ITER=2;OPTIONS=-p depth=128"

(documented in doc/testing.md|html)

Still, I agree it might be a good idea to have make test TEST=... run
fewer forks/iterations by default, since this shortcut was primarily
intended as a means for devs to quickly verify specific microbenchmarks.

However, reducing forks and iterations only cuts down one factor of the
total runtime: others being the number of methods and the number of
chosen parameters (as controlled by @Param-annotated fields).

So IMHO, what should be thought through carefully when adding micros is
whether all methods are necessary or at least useful and whether we can
reduce the number of parameter chosen.

For these two, I think picking 2 or 3 values for depth (lowest, highest,
maybe a midpoint) might be sufficient. This would cut total runtime
budget by 60% or 40%, respectively.

Thanks!

/Claes

On 2019-09-19 20:09, Brent Christian wrote:
> Hi, Mandy
> 
> Yes, that 'make' job would take ~7 hours on my machine.
> I believe this is typical for running micros using 'make'.  AFAICT, the 
> jmh defaults are used, so it forks (running all the warmups + 
> iterations) 5 times.  Trying another (>100 loc) benchmark  at random, 
> the java.lang.ArrayCopy one would take 25 hours.
> 
> There might be a way to pass jmh args (e.g. '-f 1') via 'make test 
> TEST=...', though I really only ever run benchmarks using 'java -jar 
> path/to/benchmark.jar'.
> 
> It may be worth looking into updating the make target to setup more 
> practical JMH runtimes by default.  But I don't think this an issue 
> particular the new benchmarks.
> 
> Thanks,
> -Brent
> On 9/18/19 7:33 PM, Mandy Chung wrote:
>> Hi Brent,
>>
>> $ make test TEST="micro:java.lang.StackWalkBench"
>>
>> It took very long that I killed the job.  Does this happen to you?
>>
>> Mandy
>>
>> On 9/13/19 3:07 PM, Brent Christian wrote:
>>> Hi,
>>>
>>> Please review these StackWalker and Throwable benchmarks for addition 
>>> into the JDK microbenchmarks.
>>>
>>> Bug:
>>> https://bugs.openjdk.java.net/browse/JDK-8221623
>>> Webrev:
>>> http://cr.openjdk.java.net/~bchristi/8221623/webrev07/
>>>
>>> The StackWalker benchmarks use StackWalker's forEach(), walk(), and 
>>> Options to measure retrieval of various types of information from the 
>>> call stack.
>>>
>>> The Throwable benchmarks do corresponding exercises; there are also a 
>>> couple of Logging benchmarks.
>>>
>>> A JMH @Param is used to test a variety of call stack depths.
>>>
>>>
>>> In the future, we might consider a benchmark for 
>>> Reflection.getCallerClass().  (It is more involved today to benchmark 
>>> that method than at the time these benchmarks were originally 
>>> written, so that one's commented out.) See JDK-8230976.
>>>
>>> Thanks,
>>> -Brent
>>>
>>> 1. https://bugs.openjdk.java.net/browse/JDK-8230976
>>
>