Shenandoah Events

[Roman Kennke]

What kind of machine are you running it on? We have observed fairly wild
variance on laptops, for example, because of throttling/powersaving etc.

Roman

> Hi Aleksey,
> 
> I've ran the specJBB a number of times since this email and I'm unable to
> get any sort of consistency for either case. Any insights as to why that
> might be?
> 
> Thanks,
> 
> Ken Dobson
> 
> On Wed, Feb 6, 2019 at 3:35 PM Ken Dobson <kdobson at redhat.com> wrote:
> 
>> Hi all,
>>
>> Some updates regarding testing with the benchmarks.
>>
>> For specJVM after a number of tests I've noticed no significant
>> differences in performance between the tests that are recorded and the
>> tests that aren't. That being said the specJVM benchmark only emits ~5000
>> heap region transition events which seems to be 1-2 orders of magnitude
>> smaller than what I'd expect from a normal process so I don't think this
>> provides any quality information regarding the performance impact.
>>
>> With SpecJBB the jOPS numbers I've gotten were:
>>
>> WithRecording
>> Max =14096
>> Geomean=5899
>>
>> NoRecording
>> Max=12177
>> Geomean=5737
>>
>> Not sure why the results are the opposite of what would be expected so any
>> insight would be appreciated. I ran the test on this machine:
>> https://beaker.engineering.redhat.com/view/hp-dl785g6-01.rhts.eng.bos.redhat.com#details
>> with -Xmx=50G and -Xms=50G.
>>
>> I can zip up the whole results page if that would be helpful.
>>
>> Thanks,
>>
>> Ken Dobson
>>
>>
>>
>>
>> On Wed, Jan 30, 2019 at 12:03 PM Ken Dobson <kdobson at redhat.com> wrote:
>>
>>> Thank you this is great. I don't have the benchmarks no, drop them
>>> wherever is easiest for you.
>>>
>>> Thanks,
>>>
>>> Ken
>>>
>>> On Wed, Jan 30, 2019 at 11:28 AM <zgu at redhat.com> wrote:
>>>
>>>> On Wed, 2019-01-30 at 10:54 -0500, Ken Dobson wrote:
>>>>> Hi Zhengyu,
>>>>>
>>>>> We should still find out the impact when those events are being
>>>>> recorded to ensure it's not too significant. Would you be able to
>>>>> instruct me as to how to run the benchmarks so that I can measure the
>>>>> performance while the JVM is being recorded vs recording disabled?
>>>> Okay, we usually run specJVM and specJBB, do you have the benchmarks?
>>>> If not, where can I drop them?
>>>>
>>>> For specJVM, the commandline I use:
>>>> ${JAVA_HOME}/bin/java -jar jmh-specjvm2016.jar Derby  --jvmArgs "-Xmx1g
>>>> -Xms1g -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC ..." -f 3
>>>>
>>>> For specJBB, my script attached.
>>>>
>>>> Thanks,
>>>>
>>>> -Zhengyu
>>>>
>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Ken
>>>>>
>>>>> On Tue, Jan 29, 2019 at 1:05 PM <zgu at redhat.com> wrote:
>>>>>> On Tue, 2019-01-29 at 18:25 +0100, Aleksey Shipilev wrote:
>>>>>>> On 1/29/19 6:03 PM, Ken Dobson wrote:
>>>>>>>> Just following up on the possibility of running the benchmarks
>>>>>> to
>>>>>>>> measure the performance overhead.
>>>>>>>> Please let me know if this would be possible and what I would
>>>>>> have
>>>>>>>> to do to get this done.
>>>>>> I was initially worry about the amount of region state transition
>>>>>> events generated. After adding should_commit() guard, I am now less
>>>>>> concerned.
>>>>>>
>>>>>> Some overheads during recoding time, I think, are expected. So the
>>>>>> overhead, that we are talking about, is down to the additional
>>>>>> guard
>>>>>> test when recording is off, I doubt it is measurable.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> -Zhengyu
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> It is possible, and should be as simple as running the benchmarks
>>>>>>> with/without -XX:+FlightRecorder?
>>>>>>> You are working with Zhengyu on JFR support, right? Zhengyu knows
>>>>>> how
>>>>>>> to run benchmarks.
>>>>>>>
>>>>>>>> On Thu, Jan 24, 2019 at 11:28 AM Ken Dobson <kdobson at redhat.com
>>>>>>>> <mailto:kdobson at redhat.com>> wrote:
>>>>>>>>     The G1 version currently intercepts individual transitions
>>>>>> so
>>>>>>>> I'd hope they've measured the
>>>>>>>>     overhead and found it was acceptable but can't be certain
>>>>>> of
>>>>>>>> that. Yes I agree that's definitely
>>>>>>>>     the first step. Generally the default JFR profiling
>>>>>>>> configuration is ~2% overhead but detailed
>>>>>>>>     events such as these are not enabled in that configuration.
>>>>>>>> When using these events I think it
>>>>>>>>     would be best to disable all the default events and only
>>>>>> enable
>>>>>>>> the two Shenandoah Events to
>>>>>>>>     reduce the overhead. If you think measuring the benchmarks
>>>>>> is
>>>>>>>> the best way to get this data I'd
>>>>>>>>     be happy to do this if you can point me in the right
>>>>>> direction.
>>>>>>>
>>>>>>> The first rule of benchmarking is not assuming anything,
>>>>>> including
>>>>>>> that someone else did them,
>>>>>>> especially for a different implementation.
>>>>>>>
>>>>>>> There is also a bigger question: how much additional latency this
>>>>>>> brings to Shenandoah (tiny)
>>>>>>> pauses, when there are lots of transitions happen? Shenandoah
>>>>>> logs
>>>>>>> times with -Xlog:gc, and summary
>>>>>>> times with -Xlog:gc+stats.
>>>>>>>
>>>>>>> -Aleksey
>>>
>>>