Shenandoah Events

[Ken Dobson]

Hi Aleksey,

I've ran the specJBB a number of times since this email and I'm unable to
get any sort of consistency for either case. Any insights as to why that
might be?

Thanks,

Ken Dobson

On Wed, Feb 6, 2019 at 3:35 PM Ken Dobson <kdobson at redhat.com> wrote:

> Hi all,
>
> Some updates regarding testing with the benchmarks.
>
> For specJVM after a number of tests I've noticed no significant
> differences in performance between the tests that are recorded and the
> tests that aren't. That being said the specJVM benchmark only emits ~5000
> heap region transition events which seems to be 1-2 orders of magnitude
> smaller than what I'd expect from a normal process so I don't think this
> provides any quality information regarding the performance impact.
>
> With SpecJBB the jOPS numbers I've gotten were:
>
> WithRecording
> Max =14096
> Geomean=5899
>
> NoRecording
> Max=12177
> Geomean=5737
>
> Not sure why the results are the opposite of what would be expected so any
> insight would be appreciated. I ran the test on this machine:
> https://beaker.engineering.redhat.com/view/hp-dl785g6-01.rhts.eng.bos.redhat.com#details
> with -Xmx=50G and -Xms=50G.
>
> I can zip up the whole results page if that would be helpful.
>
> Thanks,
>
> Ken Dobson
>
>
>
>
> On Wed, Jan 30, 2019 at 12:03 PM Ken Dobson <kdobson at redhat.com> wrote:
>
>> Thank you this is great. I don't have the benchmarks no, drop them
>> wherever is easiest for you.
>>
>> Thanks,
>>
>> Ken
>>
>> On Wed, Jan 30, 2019 at 11:28 AM <zgu at redhat.com> wrote:
>>
>>> On Wed, 2019-01-30 at 10:54 -0500, Ken Dobson wrote:
>>> > Hi Zhengyu,
>>> >
>>> > We should still find out the impact when those events are being
>>> > recorded to ensure it's not too significant. Would you be able to
>>> > instruct me as to how to run the benchmarks so that I can measure the
>>> > performance while the JVM is being recorded vs recording disabled?
>>> Okay, we usually run specJVM and specJBB, do you have the benchmarks?
>>> If not, where can I drop them?
>>>
>>> For specJVM, the commandline I use:
>>> ${JAVA_HOME}/bin/java -jar jmh-specjvm2016.jar Derby  --jvmArgs "-Xmx1g
>>> -Xms1g -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC ..." -f 3
>>>
>>> For specJBB, my script attached.
>>>
>>> Thanks,
>>>
>>> -Zhengyu
>>>
>>>
>>> >
>>> > Thanks,
>>> >
>>> > Ken
>>> >
>>> > On Tue, Jan 29, 2019 at 1:05 PM <zgu at redhat.com> wrote:
>>> > > On Tue, 2019-01-29 at 18:25 +0100, Aleksey Shipilev wrote:
>>> > > > On 1/29/19 6:03 PM, Ken Dobson wrote:
>>> > > > > Just following up on the possibility of running the benchmarks
>>> > > to
>>> > > > > measure the performance overhead.
>>> > > > > Please let me know if this would be possible and what I would
>>> > > have
>>> > > > > to do to get this done.
>>> > > I was initially worry about the amount of region state transition
>>> > > events generated. After adding should_commit() guard, I am now less
>>> > > concerned.
>>> > >
>>> > > Some overheads during recoding time, I think, are expected. So the
>>> > > overhead, that we are talking about, is down to the additional
>>> > > guard
>>> > > test when recording is off, I doubt it is measurable.
>>> > >
>>> > > Thanks,
>>> > >
>>> > > -Zhengyu
>>> > >
>>> > >
>>> > > >
>>> > > > It is possible, and should be as simple as running the benchmarks
>>> > > > with/without -XX:+FlightRecorder?
>>> > > > You are working with Zhengyu on JFR support, right? Zhengyu knows
>>> > > how
>>> > > > to run benchmarks.
>>> > > >
>>> > > > > On Thu, Jan 24, 2019 at 11:28 AM Ken Dobson <kdobson at redhat.com
>>> > > > > <mailto:kdobson at redhat.com>> wrote:
>>> > > > >     The G1 version currently intercepts individual transitions
>>> > > so
>>> > > > > I'd hope they've measured the
>>> > > > >     overhead and found it was acceptable but can't be certain
>>> > > of
>>> > > > > that. Yes I agree that's definitely
>>> > > > >     the first step. Generally the default JFR profiling
>>> > > > > configuration is ~2% overhead but detailed
>>> > > > >     events such as these are not enabled in that configuration.
>>> > > > > When using these events I think it
>>> > > > >     would be best to disable all the default events and only
>>> > > enable
>>> > > > > the two Shenandoah Events to
>>> > > > >     reduce the overhead. If you think measuring the benchmarks
>>> > > is
>>> > > > > the best way to get this data I'd
>>> > > > >     be happy to do this if you can point me in the right
>>> > > direction.
>>> > > >
>>> > > > The first rule of benchmarking is not assuming anything,
>>> > > including
>>> > > > that someone else did them,
>>> > > > especially for a different implementation.
>>> > > >
>>> > > > There is also a bigger question: how much additional latency this
>>> > > > brings to Shenandoah (tiny)
>>> > > > pauses, when there are lots of transitions happen? Shenandoah
>>> > > logs
>>> > > > times with -Xlog:gc, and summary
>>> > > > times with -Xlog:gc+stats.
>>> > > >
>>> > > > -Aleksey
>>
>>