Shenandoah Events

[Ken Dobson]

Hi Roman,

In the email above I linked the machine I reserved off of beaker to run
them on.

Ken Dobson

On Wed, Feb 20, 2019 at 10:56 AM Roman Kennke <rkennke at redhat.com> wrote:

> What kind of machine are you running it on? We have observed fairly wild
> variance on laptops, for example, because of throttling/powersaving etc.
>
> Roman
>
> > Hi Aleksey,
> >
> > I've ran the specJBB a number of times since this email and I'm unable to
> > get any sort of consistency for either case. Any insights as to why that
> > might be?
> >
> > Thanks,
> >
> > Ken Dobson
> >
> > On Wed, Feb 6, 2019 at 3:35 PM Ken Dobson <kdobson at redhat.com> wrote:
> >
> >> Hi all,
> >>
> >> Some updates regarding testing with the benchmarks.
> >>
> >> For specJVM after a number of tests I've noticed no significant
> >> differences in performance between the tests that are recorded and the
> >> tests that aren't. That being said the specJVM benchmark only emits
> ~5000
> >> heap region transition events which seems to be 1-2 orders of magnitude
> >> smaller than what I'd expect from a normal process so I don't think this
> >> provides any quality information regarding the performance impact.
> >>
> >> With SpecJBB the jOPS numbers I've gotten were:
> >>
> >> WithRecording
> >> Max =14096
> >> Geomean=5899
> >>
> >> NoRecording
> >> Max=12177
> >> Geomean=5737
> >>
> >> Not sure why the results are the opposite of what would be expected so
> any
> >> insight would be appreciated. I ran the test on this machine:
> >>
> https://beaker.engineering.redhat.com/view/hp-dl785g6-01.rhts.eng.bos.redhat.com#details
> >> with -Xmx=50G and -Xms=50G.
> >>
> >> I can zip up the whole results page if that would be helpful.
> >>
> >> Thanks,
> >>
> >> Ken Dobson
> >>
> >>
> >>
> >>
> >> On Wed, Jan 30, 2019 at 12:03 PM Ken Dobson <kdobson at redhat.com> wrote:
> >>
> >>> Thank you this is great. I don't have the benchmarks no, drop them
> >>> wherever is easiest for you.
> >>>
> >>> Thanks,
> >>>
> >>> Ken
> >>>
> >>> On Wed, Jan 30, 2019 at 11:28 AM <zgu at redhat.com> wrote:
> >>>
> >>>> On Wed, 2019-01-30 at 10:54 -0500, Ken Dobson wrote:
> >>>>> Hi Zhengyu,
> >>>>>
> >>>>> We should still find out the impact when those events are being
> >>>>> recorded to ensure it's not too significant. Would you be able to
> >>>>> instruct me as to how to run the benchmarks so that I can measure the
> >>>>> performance while the JVM is being recorded vs recording disabled?
> >>>> Okay, we usually run specJVM and specJBB, do you have the benchmarks?
> >>>> If not, where can I drop them?
> >>>>
> >>>> For specJVM, the commandline I use:
> >>>> ${JAVA_HOME}/bin/java -jar jmh-specjvm2016.jar Derby  --jvmArgs
> "-Xmx1g
> >>>> -Xms1g -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC ..." -f 3
> >>>>
> >>>> For specJBB, my script attached.
> >>>>
> >>>> Thanks,
> >>>>
> >>>> -Zhengyu
> >>>>
> >>>>
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>> Ken
> >>>>>
> >>>>> On Tue, Jan 29, 2019 at 1:05 PM <zgu at redhat.com> wrote:
> >>>>>> On Tue, 2019-01-29 at 18:25 +0100, Aleksey Shipilev wrote:
> >>>>>>> On 1/29/19 6:03 PM, Ken Dobson wrote:
> >>>>>>>> Just following up on the possibility of running the benchmarks
> >>>>>> to
> >>>>>>>> measure the performance overhead.
> >>>>>>>> Please let me know if this would be possible and what I would
> >>>>>> have
> >>>>>>>> to do to get this done.
> >>>>>> I was initially worry about the amount of region state transition
> >>>>>> events generated. After adding should_commit() guard, I am now less
> >>>>>> concerned.
> >>>>>>
> >>>>>> Some overheads during recoding time, I think, are expected. So the
> >>>>>> overhead, that we are talking about, is down to the additional
> >>>>>> guard
> >>>>>> test when recording is off, I doubt it is measurable.
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> -Zhengyu
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> It is possible, and should be as simple as running the benchmarks
> >>>>>>> with/without -XX:+FlightRecorder?
> >>>>>>> You are working with Zhengyu on JFR support, right? Zhengyu knows
> >>>>>> how
> >>>>>>> to run benchmarks.
> >>>>>>>
> >>>>>>>> On Thu, Jan 24, 2019 at 11:28 AM Ken Dobson <kdobson at redhat.com
> >>>>>>>> <mailto:kdobson at redhat.com>> wrote:
> >>>>>>>>     The G1 version currently intercepts individual transitions
> >>>>>> so
> >>>>>>>> I'd hope they've measured the
> >>>>>>>>     overhead and found it was acceptable but can't be certain
> >>>>>> of
> >>>>>>>> that. Yes I agree that's definitely
> >>>>>>>>     the first step. Generally the default JFR profiling
> >>>>>>>> configuration is ~2% overhead but detailed
> >>>>>>>>     events such as these are not enabled in that configuration.
> >>>>>>>> When using these events I think it
> >>>>>>>>     would be best to disable all the default events and only
> >>>>>> enable
> >>>>>>>> the two Shenandoah Events to
> >>>>>>>>     reduce the overhead. If you think measuring the benchmarks
> >>>>>> is
> >>>>>>>> the best way to get this data I'd
> >>>>>>>>     be happy to do this if you can point me in the right
> >>>>>> direction.
> >>>>>>>
> >>>>>>> The first rule of benchmarking is not assuming anything,
> >>>>>> including
> >>>>>>> that someone else did them,
> >>>>>>> especially for a different implementation.
> >>>>>>>
> >>>>>>> There is also a bigger question: how much additional latency this
> >>>>>>> brings to Shenandoah (tiny)
> >>>>>>> pauses, when there are lots of transitions happen? Shenandoah
> >>>>>> logs
> >>>>>>> times with -Xlog:gc, and summary
> >>>>>>> times with -Xlog:gc+stats.
> >>>>>>>
> >>>>>>> -Aleksey
> >>>
> >>>
>
>