Increased safepoint cleanup tasks in Java 11 (vs 8)

[Vitaly Davidovich]

On Wed, Jun 5, 2019 at 6:55 PM David Holmes <david.holmes at oracle.com> wrote:

> On 6/06/2019 2:31 am, Vitaly Davidovich wrote:
> > On Wed, Jun 5, 2019 at 12:15 PM Daniel D. Daugherty <
> > daniel.daugherty at oracle.com> wrote:
> >
> >> On 6/5/19 12:04 PM, Vitaly Davidovich wrote:
> >>
> >>
> >>
> >> On Wed, Jun 5, 2019 at 11:44 AM Daniel D. Daugherty <
> >> daniel.daugherty at oracle.com> wrote:
> >>
> >>> On 6/5/19 11:37 AM, Vitaly Davidovich wrote:
> >>>> On Wed, Jun 5, 2019 at 8:58 AM Vitaly Davidovich <vitalyd at gmail.com>
> >>> wrote:
> >>>>
> >>>>> Hey Aleksey,
> >>>>>
> >>>>> On Wed, Jun 5, 2019 at 8:52 AM Aleksey Shipilev <shade at redhat.com>
> >>> wrote:
> >>>>>
> >>>>>> On 6/5/19 2:33 PM, Vitaly Davidovich wrote:
> >>>>>>> Does anyone have ideas on pinpointing this further?
> >>>>>> I have the educated guess:
> >>>>>>     https://bugs.openjdk.java.net/browse/JDK-8181859
> >>>>>>
> >>>>>> Which means you can try if -XX:MonitorUsedDeflationThreshold=0
> reduces
> >>>>>> the number of "guaranteed"
> >>>>>> safepoints.
> >>>>>>
> >>>>> Hmm, the application doesn't use JVM monitors (in any meaningful
> >>>>> capacity).  But interesting JBS entry - I will try it out, thanks!
> The
> >>>>> description of safepoint clean up in there is/was consistent with my
> >>>>> understanding as well.  On Java 8, we'd see the "no vm operation"
> >>> safepoint
> >>>>> operations, and I always assumed it was IC buffer cleaning (even had
> a
> >>> mail
> >>>>> thread on that topic a few years ago, either on this list or
> >>>>> hotspot-compiler).  That tended to stabilize (i.e. they went away)
> >>> after
> >>>>> the application reached steady state, whereas I'm seeing essentially
> >>>>> non-stop safepoint cleanups on 11.
> >>>>>
> >>>> Ok, adding -XX:MonitorUsedDeflationThreshold=0 seems to alleviate the
> >>> issue
> >>>> - # of entered safepoint cleanups looks on par with Java 8, although
> I'm
> >>>> still assessing a bit more.
> >>>
> >>> This observation doesn't really match with your earlier comment:
> >>>
> >>>> Hmm, the application doesn't use JVM monitors (in any meaningful
> >>>> capacity).
> >>>
> >>> The default value for MonitorUsedDeflationThreshold is 90. So a cleanup
> >>> safepoint should only be triggered if monitor use exceeds 90% of the
> >>> monitor pool.
> >>>
> >> Hmm, ok - I see what you mean:
> >>
> https://github.com/openjdk/jdk11u/blob/d46b56eb81b07b452141c7dcd078d33fc8e49d63/src/hotspot/share/runtime/safepoint.cpp#L604
> >>
> >> So there's either monitor usage that I'm not aware of (or forgot about)
> or
> >> the ratio is > 90 but the absolute # of them is small (which I'd
> consider
> >> "don't use in any meaningful capacity").  Is there a way to print out
> >> monitor population so I can see?
> >>
> >>
> >> I've been adding a lot of logging support to the Java Monitor subsystem,
> >> but those changes are in JDK13 and not in JDK11. I'm trying to remember
> >> what there is in JDK11, but ...
> >>
> > Yeah, I'm having a hard time keeping track of what features/changes are
> in
> > what Java release or update version or upstream/downstream backport or
> ...
> >
> > By the way, do JNI (specifically, Java calling into native) calls cause
> > monitor inflation? The app in question does have substantial JNI usage.
> I
> > guess by not using java monitors extensively, I meant the vanilla flavor
> of
> > Java code with synchronized blocks/methods.
>
> JNI calls do not in themselves impact monitors. However if native code
> uses the JNI MonitorEnter then that does trigger inflation.
>
> David
> -----

Thanks for clarifying, David.

Pretty certain that JNI code I have is not calling MonitorEnter, but will
double check.

So beyond monitor contention via vanilla Java code, are there other
scenarios resulting in inflation? Is there perhaps a list of these that
someone could point to? I can spelunk through hotspot code, but maybe
someone can spare me that exercise :).

Thanks!

>
>
> >
> >>
> >> Dan
> >>
> >>
> >>
> >> Thanks Dan
> >>
> >>>
> >>> Dan
> >>>
> >>>
> >>>>
> >>>> Of course now the trouble is this is an experimental option, and
> >>> enabling
> >>>> things like that in production raises eyebrows.  Is it safe to assume,
> >>>> however, that setting this to 0 essentially behaves like Java 8? It
> >>> appears
> >>>> so, based on that JBS Aleksey linked above.  But just double checking.
> >>>>
> >>>> If that's the case, then perhaps we can overlook the experimental bit
> >>> and
> >>>> wait until this is done outside safepoints, per Robbin's email.
> >>>>
> >>>> Thanks to everyone that responded on this thread! Very helpful.
> >>>>
> >>>>>> --
> >>>>>> Thanks,
> >>>>>> -Aleksey
> >>>>>>
> >>>>>>
> >>>
> >>>
> >>
>
-- 
Sent from my phone