What actions are allowed in an JVMTI ResourceExhausted event handler?

[Thomas Stüfe]

I did open a bug to track this: https://bugs.openjdk.java.net/browse/JDK-8213834
On Wed, Nov 14, 2018 at 7:38 AM Thomas Stüfe <thomas.stuefe at gmail.com> wrote:
>
> On Wed, Nov 14, 2018 at 6:58 AM David Holmes <david.holmes at oracle.com> wrote:
> >
> > On 14/11/2018 3:37 pm, Thomas Stüfe wrote:
> > > On Wed, Nov 14, 2018, 06:32 David Holmes <david.holmes at oracle.com
> > > <mailto:david.holmes at oracle.com> wrote:
> > >
> > >     Hi Thomas,
> > >
> > >     On 14/11/2018 6:50 am, Thomas Stüfe wrote:
> > >      > Hi all,
> > >      >
> > >      > We have a client using CloudFoundry and its "jvmkill" agent. That
> > >     is a
> > >      > tiny JVMTI agent (see https://github.com/cloudfoundry/jvmkill) which
> > >      > subscribes to the JVMTI ResourceExhausted Event. In the handler it
> > >      > then does call JVMTI FollowReferences() to produce a heap histogram.
> > >      >
> > >      > The thing is, at our client we seem to run out of Metaspace in a
> > >      > compiler thread. That thread normally would swallow the Metaspace OOM
> > >      > and just bailout from the compilation. But as part of the metaspace
> > >      > OOME handling the ResourceExhausted event gets posted, the handler
> > >      > then uses JVMTI FollowReferences() and attempts to print out the heap
> > >      > histogram, then runs into a guarantee since the compiler thread
> > >     cannot
> > >      > call java methods.
> > >      >
> > >      > My question is: are there any limitations about what one can do
> > >     inside
> > >      > a ResourceExhausted event handler?
> > >
> > >     Not specified no. But the reality of JVM TI is that you can't
> > >     anticipate
> > >     every execution context and there are times when there are implicit
> > >     constraints imposed by the implementation.
> > >
> > >     In this case I think we have a mismatch between the fact we post the
> > >     event from the compiler thread, but that a compiler thread is not a
> > >     true
> > >     "Java thread" and so can not execute arbitrary JNI or JVM TI code,
> > >     or in
> > >     particular can not lead to executing Java code. I think we should
> > >     not be
> > >     posting the event from the compiler thread in this case.
> > >
> > >     Cheers,
> > >     David
> > >
> > >
> > > Hi David,
> > >
> > > Yes I thought so too. I'll prepare a fix.
> >
> > My thought on the fix is that we need to check if
> > Thread::current()->can_call_java(). And that should probably be inside
> > the JvmtiExport::should_post_xxx implementation.
> >
>
> I wonder whether that may be too harsh. JVMTI agents may not
> necessarily call into java as reaction to ResourceExhausted. I would
> have limited this to !CompilerThread, and only in Metaspace.
>
> Also, looking at CompilerThread::can_call_java(), I see that we return
> true for jvmci compilers. Still do we want to post it there?
>
> But I am not sure. What do you think?
>
> > Cheers,
> > David
> >
> > > Thanks, Thomas
> > >
> > >
> > >      > I checked the
> > >     https://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html
> > >      > documentation, but I cannot find any mentioning of limitations in
> > >     that
> > >      > case.
> > >      >
> > >      > Thanks and Best Regards, Thomas
> > >      >
> > >