What actions are allowed in an JVMTI ResourceExhausted event handler?

[Thomas Stüfe]

On Wed, Nov 14, 2018 at 6:58 AM David Holmes <david.holmes at oracle.com> wrote:
>
> On 14/11/2018 3:37 pm, Thomas Stüfe wrote:
> > On Wed, Nov 14, 2018, 06:32 David Holmes <david.holmes at oracle.com
> > <mailto:david.holmes at oracle.com> wrote:
> >
> >     Hi Thomas,
> >
> >     On 14/11/2018 6:50 am, Thomas Stüfe wrote:
> >      > Hi all,
> >      >
> >      > We have a client using CloudFoundry and its "jvmkill" agent. That
> >     is a
> >      > tiny JVMTI agent (see https://github.com/cloudfoundry/jvmkill) which
> >      > subscribes to the JVMTI ResourceExhausted Event. In the handler it
> >      > then does call JVMTI FollowReferences() to produce a heap histogram.
> >      >
> >      > The thing is, at our client we seem to run out of Metaspace in a
> >      > compiler thread. That thread normally would swallow the Metaspace OOM
> >      > and just bailout from the compilation. But as part of the metaspace
> >      > OOME handling the ResourceExhausted event gets posted, the handler
> >      > then uses JVMTI FollowReferences() and attempts to print out the heap
> >      > histogram, then runs into a guarantee since the compiler thread
> >     cannot
> >      > call java methods.
> >      >
> >      > My question is: are there any limitations about what one can do
> >     inside
> >      > a ResourceExhausted event handler?
> >
> >     Not specified no. But the reality of JVM TI is that you can't
> >     anticipate
> >     every execution context and there are times when there are implicit
> >     constraints imposed by the implementation.
> >
> >     In this case I think we have a mismatch between the fact we post the
> >     event from the compiler thread, but that a compiler thread is not a
> >     true
> >     "Java thread" and so can not execute arbitrary JNI or JVM TI code,
> >     or in
> >     particular can not lead to executing Java code. I think we should
> >     not be
> >     posting the event from the compiler thread in this case.
> >
> >     Cheers,
> >     David
> >
> >
> > Hi David,
> >
> > Yes I thought so too. I'll prepare a fix.
>
> My thought on the fix is that we need to check if
> Thread::current()->can_call_java(). And that should probably be inside
> the JvmtiExport::should_post_xxx implementation.
>

I wonder whether that may be too harsh. JVMTI agents may not
necessarily call into java as reaction to ResourceExhausted. I would
have limited this to !CompilerThread, and only in Metaspace.

Also, looking at CompilerThread::can_call_java(), I see that we return
true for jvmci compilers. Still do we want to post it there?

But I am not sure. What do you think?

> Cheers,
> David
>
> > Thanks, Thomas
> >
> >
> >      > I checked the
> >     https://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html
> >      > documentation, but I cannot find any mentioning of limitations in
> >     that
> >      > case.
> >      >
> >      > Thanks and Best Regards, Thomas
> >      >
> >