Do compiler threads allocate metaspace?

[Thomas Stüfe]

Hi Vladimir,

thank you for your answers. I looked into the code and I think I
understand this a bit better.

Please find some more questions inline.

On Tue, Nov 13, 2018 at 7:15 PM Vladimir Kozlov
<vladimir.kozlov at oracle.com> wrote:
>
> Yes, compilers (C1 and C2) do allocations in Metaspace - MethodData and MethodCounter objects.
>
> On 11/13/18 9:26 AM, Thomas Stüfe wrote:
> > Hi all,
> >
> > can compiler threads allocate Metaspace?
> >
> > Background:
> >
> > Clients of us use a "jvmkill" jvmti agent in their cloudfoundry
> > infrastructure. That agent (see
> > https://github.com/cloudfoundry/jvmkill) subscribes to the JVMTI
> > ResourceExhausted Event and then, in the handler, uses JVMTI
> > FollowReferences() to produce a heap histogram.
> >
> > What seems to happen is this: We run into a Metaspace OOM, then the
> > JVMTI agent attempts to print its histogram, walks part of the heap,
> > then we run into a "guarantee(!thread->is_Compiler_thread())".
> >
> > Therefore I wonder whether compiler threads can allocate Metaspace?
> > Looking at the sources, I see potential call paths from the compiler,
> > e.g. Method::build_interpreter_method_data(), which will allocate
> > space in Metaspace for a MethodData object, and which in turn, to me,
> > looks like it could be invoked from compiler code (e.g. during
> > deoptimization).
>
> MethodData and MethodCounter are used for profiling (the main purpose) and record deoptimization reasons as you pointed.
> Interpreter and C1 generated code are accessing it to write profiling data. Both C1 and C2 can create such objects
> during compilation if Interpreter did not create them yet. The profiling data is accesses by C2 during compilation -
> this is the main purpose.
>
> >
> > If yes - compiler can allocate metaspace - how would they deal with
> > Metaspace OOMs?
>
> If MethodData is not available (failed to allocate) we create empty ciMethodData and use it during C2 compilation
> (compilation will be similar to -Xcomp compilation):
>
> http://hg.openjdk.java.net/jdk/jdk/file/2e64b70c03b1/src/hotspot/share/ci/ciMethod.cpp#l961
>
> We bailout C1 Tier3 compilation (with profiling) if no data available:
>
> http://hg.openjdk.java.net/jdk/jdk/file/2e64b70c03b1/src/hotspot/share/c1/c1_Compilation.cpp#l381
>

Ah. I wondered how the compiler passes the OOM on to the user, but I
see that it just doesn't. When the compiler fails to allocate
MethodData or MethodCounter object, it always seems to clean the OOM
exception afterwards. So that a compiler thread will never leave a
Metaspace OOM pending? Worst thing, the next caller attempting to
allocate from metaspace will hit the same path and re-rise the OOM?

Also (just to check that I understood it): I looked at all callers of
"Method::build_interpreter_method_data()" - which could leak a OOM and
does require its callers to clear it - and all callers clear do so but
some do also assert that it is in fact an OOM, some do not. Namely,
TieredThresholdPolicy::create_mdo() and ciMethod::ensure_method_data
do no assert for type==OOM. Is that just coincidence or is there a
deeper meaning (e..g at these two points exceptions other than OOM
could happen)?

Also, I wonder: if we never want OOMs to be thrown for failed
MethodData and MethodCounter allocations, would it not make sense to
not throw those exceptions in metaspace in the first place and save on
all the exception clearing afterwards?

About the JVMTI ResourceExhausted Event, I think I ask over at
serviceability what the rules are for ResourceExhausted handlers.

Thanks a lot!

Thomas

> Vladimir
>
> >
> > Also note that I am not sure at all whether that type of usage -
> > calling FollowReferences() when handling ResourceExhausted - is even
> > allowed in JVMTI.
> >
> > Thanks a lot,
> >
> > Best Regards, Thomas
> >