[13] RFR(L) 8220623: [JVMCI] Update JVMCI to support JVMCI based Compiler compiled into shared library

[Robbin Ehn]

Hi Doug,

> This change was made primarily in the context of libgraal.
> 
> It can happen that a JVMCI compiler thread acquires a lock in libgraal, enters HotSpot
> and goes to sleep in the shutdown safepoint. Another JVMCI compiler thread then
> attempts to acquire the same lock and goes to sleep in libgraal which from HotSpot’s
> perspective is the _thread_in_native state.

Ok.

> 
> This is the original fix I had for this:
> 
>            CompilerThread* ct = (CompilerThread*) thr;
>            if (ct->compiler() == NULL || !ct->compiler()->is_jvmci() JVMCI_ONLY(|| !UseJVMCINativeLibrary)) {
>              num_active_compiler_thread++;
>            } else {
>              // When using a compiler in a JVMCI shared library, it's possible
>              // for one compiler thread to grab a lock in the shared library,
>              // enter HotSpot and go to sleep on the shutdown safepoint. Another
>              // JVMCI shared library compiler thread can then attempt to grab the
>              // lock and thus never make progress.
>            }
> 
> which is probably the right one. I hadn’t realized that a JavaGraal
> (as opposed to libgraal) JVMCI compiler thread blocked on a lock will be in
> the blocked state, not in the _thread_in_native state.

Yes, makes more sense.

Another thing is this HandleMark:

  JvmtiAgentThread::call_start_function() {
+    HandleMark hm(this);
      ThreadToNativeFromVM transition(this);

Since a safepoint can happen at any time when you are in native, I don't see how 
using a Handle in native would be safe or correct. I'm guessing you are missing 
a HandleMark somewhere when you re-enter VM?

Thanks, Robbin

> 
> -Doug
>