RFR: 8318694: [JVMCI] disable can_call_java in most contexts for libjvmci compiler threads

[Doug Simon]

On Thu, 26 Oct 2023 17:39:46 GMT, Doug Simon <dnsimon at openjdk.org> wrote:

> This PR reduces the context in which a libjvmci CompilerThread can make a Java call. By default, a CompileThread for a JVMCI compiler can make Java calls (as jarjvmci only works that way). When libjvmci calls into the VM via a CompilerToVM native method, it enters a scope where Java calls are disabled until either the call returns or a nested scope is entered that re-enables Java calls. The latter is required for the Truffle use case described in [JDK-8318694](https://bugs.openjdk.org/browse/JDK-8318694) as well as for a few other VM entry points where libgraal currently still requires the ability to make Java calls (e.g. to load the Java classes used for boxing primitives). We may be able to eventually eliminate all need for libgraal to make Java calls but this PR is a first step in the right direction.

src/hotspot/share/jvmci/jvmciCompilerToVM.cpp line 224:

> 222:     jint __throw_res = env->ThrowNew(JNIJVMCI::name::clazz(), msg);               \
> 223:     if (__throw_res != JNI_OK) {                                                  \
> 224:       tty->print_cr("Throwing " #name " in " caller " returned %d", __throw_res); \

The VM should prefer event logging over printing to the console.

src/hotspot/share/jvmci/jvmciCompilerToVM.cpp line 583:

> 581: 
> 582: C2V_VMENTRY_NULL(jobject, lookupType, (JNIEnv* env, jobject, jstring jname, ARGUMENT_PAIR(accessing_klass), jint accessing_klass_loader, jboolean resolve))
> 583:   CompilerThreadCanCallJava canCallJava(thread, resolve); // Resolution requires Java calls

This is currently required by libgraal - it may be fixable in future.

src/hotspot/share/jvmci/jvmciCompilerToVM.cpp line 597:

> 595:     if (strstr(val, "<trace>") != nullptr) {
> 596:       tty->print_cr("CompilerToVM.lookupType: %s", str);
> 597:     } else if (strstr(str, val) != nullptr) {

This fixes an existing bug: the test is meant to be whether `val` is a substring of `str`, not the other way around.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16383#discussion_r1373521606
PR Review Comment: https://git.openjdk.org/jdk/pull/16383#discussion_r1373846457
PR Review Comment: https://git.openjdk.org/jdk/pull/16383#discussion_r1373522668