RFR: 8340733: Add scope for relaxing constraint on JavaCalls from CompilerThread [v2]

[Tomáš Zezula]

On Fri, 4 Oct 2024 15:04:49 GMT, Tomáš Zezula <duke at openjdk.org> wrote:

>> [JDK-8318694](https://bugs.openjdk.org/browse/JDK-8318694) limited the ability for JVMCI CompilerThreads to make Java upcalls. This is to mitigate against deadlock when an upcall does class loading. Class loading can easily create deadlock situations in -Xcomp or -Xbatch mode.
>> 
>> However, for Truffle, upcalls are unavoidable if Truffle partial evaluation occurs as part of JIT compilation inlining. This occurs when the Graal inliner sees a constant Truffle AST node which allows a Truffle-specific inlining extension to perform Truffle partial evaluation (PE) on the constant. Such PE involves upcalls to the Truffle runtime (running in Java).
>> 
>> This PR provides the escape hatch such that Truffle specific logic can put a compiler thread into "allow Java upcall" mode during the scope of the Truffle logic.
>
> Tomáš Zezula has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits:
> 
>  - Libgraal does not allow _can_call_java.
>  - rename changeCompilerThreadCanCallJava to updateCompilerThreadCanCallJava
>  - added CompilerThreadCanCallJavaScope

I have simplified the `_can_call_java` transitions. The only feature in the libjvmci compiler that requires Java calls is Truffle compiler, which utilizes JNI to invoke the Truffle runtime methods. Given that we now have `CompilerThreadCanCallJavaScope`, which Truffle can use to explicitly enable Java calls, we can safely disable Java calls by default for the libjvmci compiler.

For the Java JVMCI compiler, we still need to permit Java calls to accommodate upcalls to the Graal compiler and for InterpreterRuntime while running the Java JVMCI compiler.

The simplification eliminates the need for `TriBool` for `_can_call_java`; it can remain a `bool`.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21285#issuecomment-2393939688