RFR: 8306992: [JVMCI] mitigate more against JVMCI related OOME causing VM to exit

[Doug Simon]

On Wed, 10 May 2023 14:00:51 GMT, Doug Simon <dnsimon at openjdk.org> wrote:

> This PR makes the following changes to mitigate against an OOME or other recoverable error in JVMCI from causing the VM to exit:
> * Tracks upcalls into libjvmci or creation of libjvmci.
> * If 10% or more of these calls fail with an uncaught exception, then JVMCI compilation is disabled (i.e. future compilations fall back to Tier 1).
> 
> When JVMCI compilation is disabled, a warning is emitted:
> 
> [0.064s][warning][jit,compilation] JVMCI compiler disabled after 11 of 15 upcalls had errors (Last error: "uncaught exception in call_HotSpotJVMCIRuntime_compileMethod"). Use -Xlog:jit+compilation for more detail.
> 
> 
> With `-Xlog:jit+compilation`, the extra detail shown is:
> 
> [0.182s][info][jit,compilation] uncaught exception in call_HotSpotJVMCIRuntime_compileMethod while compiling java.util.stream.StreamOpFlag.fromCharacteristics(Ljava/util/Spliterator;)I
> Exception in thread "JVMCI-native CompilerThread0": java.lang.InternalError
> java.lang.InternalError: aborting compilation of HotSpotMethod<Object.<init>()>
> 	at jdk.internal.vm.ci at 20.0.2-internal/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.compileMethod(HotSpotJVMCIRuntime.java:923)
> 
> 
> Note that the errors treated by the changes in the PR are expected to be exceedingly rare. For example, an OOME while starting a libgraal isolate or initializing the JVMCI compiler. Exceptions thrown during compilation are already handled by the Graal [CompilationWrapper](https://github.com/oracle/graal/blob/431ecf7d26f56cee49708854fe0e89b05514492b/compiler/src/jdk.internal.vm.compiler/src/org/graalvm/compiler/core/CompilationWrapper.java#L65).

src/hotspot/share/jvmci/jvmci.cpp line 236:

> 234:       JavaThreadState state = JavaThread::cast(thread)->thread_state();
> 235:       if (state == _thread_in_vm || state == _thread_in_Java || state == _thread_new) {
> 236:         tty->print("JVMCITrace-%d[%s]:%*c", level, thread->name(), level, ' ');

This change helps correlate threads in a trace that transition in and out of libgraal (thread name is not available when in libgraal).

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/13905#discussion_r1189978954