RFR: 8369515: Deadlock between JVMTI and JNI ReleasePrimitiveArrayCritical

[David Holmes]

On Fri, 12 Dec 2025 04:20:11 GMT, David Holmes <dholmes at openjdk.org> wrote:

> As discussed in JBS the deadlock occurs when the call to `ReleasePrimitiveArrayCritical` performs the transition from native to VM, and in the process checks for special runtime exit conditions - which includes the `obj_deopt_suspend` request. The simple solution is to define a custom JNI ENTRY with custom `ThreadInVMfromNative` that elides the exit check.
> 
> The change is limited to `ReleasePrimitiveArrayCritical` and `ReleaseStringCritical`.
> 
> There is no regression test as this has only been seen in long running stress tests.
> 
> Testing:
>    -tiers 1-6

> I do think that also entering a critical region is problematic if it is nested.

Isn't nesting critical regions against the rules of using critical regions?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/28779#issuecomment-3658251463