RFR: 8369515: Deadlock between JVMTI and JNI ReleasePrimitiveArrayCritical [v3]

[Richard Reingruber]

On Fri, 19 Dec 2025 01:55:29 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> As discussed in JBS the deadlock occurs when the call to `ReleasePrimitiveArrayCritical` performs the transition from native to VM, and in the process checks for special runtime exit conditions - which includes the `obj_deopt_suspend` request. The simple solution is to define a custom JNI ENTRY with custom `ThreadInVMfromNative` that elides the exit check.
>> 
>> The change is limited to `ReleasePrimitiveArrayCritical` and `ReleaseStringCritical`.
>> 
>> UPDATE: we are now employing a much simpler solution.
>> 
>> There is no regression test as this has only been seen in long running stress tests.
>> 
>> Testing:
>>    -tiers 1-6
>
> David Holmes has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Exapnd comment

Looks good.
Thanks, Richard.

A deadlock can still occur if the debugger suspends the thread in the critical region, e.g. to read a local variable. After [JDK-8373839](https://bugs.openjdk.org/browse/JDK-8373839) this shouldn't be possible anymore.

-------------

Marked as reviewed by rrich (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/28779#pullrequestreview-3598687723
PR Comment: https://git.openjdk.org/jdk/pull/28779#issuecomment-3675155037