RFR: 8366659: ObjectMonitor::wait() can deadlock with a suspension request [v9]

[Daniel D. Daugherty]

On Mon, 17 Nov 2025 13:50:30 GMT, Anton Artemov <aartemov at openjdk.org> wrote:

>> Hi, please consider the following changes:
>> 
>> If suspension is allowed when a thread is re-entering an object monitor (OM), then a deadlock is possible:
>> 
>> The waiting thread is made to be a successor and is unparked. Upon a suspension request, the thread will suspend itself whilst clearing the successor. The OM will be left unlocked (not grabbed by any thread), while the other threads are parked until a thread grabs the OM and the exits it. The suspended thread is on the entry-list and can be selected as a successor again. None of other threads can be woken up to grab the OM until the suspended thread has been resumed and successfully releases the OM.
>> 
>> This can happen in two places where the successor could be suspended: 
>> 1:
>> https://github.com/openjdk/jdk/blob/6322aaba63b235cb6c73d23a932210af318404ec/src/hotspot/share/runtime/objectMonitor.cpp#L1897
>> 
>> 2:
>> https://github.com/openjdk/jdk/blob/6322aaba63b235cb6c73d23a932210af318404ec/src/hotspot/share/runtime/objectMonitor.cpp#L1149
>> 
>> The issues are addressed by not allowing suspension in case 1, and by handling the suspension request at a later stage, after the thread has grabbed the OM in `reenter_internal()` in case 2. In case of a suspension request, the thread exits the OM and enters it again once resumed. 
>> 
>> The JVMTI `waited` event posting (2nd one) is postponed until the suspended thread is resumed and has entered the OM again.  The `enter` to the OM (in case `ExitOnSuspend` did exit) is done without posting any events.
>> 
>> Tests are added for both scenarios. 
>> 
>> Tested in tiers 1 - 7.
>
> Anton Artemov has updated the pull request incrementally with one additional commit since the last revision:
> 
>   8366659: Addressed reviewer's comments.

Marked as reviewed by dcubed (Reviewer).

test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorWait/SuspendWithObjectMonitorWait.java line 140:

> 138: // launch resumer       enter threadLock
> 139: // <launch returns>     while !READY_TO_NOTIFY  resumer running
> 140: // :                    threadLock.wait(1)      wait for notify

The wait(1) call is in the while-loop. Please insert two spaces before the `threadlock.wait(1)` call.

test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorWait/SuspendWithObjectMonitorWait.java line 235:

> 233:                 } catch (NumberFormatException nfe) {
> 234:                     System.err.println("'" + args[argIndex] +
> 235:                             "': invalid test_case value.");

The leading double quote should be lined up with the one on the previous line.

test/hotspot/jtreg/serviceability/jvmti/SuspendWithObjectMonitorWait/SuspendWithObjectMonitorWait.java line 251:

> 249:                 } catch (NumberFormatException nfe) {
> 250:                     System.err.println("'" + args[argIndex] +
> 251:                             "': invalid time_max value.");

The leading double quote should be lined up with the one on the previous line.

-------------

PR Review: https://git.openjdk.org/jdk/pull/27040#pullrequestreview-3474239279
PR Review Comment: https://git.openjdk.org/jdk/pull/27040#discussion_r2535265622
PR Review Comment: https://git.openjdk.org/jdk/pull/27040#discussion_r2535273619
PR Review Comment: https://git.openjdk.org/jdk/pull/27040#discussion_r2535274696