RFR: 8286960: Test serviceability/jvmti/vthread/SuspendResume2 crashed: missing ThreadsListHandle in calling context

[Serguei Spitsyn]

A part of this issue was contributed with the following changeset:

commit ea23e7333e03abb4aca3e9f3854bab418a4b70e2
Author: Daniel D. Daugherty <[dcubed at openjdk.org](mailto:dcubed at openjdk.org)>
Date: Mon Nov 8 14:45:04 2021 +0000

    8249004: Reduce ThreadsListHandle overhead in relation to direct handshakes
    Reviewed-by: coleenp, sspitsyn, dholmes, rehn

The following change in `src/hotspot/share/runtime/thread.cpp` added new assert:

bool JavaThread::java_suspend() {
- ThreadsListHandle tlh;
- if (!tlh.includes(this)) {
- log_trace(thread, suspend)("JavaThread:" INTPTR_FORMAT " not on ThreadsList, no suspension", p2i(this));
- return false;
- }
+ guarantee(Thread::is_JavaThread_protected(this, /* checkTLHOnly */ true),
 + "missing ThreadsListHandle in calling context.");
  return this->handshake_state()->suspend();
}

This new assert misses a check for target thread as being current `JavaThread`.

Also, the JVMTI SuspendThread is protected with TLH:

JvmtiEnv::SuspendThread(jthread thread) {
  JavaThread* current = JavaThread::current();
  ThreadsListHandle tlh(current);              <= TLS defined here!!!

   oop thread_oop = NULL;
   {
     JvmtiVTMSTransitionDisabler disabler(true); 


However, it is possible that a new carrier thread (and an associated `JavaThread`) can be created after the `TLH` was set and the target virtual thread can be mounted on new carrier thread. Then target virtual thread will be associated with newly created `JavaThread` which is unprotected by the TLH.
The right way to be protected from this situation it is to prevent mount state transitions with `JvmtiVTMSTransitionDisabler` before the TLH is set as in the change below:


@@ -929,13 +929,13 @@ JvmtiEnv::GetAllThreads(jint* threads_count_ptr, jthread** threads_ptr) {
 jvmtiError
 JvmtiEnv::SuspendThread(jthread thread) {
   JavaThread* current = JavaThread::current();
-  ThreadsListHandle tlh(current);

   jvmtiError err;
   JavaThread* java_thread = NULL;
   oop thread_oop = NULL;
   {
     JvmtiVTMSTransitionDisabler disabler(true);
+    ThreadsListHandle tlh(current);

     err = get_threadOop_and_JavaThread(tlh.list(), thread, &java_thread, &thread_oop);
     if (err != JVMTI_ERROR_NONE) {



This problem exist in all JVMTI Suspend functions:
 `SuspendThread`, `SuspendThreadList` and `SuspendAllVirtualThreads`.

-------------

Commit messages:
 - 8286960: Test serviceability/jvmti/vthread/SuspendResume2 crashed: missing ThreadsListHandle in calling context

Changes: https://git.openjdk.java.net/jdk/pull/8878/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=8878&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8286960
  Stats: 17 lines in 2 files changed: 8 ins; 7 del; 2 mod
  Patch: https://git.openjdk.java.net/jdk/pull/8878.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/8878/head:pull/8878

PR: https://git.openjdk.java.net/jdk/pull/8878