RFR: 8341273: JVMTI is not properly hiding some continuation related methods [v3]

[Serguei Spitsyn]

On Tue, 15 Oct 2024 21:21:07 GMT, Alex Menkov <amenkov at openjdk.org> wrote:

>> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Disallow NotifyFramePop for enter/enter0/VirtualThread.run/VThreadContinuation.run
>
> src/hotspot/share/prims/jvmtiExport.cpp line 1814:
> 
>> 1812:   }
>> 1813:   // pure continuations have no VTMS transitions but they use methods annotated with JvmtiMountTransition
>> 1814:   if ((mh->jvmti_mount_transition() && state->is_virtual()) || thread->is_in_any_VTMS_transition()) {
> 
> Could you please explain the change (and other similar changes in jvmtiExport.cpp)
> Before events were not posted for any `@JvmtiMountTransition` methods, and now only for virtual threads? I.e. events for `@JvmtiMountTransition` methods of carrier threads are posted?

Good question. Some `jvmtiNotify*` notifications can be called in a context of carrier thread and exited in a context of virtual thread and vice versa. So, you are right we should not post these events for `@JvmtiMountTransition` methods, in a context of carrier threads.

So, we need to adjust these conditions even more with something like below:

-  if ((mh->jvmti_mount_transition() && state->is_virtual()) || thread->is_in_any_VTMS_transition()) {
+  if ((mh->jvmti_mount_transition() && (state->is_virtual() || thread->last_continuation() == nullptr)) ||
+    thread->is_in_any_VTMS_transition()) {
     return; // no events should be posted if thread is in any VTMS transition
   }

The check for `thread->last_continuation() == nullptr` will help to skip these events for carrier threads as well but won't skip them for pure continuations. In fact, I do not like the complexity of this check. :(

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21397#discussion_r1803512093