RFR: 8341273: JVMTI is not properly hiding some continuation related methods [v11]

[Alex Menkov]

On Thu, 24 Oct 2024 14:52:27 GMT, Serguei Spitsyn <sspitsyn at openjdk.org> wrote:

>> This fixes a problem in the VTMS (Virtual Thread Mount State) transition frames hiding mechanism.
>> Please, see a fix description in the first comment.
>> 
>> Testing:
>>  - Verified with new test `vthread/CheckHiddenFrames`
>>  - Mach5 tiers 1-6 are passed
>
> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision:
> 
>   review: removed asserts from continuationFreezeThaw.cpp again

src/hotspot/share/prims/jvmtiEnvBase.cpp line 667:

> 665: 
> 666: javaVFrame*
> 667: JvmtiEnvBase::check_and_skip_hidden_frames(bool is_in_VTMS_transition, javaVFrame* jvf) {

reworked function looks much better! Now it's clear what the function does and I have a question what it should do.
The function checks `@JvmtiMountTransition` annotation first even if the thread is in transition.
If `@JvmtiMountTransition` is present, the code doesn't care about `@ChangesCurrentThread` (and doesn't case about `@JvmtiMountTransition` if in_transition + `@ChangesCurrentThread`).
But we have 2 methods in VirtualThread class (`switchToCarrierThread()` and `switchToVirtualThread()`) with both annotations.
So if the function is called when `switchToCarrierThread` is on top, the function skips this frame, but if the thread calls some other function without `@JvmtiMountTransition` annotation from `switchToCarrierThread`, the function returns `switchToCarrierThread`.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21397#discussion_r1815790409