RFR: 8341273: JVMTI is not properly hiding some continuation related methods [v11]

[Serguei Spitsyn]

On Sat, 26 Oct 2024 06:34:08 GMT, Serguei Spitsyn <sspitsyn at openjdk.org> wrote:

>> src/hotspot/share/prims/jvmtiEnvBase.cpp line 667:
>> 
>>> 665: 
>>> 666: javaVFrame*
>>> 667: JvmtiEnvBase::check_and_skip_hidden_frames(bool is_in_VTMS_transition, javaVFrame* jvf) {
>> 
>> reworked function looks much better! Now it's clear what the function does and I have a question what it should do.
>> The function checks `@JvmtiMountTransition` annotation first even if the thread is in transition.
>> If `@JvmtiMountTransition` is present, the code doesn't care about `@ChangesCurrentThread` (and doesn't case about `@JvmtiMountTransition` if in_transition + `@ChangesCurrentThread`).
>> But we have 2 methods in VirtualThread class (`switchToCarrierThread()` and `switchToVirtualThread()`) with both annotations.
>> So if the function is called when `switchToCarrierThread` is on top, the function skips this frame, but if the thread calls some other function without `@JvmtiMountTransition` annotation from `switchToCarrierThread`, the function returns `switchToCarrierThread`.
>
> Good catch, thanks.
> These two functions are impacted by temporary VTMS transitions. It seems we fail to hide frames in these transitions while we skip the JVMTI events in their context. I'll try to fix this issue.

I'd suggest to file a separate bug on this issue as it has to be tested well.
Please, let me know if you are okay with it.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21397#discussion_r1817700598