RFR: 8368159: Significant performance overhead when started with jdwp agent and unattached debugger [v3]

[Leonid Mesnik]

On Thu, 25 Sep 2025 00:18:02 GMT, Serguei Spitsyn <sspitsyn at openjdk.org> wrote:

>> This update removes a significant performance overhead when an application running millions of virtual threads is started with the JDWP agent but debugger has not been attached. The overhead is 4X-6X slowdown.
>> The tested app normally (without debug agent) takes around 3+ seconds. With debug agent enabled it takes 14 seconds and more. The performance overhead is caused by the `jvmti_yield_cleanup()` recursively calling `JvmtiExport::continuation_yield_cleanup()`. The reason of this overhead is because the function `JvmtiExport::can_post_frame_pop()` is used to identify a need for the JVMTI cleanup which is not precise and triggers unneeded work when debugger has not been attached yet. The fix is to trigger  the JVMTI cleanup with new function: `bool JvmtiExport::has_frame_pops(JavaThread* thread)`.
>> 
>> Testing:
>> - Insure the 4X-6X slowdown is gone for an application running millions of virtual threads and started with JDWP agent
>> - Mach5 tiers 1-6 are all passed
>
> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision:
> 
>   review: remove unneeded condition from invalidate_jvmti_stack()

Marked as reviewed by lmesnik (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/27403#pullrequestreview-3265088332