RFR: 8368159: Significant performance overhead when started with jdwp agent and unattached debugger [v2]

[Leonid Mesnik]

On Sat, 20 Sep 2025 05:10:50 GMT, Serguei Spitsyn <sspitsyn at openjdk.org> wrote:

>> This update removes a significant performance overhead when an application running millions of virtual threads is started with the JDWP agent but debugger has not been attached. The overhead is 4X-6X slowdown.
>> The tested app normally (without debug agent) takes around 3+ seconds. With debug agent enabled it takes 14 seconds and more. The performance overhead is caused by the `jvmti_yield_cleanup()` recursively calling `JvmtiExport::continuation_yield_cleanup()`. The reason of this overhead is because the function `JvmtiExport::can_post_frame_pop()` is used to identify a need for the JVMTI cleanup which is not precise and triggers unneeded work when debugger has not been attached yet. The fix is to trigger  the JVMTI cleanup with new function: `bool JvmtiExport::has_frame_pops(JavaThread* thread)`.
>> 
>> Testing:
>> - Insure the 4X-6X slowdown is gone for an application running millions of virtual threads and started with JDWP agent
>> - Mach5 tiers 1-6 are all passed
>
> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix minimal build broken by incorrect macro usage

src/hotspot/share/runtime/continuationFreezeThaw.cpp line 1622:

> 1620: 
> 1621: static void invalidate_jvmti_stack(JavaThread* thread) {
> 1622:   if (JvmtiExport::can_post_frame_pop() || thread->is_interp_only_mode()) {

Can you please explain why this change is required? Doesn't 'invalidate_cur_stack_depth' make sense only when interp_only mode is enabled for the threads only?
It is invalidated once thread switched to interp only.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/27403#discussion_r2365734395