<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p>I got my prototype ThreadNode cleanup code working again. It
seems to be doing the job of keeping the list down to a minimal
size, with usually at most a few ThreadNodes on the list. However,
that didn't help performance. The reason is because the debug
agent normally does not need to traverse the list. It maps jthread
to ThreadNode by using JVMTI thread local data, not by searching
the list, and the list is doubly linked, so ThreadNodes can be
removed quickly.</p>
<p>After the above results I talked with Serguei and he confirmed
that JVMTI also has a linked list, and there are some occasions
where it needs to be traversed, so that likely the bottleneck
here.</p>
<p>Regarding my suggestion that we may be able to disable
JVMTI_EVENT_VIRTUAL_THREAD_START/END, I did experiment with that
and when disabled there are no performance issues, so I will look
into getting this to work properly. The events need to be enable
if there are any ThreadStartRequests or ThreadDeathRequests that
do not have the PlatformThreadOnly filter enabled.</p>
<p><a class="moz-txt-link-freetext" href="https://docs.oracle.com/en/java/javase/24/docs/api/jdk.jdi/com/sun/jdi/request/ThreadStartRequest.html#addPlatformThreadsOnlyFilter()">https://docs.oracle.com/en/java/javase/24/docs/api/jdk.jdi/com/sun/jdi/request/ThreadStartRequest.html#addPlatformThreadsOnlyFilter()</a></p>
<p>Jdb by default uses this filter, and I suspect that IDEA does
also. If not, it would get a flood of ThreadStart and ThreadDeath
events for all the virtual threads.</p>
<p>Chris<br>
</p>
<div class="moz-cite-prefix">On 4/3/25 2:15 AM, Serguei Spitsyn
wrote:<br>
</div>
<blockquote type="cite" cite="mid:CH2PR10MB43251C8336A7CEB86B4FB17A8DAE2@CH2PR10MB4325.namprd10.prod.outlook.com">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Aptos;
panose-1:2 11 0 4 2 2 2 2 2 4;}@font-face
{font-family:"Times New Roman \(Body CS\)";
panose-1:2 11 6 4 2 2 2 2 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:12.0pt;
font-family:"Aptos",sans-serif;}a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Aptos",sans-serif;
color:windowtext;}.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}div.WordSection1
{page:WordSection1;}</style>
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:14.0pt">Hi Egor,<br>
<br>
Thank you for reporting this scalability issue when debugger
is enabled.<br>
It looks like a JVMTI problem and we have some guesses but
need some investigation to identify it better.<br>
<br>
Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt">Serguei<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt"><o:p> </o:p></span></p>
<div id="mail-editor-reference-message-container">
<div>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="color:black">From:
</span></b><span style="color:black">serviceability-dev
<a class="moz-txt-link-rfc2396E" href="mailto:serviceability-dev-retn@openjdk.org"><serviceability-dev-retn@openjdk.org></a> on
behalf of Egor Ushakov
<a class="moz-txt-link-rfc2396E" href="mailto:egor.ushakov@jetbrains.com"><egor.ushakov@jetbrains.com></a><br>
<b>Date: </b>Wednesday, April 2, 2025 at 3:32</span><span style="font-family:"Arial",sans-serif;color:black"> </span><span style="color:black">AM<br>
<b>To: </b>Chris Plummer
<a class="moz-txt-link-rfc2396E" href="mailto:chris.plummer@oracle.com"><chris.plummer@oracle.com></a>, serviceability-dev
<a class="moz-txt-link-rfc2396E" href="mailto:serviceability-dev@openjdk.java.net"><serviceability-dev@openjdk.java.net></a><br>
<b>Subject: </b>Re: Debugger overhead for virtual
threads creation<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:11.0pt">Thanks Chris!<br>
<br>
I've made the bug <a href="https://youtrack.jetbrains.com/issue/IDEA-365900" moz-do-not-send="true" class="moz-txt-link-freetext">https://youtrack.jetbrains.com/issue/IDEA-365900</a>
<br>
visible, there's a reproducer there.<br>
<br>
Thanks,<br>
Egor<br>
<br>
On 02.04.2025 01:39, Chris Plummer wrote:<br>
> The short answer is yes. The debug agent needs
to deal with <br>
> JVMTI_EVENT_VIRTUAL_THREAD_START/END events for
every virtual thread. <br>
> What makes it worse is when there are a large
number of virtual <br>
> threads that are currently alive. They are
tracked on a list of <br>
> ThreadNodes that starts to slow down debug
agent performance when it <br>
> gets too long. I have a work in progress that
proactively purges these <br>
> ThreadNodes so the list does not get too
big. I've been meaning to <br>
> revive this project for quite some time. If you
have a test case I'd <br>
> be willing to experiment with these changes
some more. I could not <br>
> access to the IDEA-365900 link you provided.<br>
><br>
> Note I think after the work is done to purge
ThreadNodes proactively <br>
> it might not be that hard of step to move to
not needing <br>
> JVMTI_EVENT_VIRTUAL_THREAD_START/END events
enabled, which will help <br>
> performance a lot more.<br>
><br>
> Chris<br>
><br>
> On 4/1/25 10:14 AM, Egor Ushakov wrote:<br>
>> Hi everyone!<br>
>><br>
>> Is it expected that with the debugger
attached creating virtual <br>
>> threads is much slower?<br>
>> We're getting bugs like: <br>
>> <a href="https://youtrack.jetbrains.com/issue/IDEA-365900" moz-do-not-send="true" class="moz-txt-link-freetext">https://youtrack.jetbrains.com/issue/IDEA-365900</a><br>
>> And I can reproduce it easily with jdb...<br>
>> Just attaching the debugger immediately
slows down virtual threads <br>
>> creation significantly.<br>
>><br>
>> >java <br>
>>
-agentlib:jdwp=transport=dt_shmem,server=y,suspend=n,address=8000
app<br>
>> ...<br>
>> 6808805 (1.2046688E7 threads per second)<br>
>> ...<br>
>> after >jdb -attach 8000<br>
>> ...<br>
>> 30215 (95986.055 threads per second)<br>
>> ...<br>
>><br>
>> Thanks,<br>
>> Egor<o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</body>
</html>