Debugger overhead for virtual threads creation

[Chris Plummer]

I got my prototype ThreadNode cleanup code working again. It seems to be 
doing the job of keeping the list down to a minimal size, with usually 
at most a few ThreadNodes on the list. However, that didn't help 
performance. The reason is because the debug agent normally does not 
need to traverse the list. It maps jthread to ThreadNode by using JVMTI 
thread local data, not by searching the list, and the list is doubly 
linked, so ThreadNodes can be removed quickly.

After the above results I talked with Serguei and he confirmed that 
JVMTI also has a linked list, and there are some occasions where it 
needs to be traversed, so that likely the bottleneck here.

Regarding my suggestion that we may be able to disable 
JVMTI_EVENT_VIRTUAL_THREAD_START/END, I did experiment with that and 
when disabled there are no performance issues, so I will look into 
getting this to work properly. The events need to be enable if there are 
any ThreadStartRequests or ThreadDeathRequests that do not have the 
PlatformThreadOnly filter enabled.

https://docs.oracle.com/en/java/javase/24/docs/api/jdk.jdi/com/sun/jdi/request/ThreadStartRequest.html#addPlatformThreadsOnlyFilter()

Jdb by default uses this filter, and I suspect that IDEA does also. If 
not, it would get a flood of ThreadStart and ThreadDeath events for all 
the virtual threads.

Chris

On 4/3/25 2:15 AM, Serguei Spitsyn wrote:
>
> Hi Egor,
>
> Thank you for reporting this scalability issue when debugger is enabled.
> It looks like a JVMTI problem and we have some guesses but need some 
> investigation to identify it better.
>
> Thanks,
>
> Serguei
>
> *From: *serviceability-dev <serviceability-dev-retn at openjdk.org> on 
> behalf of Egor Ushakov <egor.ushakov at jetbrains.com>
> *Date: *Wednesday, April 2, 2025 at 3:32 AM
> *To: *Chris Plummer <chris.plummer at oracle.com>, serviceability-dev 
> <serviceability-dev at openjdk.java.net>
> *Subject: *Re: Debugger overhead for virtual threads creation
>
> Thanks Chris!
>
> I've made the bug https://youtrack.jetbrains.com/issue/IDEA-365900
> visible, there's a reproducer there.
>
> Thanks,
> Egor
>
> On 02.04.2025 01:39, Chris Plummer wrote:
> > The short answer is yes. The debug agent needs to deal with
> > JVMTI_EVENT_VIRTUAL_THREAD_START/END events for every virtual thread.
> > What makes it worse is when there are a large number of virtual
> > threads that are currently alive. They are tracked on a list of
> > ThreadNodes that starts to slow down debug agent performance when it
> > gets too long. I have a work in progress that proactively purges these
> > ThreadNodes so the list does not get too big. I've been meaning to
> > revive this project for quite some time. If you have a test case I'd
> > be willing to experiment with these changes some more. I could not
> > access to the IDEA-365900 link you provided.
> >
> > Note I think after the work is done to purge ThreadNodes proactively
> > it might not be that hard of step to move to not needing
> > JVMTI_EVENT_VIRTUAL_THREAD_START/END events enabled, which will help
> > performance a lot more.
> >
> > Chris
> >
> > On 4/1/25 10:14 AM, Egor Ushakov wrote:
> >> Hi everyone!
> >>
> >> Is it expected that with the debugger attached creating virtual
> >> threads is much slower?
> >> We're getting bugs like:
> >> https://youtrack.jetbrains.com/issue/IDEA-365900
> >> And I can reproduce it easily with jdb...
> >> Just attaching the debugger immediately slows down virtual threads
> >> creation significantly.
> >>
> >> >java
> >> -agentlib:jdwp=transport=dt_shmem,server=y,suspend=n,address=8000 app
> >> ...
> >> 6808805 (1.2046688E7 threads per second)
> >> ...
> >> after >jdb -attach 8000
> >> ...
> >> 30215 (95986.055 threads per second)
> >> ...
> >>
> >> Thanks,
> >> Egor
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/serviceability-dev/attachments/20250403/85e99948/attachment.htm>