Techniques for debugging threading issues with Virtual Threads

[Dr Heinz M. Kabutz]

On 2024-11-20 09:22, Alan Bateman wrote:
>
>
> On 20/11/2024 07:10, Dr Heinz M. Kabutz wrote:
>> With platform threads, we have had good techniques for finding 
>> threading issues. For example, a thread dump would reveal deadlocks, 
>> livelocks and contention.
>>
>> However, with virtual threads, most of these techniques won't work 
>> anymore. For example, here is a SimpleDeadlock:
>>
>> import java.util.concurrent.locks.*;
>>
>> public class SimpleDeadlock {
>>     public static void main(String... args) throws 
>> InterruptedException {
>>         var monitor1 = new Object();
>>         var monitor2 = new Object();
>>         Thread.startVirtualThread(() -> {
>>             synchronized (monitor1) {
>>                 LockSupport.parkNanos(10_000_000);
>>                 synchronized (monitor2) {
>>                     System.out.println("Got both locks");
>>                 }
>>             }
>>         });
>>         Thread.startVirtualThread(() -> {
>>             synchronized (monitor2) {
>>                 LockSupport.parkNanos(10_000_000);
>>                 synchronized (monitor1) {
>>                     System.out.println("Got both locks");
>>                 }
>>             }
>>         }).join();
>>     }
>> }
>>
>> If we run this with Java 21-23 and -Djdk.trackAllThreads=false, the 
>> carrier thread indicate that they are carrying some virtual threads, 
>> but these do not appear in the full dump. In Java 24+24, the virtual 
>> threads are parked instead of pinned, and thus they vanish completely 
>> from the thread dumps.
>>
>> Is there any project or workgroup where debugging of virtual threads 
>> is being looked at?
>
> The HotSpot VM thread dump will show mounted virtual threads but not 
> unmounted virtual threads. So yes, fixing the pinning issue with 
> object monitors has an impact on thread dumps because the virtual 
> threads unmount. The thread dump taken with jcmd Thread.dump_to_file 
> shows all threads, including all unmounted virtual threads. The 
> intention is that will be expanded to include information on monitors 
> and ownable synchronizers, still some work required on that. Having a 
> parsing thread dump with lock information will allow for more 
> diagnostic tooling, including deadlock detection.
>
> -Alan

It only shows the unmounted virtual threads if we have 
-Djdk.trackAllThreads=true, which is fortunately on by default. However, 
if it is set to false, under Java 21 we would see the HotSpot VM Thread 
Dump of:

"ForkJoinPool-1-worker-1" #21 [26627] daemon prio=5 os_prio=31 
cpu=23.38ms elapsed=1957.40s tid=0x0000000127010a00 [0x0000000172006000]
    Carrying virtual thread #20
         at 
jdk.internal.vm.Continuation.run(java.base at 21.0.5/Continuation.java:248)
         at 
java.lang.VirtualThread.runContinuation(java.base at 21.0.5/VirtualThread.java:245)
         ...

"ForkJoinPool-1-worker-2" #23 [27139] daemon prio=5 os_prio=31 
cpu=21.36ms elapsed=1957.41s tid=0x000000012500c000 [0x0000000172212000]
    Carrying virtual thread #22
         at 
jdk.internal.vm.Continuation.run(java.base at 21.0.5/Continuation.java:248)
         at 
java.lang.VirtualThread.runContinuation(java.base at 21.0.5/VirtualThread.java:245)
         ...

However, virtual threads #20 and #22 do not appear in dump_to_file with 
trackAllThreads=false.

In Java 24, we see the HotSpot VM Thread Dump of:

"ForkJoinPool-1-worker-1" #24 [32771] daemon prio=5 os_prio=31 
cpu=0.85ms elapsed=6.92s tid=0x000000014a00e400 nid=32771 waiting on 
condition  [0x0000000175e4e000]
    java.lang.Thread.State: WAITING (parking)
         at jdk.internal.misc.Unsafe.park(java.base at 24-ea/Native Method)
         - parking to wait for  <0x00000003600580c8> (a 
java.util.concurrent.ForkJoinPool)
        ...

"ForkJoinPool-1-worker-2" #26 [43011] daemon prio=5 os_prio=31 
cpu=0.67ms elapsed=6.92s tid=0x000000014a024400 nid=43011 waiting on 
condition  [0x000000017605a000]
    java.lang.Thread.State: TIMED_WAITING (parking)
         at jdk.internal.misc.Unsafe.park(java.base at 24-ea/Native Method)
         - parking to wait for  <0x00000003600580c8> (a 
java.util.concurrent.ForkJoinPool)
        ...

The virtual threads don't appear to be GCed, but they also vanish from 
the detailed dump if we have -Djdk.trackAllThreads=false.

Is there any good use case for setting this to false?


Heinz