Issue w/ Recursive Virtual Thread Spawning

[Alan Bateman]

On 19/04/2023 15:38, Ryan Schmukler wrote:
> Hello,
>
> I am not sure if this is the appropriate venue or if I should be 
> reporting bugs via the Oracle bug page. Apologies if this is 
> incorrect; this is also my first mailing list.
>
> I am the author of a library that brings Loom related functionality 
> into Clojure: https://github.com/teknql/tapestry
>
> I recently stumbled upon an interesting behavior where recursively 
> spawning fibers can cause the program to block infinitely. The 
> (clojure) code that can reproduce this can be seen at this gist: 
> https://gist.github.com/rschmukler/314b786206246c906caa6a2f99d731a7
>
> Basically if you have 4 virtual threads that recursively spawn 4  
> virtual threads, which recursively spawn 4  virtual threads, which 
> recursively spawn 4  virtual threads, you will hit some sort of 
> blocking issue and your code will freeze. 4^4 = 256  virtual threads 
> which I think should be very doable for Loom.
>

There is currently a scalability issue / limitation when parking while 
holding a monitor. The limitation is that parking the virtual thread 
doesn't release the underlying carrier thread to do other work. There is 
a diagnostic option and a JFR event to help identify these issues. You 
can read more about this issue and the diagnostic options in JEP 444. I 
did a quick run of your test with -Djdk.tracePinnedThreads=full and it 
produces the output beyond. Looking for "<== monitors" in the output as 
it seems the lazy-seq methods are synchronized.

Thread[#51,ForkJoinPool-1-worker-10,5,CarrierThreads]
java.base/java.lang.VirtualThread$VThreadContinuation.onPinned(VirtualThread.java:185)
java.base/jdk.internal.vm.Continuation.onPinned0(Continuation.java:393)
     java.base/java.lang.VirtualThread.park(VirtualThread.java:595)
     java.base/java.lang.System$2.parkVirtualThread(System.java:2620)
java.base/jdk.internal.misc.VirtualThreads.park(VirtualThreads.java:54)
java.base/java.util.concurrent.locks.LockSupport.park(LockSupport.java:219)
java.base/java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1864)
java.base/java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3780)
java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3725)
java.base/java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1898)
java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2072)
     clojure.core$deref_future.invokeStatic(core.clj:2304)
     clojure.core$deref.invokeStatic(core.clj:2324)
     clojure.core$deref.invoke(core.clj:2310)
     clojure.core$map$fn__5884.invoke(core.clj:2757)
     clojure.lang.LazySeq.sval(LazySeq.java:42) <== monitors:1
     clojure.lang.LazySeq.seq(LazySeq.java:51) <== monitors:1
     clojure.lang.RT.seq(RT.java:535)
     clojure.core$seq__5419.invokeStatic(core.clj:139)
     clojure.core$apply.invokeStatic(core.clj:662)
     clojure.core$apply.invoke(core.clj:662)
     user$eval141$fn__142$fn__143.invoke(NO_SOURCE_FILE:10)
     user$ppmap$fn__136$fn__137.invoke(NO_SOURCE_FILE:10)
     clojure.lang.AFn.applyToHelper(AFn.java:152)
     clojure.lang.AFn.applyTo(AFn.java:144)
     clojure.core$apply.invokeStatic(core.clj:667)
     clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1977)
     clojure.core$with_bindings_STAR_.doInvoke(core.clj:1977)
     clojure.lang.RestFn.invoke(RestFn.java:425)
     clojure.lang.AFn.applyToHelper(AFn.java:156)
     clojure.lang.RestFn.applyTo(RestFn.java:132)
     clojure.core$apply.invokeStatic(core.clj:671)
     clojure.core$bound_fn_STAR_$fn__5767.doInvoke(core.clj:2007)
     clojure.lang.RestFn.invoke(RestFn.java:397)
     clojure.lang.AFn.run(AFn.java:22)
     java.base/java.lang.VirtualThread.run(VirtualThread.java:314)

The only way to workaround this right now is to change this code to use 
j.u.concurrent locks.

-Alan