Issue w/ Recursive Virtual Thread Spawning

[Ron Pressler]

Yes, but probably not this year. However, j.u.c locks do not have this limitation and I was told of an effort to employ those locks in Clojure’s LazySeq instead of synchronized. You should ask about it on a Clojure forum.

— Ron

On 19 Apr 2023, at 17:54, Ryan Schmukler <ryan at teknql.com<mailto:ryan at teknql.com>> wrote:

Thanks for the quick response! Is this limitation something that is planned to be changed in the future? If so, is there any estimate as to when? ie. this year, next year, some day, etc.

Cheers again!
[https://ssl.gstatic.com/ui/v1/icons/mail/images/cleardot.gif]

On Wed, Apr 19, 2023 at 9:34 AM Alan Bateman <Alan.Bateman at oracle.com<mailto:Alan.Bateman at oracle.com>> wrote:
On 19/04/2023 15:38, Ryan Schmukler wrote:
> Hello,
>
> I am not sure if this is the appropriate venue or if I should be
> reporting bugs via the Oracle bug page. Apologies if this is
> incorrect; this is also my first mailing list.
>
> I am the author of a library that brings Loom related functionality
> into Clojure: https://github.com/teknql/tapestry
>
> I recently stumbled upon an interesting behavior where recursively
> spawning fibers can cause the program to block infinitely. The
> (clojure) code that can reproduce this can be seen at this gist:
> https://gist.github.com/rschmukler/314b786206246c906caa6a2f99d731a7
>
> Basically if you have 4 virtual threads that recursively spawn 4
> virtual threads, which recursively spawn 4  virtual threads, which
> recursively spawn 4  virtual threads, you will hit some sort of
> blocking issue and your code will freeze. 4^4 = 256  virtual threads
> which I think should be very doable for Loom.
>

There is currently a scalability issue / limitation when parking while
holding a monitor. The limitation is that parking the virtual thread
doesn't release the underlying carrier thread to do other work. There is
a diagnostic option and a JFR event to help identify these issues. You
can read more about this issue and the diagnostic options in JEP 444. I
did a quick run of your test with -Djdk.tracePinnedThreads=full and it
produces the output beyond. Looking for "<== monitors" in the output as
it seems the lazy-seq methods are synchronized.

Thread[#51,ForkJoinPool-1-worker-10,5,CarrierThreads]
java.base/java.lang.VirtualThread$VThreadContinuation.onPinned(VirtualThread.java:185)
java.base/jdk.internal.vm.Continuation.onPinned0(Continuation.java:393)
     java.base/java.lang.VirtualThread.park(VirtualThread.java:595)
     java.base/java.lang.System$2.parkVirtualThread(System.java:2620)
java.base/jdk.internal.misc.VirtualThreads.park(VirtualThreads.java:54)
java.base/java.util.concurrent.locks.LockSupport.park(LockSupport.java:219)
java.base/java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1864)
java.base/java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3780)
java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3725)
java.base/java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1898)
java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2072)
     clojure.core$deref_future.invokeStatic(core.clj:2304)
     clojure.core$deref.invokeStatic(core.clj:2324)
     clojure.core$deref.invoke(core.clj:2310)
     clojure.core$map$fn__5884.invoke(core.clj:2757)
     clojure.lang.LazySeq.sval(LazySeq.java:42) <== monitors:1
     clojure.lang.LazySeq.seq(LazySeq.java:51) <== monitors:1
     clojure.lang.RT.seq(RT.java:535)
     clojure.core$seq__5419.invokeStatic(core.clj:139)
     clojure.core$apply.invokeStatic(core.clj:662)
     clojure.core$apply.invoke(core.clj:662)
     user$eval141$fn__142$fn__143.invoke(NO_SOURCE_FILE:10)
     user$ppmap$fn__136$fn__137.invoke(NO_SOURCE_FILE:10)
     clojure.lang.AFn.applyToHelper(AFn.java:152)
     clojure.lang.AFn.applyTo(AFn.java:144)
     clojure.core$apply.invokeStatic(core.clj:667)
     clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1977)
     clojure.core$with_bindings_STAR_.doInvoke(core.clj:1977)
     clojure.lang.RestFn.invoke(RestFn.java:425)
     clojure.lang.AFn.applyToHelper(AFn.java:156)
     clojure.lang.RestFn.applyTo(RestFn.java:132)
     clojure.core$apply.invokeStatic(core.clj:671)
     clojure.core$bound_fn_STAR_$fn__5767.doInvoke(core.clj:2007)
     clojure.lang.RestFn.invoke(RestFn.java:397)
     clojure.lang.AFn.run(AFn.java:22)
     java.base/java.lang.VirtualThread.run(VirtualThread.java:314)

The only way to workaround this right now is to change this code to use
j.u.concurrent locks.

-Alan


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20230419/bba42d34/attachment.htm>