<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Further diagnosis seems to show that virtual threads are not preemptible - and it seems that the fork-join pool is not stealing work, so if one thread spins - all other threads will be blocked.<div class=""><br class=""></div><div class="">Does this sound reasonable? If so, this seems like a significant limitation which will cause all sorts of spin/lock-free code to fail under virtual threads.<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Dec 27, 2022, at 4:27 PM, robert engels <<a href="mailto:rengels@ix.netcom.com" class="">rengels@ix.netcom.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi devs,<div class=""><br class=""></div><div class="">First, <br class=""><div class=""><br class=""></div><span style="caret-color: rgb(0, 0, 0);" class="">Thanks for this amazing work!!! It literally solves the only remaining problem Java had.</span></div><div class=""><span style="caret-color: rgb(0, 0, 0);" class=""><br class=""></span></div><div class=""><span style="caret-color: rgb(0, 0, 0);" class="">Sorry for the long email.</span></div><div class=""><font class=""><span style="caret-color: rgb(0, 0, 0);" class=""><br class=""></span></font><div class="">I have been very excited to test-drive Project Loom in JDK19. I have extensive experience in highly concurrent systems/HFT/HPC, so I usually :) know what I am doing.</div><div class=""><br class=""></div><div class="">For the easiest test, I took a highly threaded (connection based) server based system (Java port of Go’s <a href="http://nats.io/" class="">nats.io</a> message broker), and converted the threads to virtual threads. The project (jnatsd) is available <a href="https://github.com/robaho/jnatsd" class="">here</a>. The ‘master’ branch runs very well with excellent performance, but I thought switching to virtual threads might be able to improve things over using async IO, channels, etc. (I have a branch for this that works as well, but it is much more complex, and didn’t provide a huge performance benefit)/</div><div class=""><br class=""></div><div class="">There are two branches ’simple_virtual_threads’ and ‘virtual_threads’.</div><div class=""><br class=""></div><div class="">In the former, it is literally a 2 line change to enable the virtual threads but it doesn’t work. I narrowed it down the issue that LockSupport.unpark(thread) does not work consistently. At some point, the virtual thread is never scheduled again. I enabled the debug options and I see that the the virtual thread is in:</div><div class=""><br class=""></div><div class=""><pre class="">yield0:365, Continuation (jdk.internal.vm)
yield:357, Continuation (jdk.internal.vm)
yieldContinuation:370, VirtualThread (java.lang)
park:499, VirtualThread (java.lang)
parkVirtualThread:2606, System$2 (java.lang)
park:54, VirtualThreads (jdk.internal.misc)
park:369, LockSupport (java.util.concurrent.locks)
run:88, Connection$ConnectionWriter (com.robaho.jnatsd)
run:287, VirtualThread (java.lang)
lambda$new$0:174, VirtualThread$VThreadContinuation (java.lang)
run:-1, VirtualThread$VThreadContinuation$$Lambda$50/0x0000000801065670 (java.lang)
enter0:327, Continuation (jdk.internal.vm)
enter:320, Continuation (jdk.internal.vm)
</pre><div class="">The instance state is:</div></div><div class=""><br class=""></div><div class=""><div class="">this = {VirtualThread$VThreadContinuation@1775} </div><div class=""> target = {VirtualThread$VThreadContinuation$lambda@1777} </div><div class=""> arg$1 = {VirtualThread@1699} </div><div class=""> scheduler = {ForkJoinPool@1781} </div><div class=""> cont = {VirtualThread$VThreadContinuation@1775} </div><div class=""> runContinuation = {VirtualThread$lambda@1782} </div><div class=""> state = 2</div><div class=""> parkPermit = true</div><div class=""> carrierThread = null</div><div class=""> termination = null</div><div class=""> eetop = 0</div><div class=""> tid = 76</div><div class=""> name = ""</div><div class=""> interrupted = false</div><div class=""> contextClassLoader = {ClassLoaders$AppClassLoader@1784} </div><div class=""> inheritedAccessControlContext = {AccessControlContext@1785} </div><div class=""> holder = null</div><div class=""> threadLocals = null</div><div class=""> inheritableThreadLocals = null</div><div class=""> extentLocalBindings = null</div><div class=""> interruptLock = {Object@1786} </div><div class=""> parkBlocker = null</div><div class=""> nioBlocker = null</div><div class=""> Thread.cont = null</div><div class=""> uncaughtExceptionHandler = null</div><div class=""> threadLocalRandomSeed = 0</div><div class=""> threadLocalRandomProbe = 0</div><div class=""> threadLocalRandomSecondarySeed = 0</div><div class=""> container = {ThreadContainers$RootContainer$CountingRootContainer@1787} </div><div class=""> headStackableScopes = null</div><div class=""> arg$2 = {Connection$ConnectionWriter@1780} </div><div class=""> scope = {ContinuationScope@1776} </div><div class=""> parent = null</div><div class=""> child = null</div><div class=""> tail = {StackChunk@1778} </div><div class=""> done = false</div><div class=""> mounted = false</div><div class=""> yieldInfo = null</div><div class=""> preempted = false</div><div class=""> extentLocalCache = null</div><div class="">scope = {ContinuationScope@1776} </div><div class="">child = null</div></div><div class=""><br class=""></div><div class="">As you see in the above, the parkPermit is true, but it never runs again.</div><div class=""><br class=""></div><div class="">In the latter branch, ‘virtual_threads’, I changed the lock-free RingBuffer class to use simple synchronized primitives - under the assumption that with virtual threads lock/wait/notify should be highly efficient. It worked, but it was nearly 2x slower than the original thread based lock-free implementation. So, I added a ’spin loop’ in the RingBuffer methods. This code is completely optional and can be no-op’d, and I was able to increase performance to above that of the Thread based version.</div><div class=""><br class=""></div><div class="">I dug a little deeper, and decided that using Thread.yield() should be even more efficient than LockSupport.parkNanos(1) - problem is that changing that simple line brings back the hangs. I think there is very little semantic difference between LockSupport.parkNanos(1) and Thread.yield() but the latter should avoid any timer scheduling. The RingBuffer code there is fairly trivial.</div><div class=""><br class=""></div><div class="">So, before I dig deeper, is this a known issue that Thread.yield() does not work as expected? Is it is known issue that LockSupport.unpark() fails to reschedule threads?</div><div class=""><br class=""></div><div class="">Is it possible because the VirtualThreads do not implement the Java memory model properly?</div><div class=""><br class=""></div><div class="">Any ideas how to further diagnose?</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div></div></div></div></blockquote></div><br class=""></div></body></html>