<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Hi.
<div class=""><br class="">
</div>
<div class="">Let me first address some of the topics that have come up in this thread before I get to the main point.</div>
<div class=""><br class="">
</div>
<div class="">The meaning of Thread.yield is “inform the runtime that this thread doesn’t have anything useful to do at the moment, but it might again in the future”. What the runtime does with that information is up to the runtime. As Alan said, in JDK 20
we’ve changed the implementation to have the scheduler spend more time to more aggressively look for other work to do, but Thread.yield shouldn’t be used for scheduling control. It should be used (and very sparingly) to say “I temporarily have nothing useful
to do.”</div>
<div class=""><br class="">
</div>
<div class="">Virtual threads are scheduled preemptively, not cooperatively. This means that the runtime makes the decision when to deschedule (preempt) one thread and schedule another without cooperation from user code. However, the virtual thread scheduler
currently does not employ time-sharing, i.e. it does not decide to preempt a thread based on it exceeding some allotted time-slice quota on the CPU. The reason for that is we haven’t yet identified a use-case where time-sharing can help for the use-cases virtual
threads address (although we’re very interested to hear about such use-cases if anyone comes across one). Virtual threads are mostly intended to write servers, non-realtime kernels primarily employ time-sharing when the CPU is at 100%, servers don’t usually
run at 100% CPU and when they do people aren’t generally happy with the result. So servers don’t rely on time-sharing even without virtual threads (at least not the kind that requires special support), but when we identify a use case where time-sharing could
help server workloads we can consider adding it to the preemption considerations.</div>
<div class=""><br class="">
</div>
<div class="">Now for the central point. Virtual threads have scalability benefits for a single reason: their high number. This is due to Little’s law. Replacing a set of platform threads with virtual threads — as the JEP explains — should not yield any significant
benefits. The benefits come when employing a very high number of virtual threads. For example, the Helidon framework creates 3 million new virtual threads every seconds under high load. As a rule of thumb, if your application doesn’t create a few thousand
or so brand-new virtual threads every second, then you’re not using them in the manner for which they were intended and will not see substantial benefits. Virtual threads replace short individual *tasks* in your application, not platform threads. They are
best thought of as a business logic entity representing a task rather than an “execution resource.” For those who are used to managing threads as resources, this is a big change in how they think about threads, and will take some getting used to.</div>
<div class=""><br class="">
</div>
<div class="">— Ron<br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On 27 Dec 2022, at 22:27, robert engels <<a href="mailto:rengels@ix.netcom.com" class="">rengels@ix.netcom.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Hi devs,
<div class=""><br class="">
</div>
<div class="">First, <br class="">
<div class=""><br class="">
</div>
<span style="caret-color: rgb(0, 0, 0);" class="">Thanks for this amazing work!!! It literally solves the only remaining problem Java had.</span></div>
<div class=""><span style="caret-color: rgb(0, 0, 0);" class=""><br class="">
</span></div>
<div class=""><span style="caret-color: rgb(0, 0, 0);" class="">Sorry for the long email.</span></div>
<div class=""><font class=""><span style="caret-color: rgb(0, 0, 0);" class=""><br class="">
</span></font>
<div class="">I have been very excited to test-drive Project Loom in JDK19. I have extensive experience in highly concurrent systems/HFT/HPC, so I usually :) know what I am doing.</div>
<div class=""><br class="">
</div>
<div class="">For the easiest test, I took a highly threaded (connection based) server based system (Java port of Go’s
<a href="http://nats.io/" class="">nats.io</a> message broker), and converted the threads to virtual threads. The project (jnatsd) is available <a href="https://github.com/robaho/jnatsd" class="">here</a>. The ‘master’ branch runs very well with excellent performance,
but I thought switching to virtual threads might be able to improve things over using async IO, channels, etc. (I have a branch for this that works as well, but it is much more complex, and didn’t provide a huge performance benefit)/</div>
<div class=""><br class="">
</div>
<div class="">There are two branches ’simple_virtual_threads’ and ‘virtual_threads’.</div>
<div class=""><br class="">
</div>
<div class="">In the former, it is literally a 2 line change to enable the virtual threads but it doesn’t work. I narrowed it down the issue that LockSupport.unpark(thread) does not work consistently. At some point, the virtual thread is never scheduled again.
I enabled the debug options and I see that the the virtual thread is in:</div>
<div class=""><br class="">
</div>
<div class="">
<pre class="">yield0:365, Continuation (jdk.internal.vm)
yield:357, Continuation (jdk.internal.vm)
yieldContinuation:370, VirtualThread (java.lang)
park:499, VirtualThread (java.lang)
parkVirtualThread:2606, System$2 (java.lang)
park:54, VirtualThreads (jdk.internal.misc)
park:369, LockSupport (java.util.concurrent.locks)
run:88, Connection$ConnectionWriter (com.robaho.jnatsd)
run:287, VirtualThread (java.lang)
lambda$new$0:174, VirtualThread$VThreadContinuation (java.lang)
run:-1, VirtualThread$VThreadContinuation$$Lambda$50/0x0000000801065670 (java.lang)
enter0:327, Continuation (jdk.internal.vm)
enter:320, Continuation (jdk.internal.vm)
</pre>
<div class="">The instance state is:</div>
</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">this = {VirtualThread$VThreadContinuation@1775} </div>
<div class=""> target = {VirtualThread$VThreadContinuation$lambda@1777} </div>
<div class=""> arg$1 = {VirtualThread@1699} </div>
<div class=""> scheduler = {ForkJoinPool@1781} </div>
<div class=""> cont = {VirtualThread$VThreadContinuation@1775} </div>
<div class=""> runContinuation = {VirtualThread$lambda@1782} </div>
<div class=""> state = 2</div>
<div class=""> parkPermit = true</div>
<div class=""> carrierThread = null</div>
<div class=""> termination = null</div>
<div class=""> eetop = 0</div>
<div class=""> tid = 76</div>
<div class=""> name = ""</div>
<div class=""> interrupted = false</div>
<div class=""> contextClassLoader = {ClassLoaders$AppClassLoader@1784} </div>
<div class=""> inheritedAccessControlContext = {AccessControlContext@1785} </div>
<div class=""> holder = null</div>
<div class=""> threadLocals = null</div>
<div class=""> inheritableThreadLocals = null</div>
<div class=""> extentLocalBindings = null</div>
<div class=""> interruptLock = {Object@1786} </div>
<div class=""> parkBlocker = null</div>
<div class=""> nioBlocker = null</div>
<div class=""> Thread.cont = null</div>
<div class=""> uncaughtExceptionHandler = null</div>
<div class=""> threadLocalRandomSeed = 0</div>
<div class=""> threadLocalRandomProbe = 0</div>
<div class=""> threadLocalRandomSecondarySeed = 0</div>
<div class=""> container = {ThreadContainers$RootContainer$CountingRootContainer@1787} </div>
<div class=""> headStackableScopes = null</div>
<div class=""> arg$2 = {Connection$ConnectionWriter@1780} </div>
<div class=""> scope = {ContinuationScope@1776} </div>
<div class=""> parent = null</div>
<div class=""> child = null</div>
<div class=""> tail = {StackChunk@1778} </div>
<div class=""> done = false</div>
<div class=""> mounted = false</div>
<div class=""> yieldInfo = null</div>
<div class=""> preempted = false</div>
<div class=""> extentLocalCache = null</div>
<div class="">scope = {ContinuationScope@1776} </div>
<div class="">child = null</div>
</div>
<div class=""><br class="">
</div>
<div class="">As you see in the above, the parkPermit is true, but it never runs again.</div>
<div class=""><br class="">
</div>
<div class="">In the latter branch, ‘virtual_threads’, I changed the lock-free RingBuffer class to use simple synchronized primitives - under the assumption that with virtual threads lock/wait/notify should be highly efficient. It worked, but it was nearly
2x slower than the original thread based lock-free implementation. So, I added a ’spin loop’ in the RingBuffer methods. This code is completely optional and can be no-op’d, and I was able to increase performance to above that of the Thread based version.</div>
<div class=""><br class="">
</div>
<div class="">I dug a little deeper, and decided that using Thread.yield() should be even more efficient than LockSupport.parkNanos(1) - problem is that changing that simple line brings back the hangs. I think there is very little semantic difference between
LockSupport.parkNanos(1) and Thread.yield() but the latter should avoid any timer scheduling. The RingBuffer code there is fairly trivial.</div>
<div class=""><br class="">
</div>
<div class="">So, before I dig deeper, is this a known issue that Thread.yield() does not work as expected? Is it is known issue that LockSupport.unpark() fails to reschedule threads?</div>
<div class=""><br class="">
</div>
<div class="">Is it possible because the VirtualThreads do not implement the Java memory model properly?</div>
<div class=""><br class="">
</div>
<div class="">Any ideas how to further diagnose?</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>