<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Roboto;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:276134469;
mso-list-template-ids:-592543672;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Times New Roman";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1
{mso-list-id:886530423;
mso-list-template-ids:-1018371100;}
@list l1:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l1:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Times New Roman";}
@list l1:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-CA link=blue vlink=purple style='word-wrap:break-word'><div class=WordSection1><p class=MsoNormal><span style='mso-fareast-language:EN-US'>“</span>Writing the entire pipeline with simple blocking in mind gave us not only superior performance, but a much smaller and simpler codebase, and that would be the approach I’d recommend.”<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><b><span style='mso-fareast-language:EN-US'>Ron</span></b><span style='mso-fareast-language:EN-US'>, are there any projects you are aware of that will do this? Such as a Loom-based HTTP Server and Client library, something like Jetty, but completely based on Virtual Threads, and maybe some Structured Concurrency?<br><br>Cheers, Eric<o:p></o:p></span></p><p class=MsoNormal><span style='mso-fareast-language:EN-US'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span lang=EN-US>From:</span></b><span lang=EN-US> loom-dev <loom-dev-retn@openjdk.org> <b>On Behalf Of </b>Ron Pressler<br><b>Sent:</b> July 26, 2022 8:11 AM<br><b>To:</b> Clement Escoffier <clement.escoffier@redhat.com><br><b>Cc:</b> loom-dev@openjdk.java.net<br><b>Subject:</b> Re: Virtual Threads support in Quarkus - current integration and ideal<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Hi and thank you very much for your report! <o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>It has been our experience as well that trying to marry an asynchronous engine with virtual threads is cumbersome and often wasteful. Writing the entire pipeline with simple blocking in mind gave us not only superior performance, but a much smaller and simpler codebase, and that would be the approach I’d recommend. I expect that there will soon be HTTP servers demonstrating that simple approach. However, if you wish to use an existing async engine, I think the approach you’ve taken — spawning/unblocking a virtual thread running in the virtual thread scheduler — is probably the best one.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Integrating explicit scheduler loops with virtual thread via custom schedulers is on the roadmap, but, encouraged by the performance of servers that go the “full simple” approach, this might not be a top priority and might take some time [1]. The API was removed for the simple reason that it’s just not ready, as you noticed.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>As for memory footprint, although this might not be the cause of your issue, it might interest you to know that we’re now working on dramatically reducing the footprint of virtual thread stacks. That work also wasn’t ready for 19, but is a higher priority than custom schedulers. So I’m interested to know how much of that excess footprint is due to virtual thread stacks (those would appear as jdk.internal.vm.StackChunk objects in your heap).<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>What I’d like to hear more about is pinning, and what common causes of it you see. I would also be interested to hear your thoughts about how much of it is due to ecosystem readiness (e.g. some JDBC drivers don’t pin while others still do, although that’s expected to change).<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>— Ron<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>[1]: The “mechanical sympathy” effects you alluded to are real but too small in comparison to the throughput increase of thread-per-request code for them to be an immediate focus, especially as a work-stealing scheduler has pretty decent mechanical sympathy already. On the other hand, there are other reasons to support custom schedulers (e.g. UI event threads) that might shift the priority balance.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal><br><br><o:p></o:p></p><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><div><p class=MsoNormal>On 26 Jul 2022, at 15:13, Clement Escoffier <<a href="mailto:clement.escoffier@redhat.com">clement.escoffier@redhat.com</a>> wrote:<o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p><div><div><div><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>Hello,</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>This email reports our observations around Loom, mainly in the context of Quarkus. It discusses the current approach and our plans. </span><span style='font-size:10.5pt;font-family:Roboto;color:#0E101A'>We are sharing this information on our current success and challenges with Loom. Please let us know your thoughts and questions on our approach(es).</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><b><span style='font-family:"Arial",sans-serif;color:#0E101A'>Context</span></b><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>Since the early days of the Loom project, we have been looking at various approaches to integrate Loom (mostly virtual threads) into Quarkus. Our goal was (and still is) to dispatch <i>processing </i>(HTTP requests, Kafka messages, gRPC calls) on virtual threads. Thus, the user would not have to think about blocking or not blocking (more on that later as it relates to the Quarkus architecture) and can write synchronous code without limiting the application's concurrency.</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>To achieve this, we need to dispatch the processing on virtual threads but also have compliant <i>clients</i> to invoke remote services (HTTP, gRPC…), send messages (Kafka, AMQP), or interact with a data store (SQL or NoSQL).</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><b><span style='font-family:"Arial",sans-serif;color:#0E101A'>Quarkus Architecture</span></b><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>Before going further, we need to explain how Quarkus is structured. Quarkus is based on a reactive engine (Netty + Eclipse Vert.x), so under the hood, Quarkus uses event loops to schedule the workloads and non-blocking I/O. There is also the possibility of using Netty Native Transport (epoll, kqueue, io_uring). </span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>The <i>processing </i>can be either directly dispatched to the event loop or on a worker thread (OS thread). In the first case, the code must be written in an asynchronous and non-blocking manner. Quarkus proposes a programming model and safety guards to write such a code. In the latter case, the code can be blocking. </span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>Quarkus decides which dispatching strategy it uses for each <i>processing job</i>. The decision is based on the method signatures and annotations (for example, the user can force it to be called on an event loop or a worker thread). </span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>When using a worker thread, the request is received on an event loop and dispatched to the worker thread, and when the response is ready to be written (when it fits in memory), Quarkus switches back to the event loop.</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><b><span style='font-family:"Arial",sans-serif;color:#0E101A'>The current approach</span></b><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>The integration of Loom's virtual threads is currently based[1] on a new annotation (@RunOnVirtualThread). It introduces a third dispatching strategy, and methods annotated with this annotation are called on a virtual thread. So, we now have three possibilities:</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><ul type=disc><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1;vertical-align:baseline;font-variant-numeric:normal;font-variant-east-asian:normal'><span style='font-family:"Arial",sans-serif'><o:p> </o:p></span></li></ul><div><ul type=disc><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'>Execute<o:p></o:p></span></li><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'>the processing on an event loop thread - the code must be non-blocking <o:p></o:p></span></li></ul></div><ul type=disc><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><o:p> </o:p></span></li><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1;vertical-align:baseline;font-variant-numeric:normal;font-variant-east-asian:normal'><span style='font-family:"Arial",sans-serif'><o:p> </o:p></span></li></ul><div><ul type=disc><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'>Execute<o:p></o:p></span></li><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'>the processing on an OS (worker) thread - with the thread cost and concurrency limit<o:p></o:p></span></li></ul></div><ul type=disc><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><o:p> </o:p></span></li><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1;vertical-align:baseline;font-variant-numeric:normal;font-variant-east-asian:normal'><span style='font-family:"Arial",sans-serif'><o:p> </o:p></span></li></ul><div><ul type=disc><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'>Execute<o:p></o:p></span></li><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'>the processing on a virtual thread<o:p></o:p></span></li></ul></div><ul type=disc><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><o:p> </o:p></span></li></ul><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>The following snippet shows an elementary example:</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>@GET</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>@Path("/loom")</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><div><p class=MsoNormal><b><span style='font-family:"Arial",sans-serif;color:#0E101A'>@RunOnVirtualThread</span></b><span style='font-size:12.0pt'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>Fortune example() {</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'> var list = repo.findAll();</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'> return pickOne(list);</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>}</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal style='margin-bottom:12.0pt'><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>This support is already experimentally available in Quarkus 2.10.</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><b><span style='font-family:"Arial",sans-serif;color:#0E101A'>Previous attempts</span></b><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>The current approach is not our first attempt. We had two other approaches that we discarded, while the second one is something we want to reconsider.</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><i><span style='font-family:"Arial",sans-serif;color:#0E101A'>First Approach - All workers are virtual threads</span></i><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>The first approach was straightforward. The idea was to replace the worker (OS) threads with Virtual Threads. However, we quickly realized some limitations. Long-running (purely CPU-bound) processing would block the carrier thread as there is no preemption. While the user should be aware that long-running processing should not be executed on virtual threads, in this model, it was not explicit. We also started capturing carrier thread pinning situation (our current approach still has this issue, we will explain our bandaid later). </span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><i><span style='font-family:"Arial",sans-serif;color:#0E101A'>Second Approach - Marry event loops and carrier threads</span></i><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>Quarkus is designed to reduce the memory usage of the application. We are obsessed with RSS usage, especially when running in a container where resources are scarce. It has driven lots of our architecture choices, including the dimensioning strategies (number of event loops, number of worker threads…). </span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>Thus, we investigated the possibility of avoiding having a second carrier thread pool and reducing the number of switches between the event loops and the carrier threads. We tried to use Netty event loops as carrier threads to achieve this. We had to use private APIs (which used to be public at some point in early access builds) to implement such an approach [3]. </span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>Unfortunately, we quickly ran into issues (explaining why our method is not part of the public API). Typically we had deadlock situations when a carrier thread shared locks with virtual threads. This made it impossible to use event-loops as carriers considering the probability of lock sharing.</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>That custom scheduling strategy also prevents work stealing (Netty event loops do not handle work stealing) and must keep a strict ordering between I/O tasks.</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><b><span style='font-family:"Arial",sans-serif;color:#0E101A'>Pros and Cons of the current approach</span></b><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>Our current approach (based on @RunOnVirtualThread) integrates smoothly with the rest of Quarkus (even if the integration is limited to the HTTP part at that moment, as the integration with Kafka and gRPC are slightly more complicated but not impossible). </span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>The user's code is written synchronously, and the users are aware of the dispatching strategy. Due to the limitation mentioned before, we still believe it's a good trade-off, even if not ideal. </span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>However, the chances of pinning the carrier threads are still very high (</span><span style='font-size:10.5pt;font-family:Roboto;color:#202124'>caused by pervasive usage in the ecosystem of certain common JDK features - synchronized, JNI, etc.)</span><span style='font-family:"Arial",sans-serif;color:#0E101A'>. Because we would like to reduce the number of carrier threads to the bare minimum (to limit the RSS usage), we can end up with an underperforming application, which would have a concurrency level lower than the <i>classic </i>worker thread approach with pretty lousy response times. </span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><b><span style='font-family:"Arial",sans-serif;color:#0E101A'>The Netty / Loom dance</span></b><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>We implemented a bandaid to reduce the chance of pinning while not limiting the users to a small set of Quarkus APIs. Remember, Quarkus is based on a reactive core, and most of our APIs are available in two forms:</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><ul type=disc><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l1 level1 lfo2;vertical-align:baseline;font-variant-numeric:normal;font-variant-east-asian:normal'><span style='font-family:"Arial",sans-serif'><o:p> </o:p></span></li></ul><div><ul type=disc><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l1 level1 lfo2;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'>An imperative<o:p></o:p></span></li><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l1 level1 lfo2;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'>form blocking the caller thread when dealing with I/O<o:p></o:p></span></li></ul></div><ul type=disc><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l1 level1 lfo2;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><o:p> </o:p></span></li><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l1 level1 lfo2;vertical-align:baseline;font-variant-numeric:normal;font-variant-east-asian:normal'><span style='font-family:"Arial",sans-serif'><o:p> </o:p></span></li></ul><div><ul type=disc><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l1 level1 lfo2;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'>A reactive<o:p></o:p></span></li><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l1 level1 lfo2;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'>form that is non-blocking (reactive programming)<o:p></o:p></span></li></ul></div><ul type=disc><li class=MsoNormal style='color:#0E101A;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l1 level1 lfo2;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><o:p> </o:p></span></li></ul><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>To avoid thread pinning when running on a virtual thread, we offer the possibility to use the reactive form of our APIs but block the virtual thread while the result is still being computed. These <i>awaits</i> do not block the carrier thread and can be used with API returning 0 or one result, but also with streams (like Kafka topics) where you receive an iterator. </span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>As said above, this is a band-aid until we have non-pinning clients/APIs.</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>Under the hood, there is a dance between the virtual thread and the netty event loop (used by the reactive API). It introduces a few unfortunate switches but workaround the pinning issue.</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><b><span style='font-family:"Arial",sans-serif;color:#0E101A'>Observations</span></b><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>Over the past year, we ran many tests to design and implement our integration. The current approach is far from ideal, but it works fine. </span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>We have excellent results when we compared with a full reactive approach and a worker approach (Quarkus can have the three variants in the same app).</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>The response time under load is close enough to the reactive approach. It is far better than the classic worker thread approach [1][2].</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>However (remember we are obsessed with RSS), the RSS usage is very high. Even higher than the worker thread approach. At that moment, we are investigating where these objects come from. We hope to have a better understanding after the summer. Our observations show that the performance penalty is likely due to memory consumption (and GC cycles). However, as said, we are still investigating.</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><b><span style='font-family:"Arial",sans-serif;color:#0E101A'>Ideally</span></b><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>For us (Quarkus) and probably several other Java frameworks based on Netty, it would be terrific if we could find a way to reconcile the two scheduling strategies (in a sense, we would use the event loops as carrier thread). Of course, there will be trade-offs and limitations. Our initial attempt didn't end well, but that does not mean it's a dead end. </span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>An <i>event-loop carrier thread</i> would greatly benefit the underlying reactive engine (Netty/Vert.x in the case of Quarkus). It retains some event-loop execution semantics: code is multithreaded (in the virtual thread meaning) yet executed with a single carrier thread that respects the event-loop principles and shall have decent mechanical sympathy. In addition, it should enable using classic blocking constructs (e.g., <i>java.util.lock.Lock</i>), whereas currently, it can only block on Vert.x (e.g., a Vert.x futures but not <i>java.util.lock.Lock</i>) as Vert.x needs to be aware of the thread suspension to schedule event dispatching in a race-free / deadlock-free manner.</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>With such integration, virtual threads would be executed on the event loop. When they "block", they would be unmounted, and I/O or another virtual thread would be processed. That would reduce the number of switches between threads, reduce RSS usage, and allow lots of Java frameworks to leverage Loom virtual threads quickly. Of course, this approach can only be validated empirically. Typically, it adds latency to every virtual thread dispatch. In addition, watchdogs would need to be implemented to prevent (or at least warn the user) the execution of long CPU-intensive actions that do not yield in an acceptable time.</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><b><span style='font-family:"Arial",sans-serif;color:#0E101A'>Conclusion</span></b><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>Our integration of Loom virtual threads in Quarkus is already available to our users, and we will be collecting feedback.</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>As explained in this email, we have thus identified <b><i>two issues</i></b>.</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>The first one is purely about performance, and we were able to measure it empirically: the interaction between Loom and the Netty/Vert.x reactive stack seems to create an abundance of data structures that put pressure on the GC and degrade the overall performance of the application. As said above, we are investigating.</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>The second one is more general and also impacts <i>programming with Quarkus/Vert.x Loom. </i>The goal is to reconcile the scheduling strategies of Loom and Netty/Vert.x. This could improve performance by decreasing the number of context switches (Loom-Netty dance) and the RSS of an application. Moreover, it would enable the use of classic blocking constructs in Vert.x <i>directly</i> -i.e., without wrapping them in Vert.x own abstractions). We could not validate and/or characterize the performance improvement of such a model yet. The result is unclear as we don’t know if the decrease in context switches would be outweighed by the additional latency in virtual threads dispatch.</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-size:10.5pt;font-family:Roboto;color:#0E101A'>We are sharing this information on our current success and challenges with Loom.</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-size:10.5pt;font-family:Roboto;color:#0E101A'> Please let us know your thoughts and concerns on our approach(es). Thanks!</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.5pt;font-family:Roboto;color:#0E101A'><br><br></span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.5pt;font-family:Roboto;color:#0E101A'>Clement</span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.5pt;font-family:Roboto;color:#0E101A'><br><br></span><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>[1] - </span><a href="https://developers.redhat.com/devnation/tech-talks/integrate-loom-quarkus"><span style='font-family:"Arial",sans-serif'>https://developers.redhat.com/devnation/tech-talks/integrate-loom-quarkus</span></a><span style='font-size:12.0pt'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>[2] - </span><a href="https://github.com/anavarr/fortunes_benchmark"><span style='font-family:"Arial",sans-serif'>https://github.com/anavarr/fortunes_benchmark</span></a><span style='font-size:12.0pt'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:"Arial",sans-serif;color:#0E101A'>[3] - </span><a href="https://github.com/openjdk/loom/commit/cad26ce74c98e28854f02106117fe03741f69ba0"><span style='font-family:"Arial",sans-serif'>https://github.com/openjdk/loom/commit/cad26ce74c98e28854f02106117fe03741f69ba0</span></a><span style='font-size:12.0pt'><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p></div></div></div></blockquote></div><p class=MsoNormal><o:p> </o:p></p></div></div></body></html>