<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=koi8-r">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi,<br>
<br>
As I have spent a fair amount of time thinking about this topic specifically, there's a lot of nuance to what it *means* to back parallel streams with VirtualThreads.</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<b>Background</b></div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Given the design of the reference implementation of java.util.stream.Stream, in parallel mode the stream pipeline is split up into segments ending with an operation which needs to run to completion, collecting its output as the input for the next segment of
the pipeline. That parallelization is using customized implementations of CountedCompleters to literally (Spliterator) split the workload amongst a constructed tree of tasks.</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
CountedCompleters are specialized implementations of ForkJoinTasks (<a href="https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/util/concurrent/CountedCompleter.html" id="LPlnk">https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/util/concurrent/CountedCompleter.html</a> )
and enjoy a bit of support from ForkJoinPool (ex:<a href="https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/util/concurrent/ForkJoinPool.java#L1561" id="LPlnk">https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/util/concurrent/ForkJoinPool.java#L1561</a>)</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<b>On the topic of VirtualThread</b></div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
With that said, backing parallel streams with VirtualThreads would entail a lot of work to reimplement every bespoke CountedCompleter-based operation in parallel streams to perform the equivalent coordination of result completion to achieve the same performance
profile as current CPU-bound workloads using parallel streams.</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
There's also no real reason to believe it would provide any significant benefit on a per-workload basis.</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
For IO-bound workloads the situation is somewhat different, because they are often subject to externally-imposed sources of stalls. However, my experience tells me that the areas of IO are often well-understood and typically limited to specific points in a
pipeline (usually Input near the beginning, and Output near the end, but there are also cases where you might have request-response-style operations somewhere in the middle).</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
For that case, as Alan mentioned previously, Gatherers (Preview feature) offers a
<a href="https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/util/stream/Gatherers.html#mapConcurrent(int,java.util.function.Function)" id="LPlnk" title="https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/util/stream/Gatherers.html#mapConcurrent(int,java.util.function.Function)">
mapConcurrent(maxConcurrency, io-function)</a> which will preserve ordering of elements in the stream, and execute the supplied function inside a VirtualThread up to the configured concurrency limit, and supports cancellation of tasks in the event of short-circuiting.<br>
<br>
If you have any opportunity to test this functionality, your feedback would be much appreciated.</div>
<div class="_Entity _EType_OWALinkPreview _EId_OWALinkPreview_3 _EReadonly_1" style="color: inherit; background-color: inherit;">
<div id="LPBorder_GTaHR0cHM6Ly9kb2NzLm9yYWNsZS5jb20vZW4vamF2YS9qYXZhc2UvMjIvZG9jcy9hcGkvamF2YS5iYXNlL2phdmEvdXRpbC9zdHJlYW0vR2F0aGVyZXJzLmh0bWwjbWFwQ29uY3VycmVudChpbnQsamF2YS51dGlsLmZ1bmN0aW9uLkZ1bmN0aW9uKQ.." class="LPBorder995474" style="width: 100%; margin-top: 16px; margin-bottom: 16px; position: relative; max-width: 800px; min-width: 424px;">
<table id="LPContainer995474" role="presentation" style="padding: 12px 36px 12px 12px; width: 100%; border-width: 1px; border-style: solid; border-color: rgb(200, 200, 200); border-radius: 2px;">
<tbody>
<tr valign="top" style="border-spacing: 0px;">
<td style="width: 100%;">
<div id="LPTitle995474" style="font-size: 21px; font-weight: 300; margin-right: 8px; font-family: "wf_segoe-ui_light", "Segoe UI Light", "Segoe WP Light", "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; margin-bottom: 12px;">
<a target="_blank" id="LPUrlAnchor995474" href="https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/util/stream/Gatherers.html#mapConcurrent(int,java.util.function.Function)" style="text-decoration: none;">Gatherers (Java SE 22 & JDK 22)</a></div>
<div id="LPDescription995474" style="font-size: 14px; max-height: 100px; color: rgb(102, 102, 102); font-family: "wf_segoe-ui_normal", "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; margin-bottom: 12px; margin-right: 8px; overflow: hidden;">
declaration: module: java.base, package: java.util.stream, class: Gatherers</div>
<div id="LPMetadata995474" style="font-size: 14px; font-weight: 400; color: rgb(166, 166, 166); font-family: "wf_segoe-ui_normal", "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif;">
docs.oracle.com</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="Signature" style="color: inherit; background-color: inherit;">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Cheers,<br>
–</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<b><br>
</b></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<b>Viktor Klang</b></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Software Architect, Java Platform Group<br>
Oracle</div>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> loom-dev <loom-dev-retn@openjdk.org> on behalf of Alan Bateman <Alan.Bateman@oracle.com><br>
<b>Sent:</b> Saturday, 11 May 2024 07:16<br>
<b>To:</b> Nathan Reynolds <numeralnathan@gmail.com>; loom-dev@openjdk.org <loom-dev@openjdk.org><br>
<b>Subject:</b> Re: Stream.parallel() Uses Virtual Threads?</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">On 10/05/2024 18:36, Nathan Reynolds wrote:<br>
> Does/will Stream.parallel() use virtual threads or stay with the Fork <br>
> Join common pool threads?<br>
><br>
> Let's say I have a Stream where the "loop" is I/O bound. Here is an <br>
> example.<br>
><br>
> list.<br>
> stream().<br>
> parallel().<br>
> map(MyClass::ioBoundOperation).<br>
> ...<br>
><br>
> In Java 8, 11, and other pre-Java 21 versions, parallel() would use <br>
> the Fork Join common pool to execute ioBoundOperation() in multiple <br>
> threads. If the list has many elements, then the Fork Join common <br>
> pool will run out of threads since there are roughly N threads where N <br>
> = the number of vCPUs. Other parallel streams will then have to <br>
> wait. I hope I captured the situation correctly.<br>
><br>
> If parallel() uses virtual threads, then multiple parallel streams can <br>
> execute and they won't block each other since the number of virtual <br>
> threads can be very large.<br>
><br>
> Let's say parallel() uses virtual threads, would it make sense to <br>
> always use parallel()? I guess there is some small overhead for <br>
> creating a virtual thread. So, processing a small list with short <br>
> CPU-bound operations may take more time than it saves.<br>
><br>
We aren't planning to change parallel to use virtual threads, I don't <br>
think it would make sense in general.<br>
<br>
Have you looked at Gatherers for doing intermediate operations yet? One <br>
of the built-ins is mapConcurrent [1] so in your example you could <br>
replace parallel().map(MyClass:ioBoundOperation) with <br>
mapConcurrent(MyClass:ioBoundOperation) and have the mapping function <br>
for each element run in its own virtual thread.<br>
<br>
-Alan<br>
<br>
[1] <br>
<a href="https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/util/stream/Gatherers.html#mapConcurrent(int,java.util.function.Function)">https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/util/stream/Gatherers.html#mapConcurrent(int,java.util.function.Function)</a><br>
</div>
</span></font></div>
</body>
</html>