<div dir="ltr">So i have played around a bit and managed to come up with some implementation based on array of streams, you can find it here: <a href="https://github.com/Evemose/nconcat/blob/master/src/main/java/nconcat/NConcatSpliterator.java">https://github.com/Evemose/nconcat/blob/master/src/main/java/nconcat/NConcatSpliterator.java</a><div><br></div><div>I have also added a small benchmark to the project, and the numbers are:<br><br>Benchmark (streamCount) Mode Cnt Score Error Units<br>NConcatBenchmark.nConcatFindFirst 4 avgt 10 131.616 � 15.474 ns/op<br>NConcatBenchmark.nConcatFindFirst 8 avgt 10 187.929 � 6.544 ns/op<br>NConcatBenchmark.nConcatFindFirst 16 avgt 10 322.342 � 6.940 ns/op<br>NConcatBenchmark.nConcatFindFirst 32 avgt 10 659.856 � 85.509 ns/op<br>NConcatBenchmark.nConcatFindFirst 64 avgt 10 1214.133 � 22.156 ns/op<br>NConcatBenchmark.nConcatMethod 4 avgt 10 1910.150 � 25.269 ns/op<br>NConcatBenchmark.nConcatMethod 8 avgt 10 3865.364 � 112.536 ns/op<br>NConcatBenchmark.nConcatMethod 16 avgt 10 7743.097 � 74.655 ns/op<br>NConcatBenchmark.nConcatMethod 32 avgt 10 15840.551 � 440.659 ns/op<br>NConcatBenchmark.nConcatMethod 64 avgt 10 32891.336 � 1122.630 ns/op<br>NConcatBenchmark.nConcatToListWithFilter 4 avgt 10 9527.120 � 376.325 ns/op<br>NConcatBenchmark.nConcatToListWithFilter 8 avgt 10 20260.027 � 552.444 ns/op<br>NConcatBenchmark.nConcatToListWithFilter 16 avgt 10 44724.856 � 5040.069 ns/op<br>NConcatBenchmark.nConcatToListWithFilter 32 avgt 10 82577.518 � 2050.955 ns/op<br>NConcatBenchmark.nConcatToListWithFilter 64 avgt 10 181460.219 � 20809.669 ns/op<br>NConcatBenchmark.nconcatToList 4 avgt 10 9268.814 � 712.883 ns/op<br>NConcatBenchmark.nconcatToList 8 avgt 10 18164.147 � 786.803 ns/op<br>NConcatBenchmark.nconcatToList 16 avgt 10 35146.891 � 966.871 ns/op<br>NConcatBenchmark.nconcatToList 32 avgt 10 68944.262 � 5321.730 ns/op<br>NConcatBenchmark.nconcatToList 64 avgt 10 136845.984 � 3491.562 ns/op<br>NConcatBenchmark.standardStreamConcat 4 avgt 10 1951.522 � 85.130 ns/op<br>NConcatBenchmark.standardStreamConcat 8 avgt 10 3990.410 � 190.517 ns/op<br>NConcatBenchmark.standardStreamConcat 16 avgt 10 8599.869 � 685.878 ns/op<br>NConcatBenchmark.standardStreamConcat 32 avgt 10 17923.603 � 361.874 ns/op<br>NConcatBenchmark.standardStreamConcat 64 avgt 10 46797.408 � 4458.069 ns/op<br>NConcatBenchmark.standardStreamConcatFindFirst 4 avgt 10 125.192 � 3.123 ns/op<br>NConcatBenchmark.standardStreamConcatFindFirst 8 avgt 10 303.791 � 8.670 ns/op<br>NConcatBenchmark.standardStreamConcatFindFirst 16 avgt 10 907.429 � 52.620 ns/op<br>NConcatBenchmark.standardStreamConcatFindFirst 32 avgt 10 2964.749 � 320.141 ns/op<br>NConcatBenchmark.standardStreamConcatFindFirst 64 avgt 10 11749.653 � 189.300 ns/op<br>NConcatBenchmark.standardStreamConcatToList 4 avgt 10 7059.642 � 740.735 ns/op<br>NConcatBenchmark.standardStreamConcatToList 8 avgt 10 13714.980 � 250.208 ns/op<br>NConcatBenchmark.standardStreamConcatToList 16 avgt 10 27028.052 � 565.047 ns/op<br>NConcatBenchmark.standardStreamConcatToList 32 avgt 10 53537.731 � 853.363 ns/op<br>NConcatBenchmark.standardStreamConcatToList 64 avgt 10 105847.755 � 3179.918 ns/op<br>NConcatBenchmark.standardStreamConcatToListWithFilter 4 avgt 10 9736.527 � 154.817 ns/op<br>NConcatBenchmark.standardStreamConcatToListWithFilter 8 avgt 10 20607.061 � 713.083 ns/op<br>NConcatBenchmark.standardStreamConcatToListWithFilter 16 avgt 10 41241.199 � 1171.672 ns/op<br>NConcatBenchmark.standardStreamConcatToListWithFilter 32 avgt 10 83029.244 � 1843.176 ns/op<br>NConcatBenchmark.standardStreamConcatToListWithFilter 64 avgt 10 182349.009 � 11282.832 ns/op</div><div><br></div><div>Basically, the conclusion is following (guilty of using AI for summarizing):</div><div><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">The comprehensive benchmarks reveal that <strong>NConcat significantly outperforms the standard library for processing-intensive operations</strong> while trailing in simple collection scenarios. For short-circuit operations like <code class="gmail-bg-text-200/5 gmail-border gmail-border-0.5 gmail-border-border-300 gmail-text-danger-000 gmail-whitespace-pre-wrap gmail-rounded-[0.4rem] gmail-px-1 gmail-py-px gmail-text-[0.9rem]">findFirst()</code>, NConcat delivers 38-90% better performance as stream count increases, reaching nearly 10x faster execution at 64 streams due to superior scaling (19ns/stream vs 184ns/stream). Full traversal operations like <code class="gmail-bg-text-200/5 gmail-border gmail-border-0.5 gmail-border-border-300 gmail-text-danger-000 gmail-whitespace-pre-wrap gmail-rounded-[0.4rem] gmail-px-1 gmail-py-px gmail-text-[0.9rem]">forEach </code>consistently favor NConcat by 2-30%, with the advantage growing at scale. However, simple collection operations (<code class="gmail-bg-text-200/5 gmail-border gmail-border-0.5 gmail-border-border-300 gmail-text-danger-000 gmail-whitespace-pre-wrap gmail-rounded-[0.4rem] gmail-px-1 gmail-py-px gmail-text-[0.9rem]">toList()</code>) consistently run 22-24% faster with the standard library across all stream counts.</blockquote><div> </div>
<p class="gmail-whitespace-normal gmail-break-words">I have tried multiple approaches to optimize toList with know size of all sub-streams (which is clearly the reason why standard implementation wins here), and am sure that there is still plenty of room for improvement, especially in parallel, but the takeaway is, even a naive implementation like mine could bring a significant performance improvement to the table in early short-circuiting and full traversal cases that do not depend on size of the spliterator.<br><br>Besides the performance part, of course, the most significant advantage of my proposal, as I think, is still developer experience, both reading and writing stream code.<br><br>Please let me know your thoughts on the results of prototype and possible ways forward.<br><br>Best regards</p></div></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Wed, Sep 17, 2025 at 6:04 PM Olexandr Rotan <<a href="mailto:rotanolexandr842@gmail.com">rotanolexandr842@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hello everyone! Thanks for your responses<br><br>I will start of by answering to Viktor<br><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span style="color:rgb(0,0,0);font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:16px">I guess a "simple" implementation of an N-ary concat could work, but it would have performance implications (think a recursive use of Stream.concat())</span></blockquote><div><br></div><div>I too find just the addition of small reduction-performing sugar methods rather unsatisfactory and most certainly not bringing enough value to be considered a valuable addition. Moreover, I have not checked it myself, but I would dare to guess that popular utility libraries such as Guava or Apache Commons already provide this sort of functionality in their utility classes. Though, if this method could bring some significant performance benefits, I think it may be a valuable candidate to consider. Though, to me as a user, the main value would be uniformity of the API and ease of use and read. The main reason I am writing about this in the first place is the unintuitive inconsistency with many other static methods-creators that happily accept varargs<br><br>I may play around with this spliterator code you have linked to to see if I could make it generalized for arrays of streams<br><br>Now, answering to Pavel<br><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Is it such a useful use case, though? I mean, it's no different from SequenceInputStream(...) or Math.min/max for that matter. I very rarely have to do Math.min(a, Math(min(b, c)) or some such.</blockquote><div><br></div><div>I certainly see your point, but I would dare to say that most applications rely on the streams much more than SequenceInputStream and Math classes, and their lookalikes. Stream.concat is primarily a way to merge a few datasource outputs into one, for later uniform processing, which, in the nutshell, is one of the most common tasks in data-centric applications. Of course, not every such use case has characteristics that incline developers to use Stream.concat, such as combination of Stream.of and Collection.stream() sources, and even if they do, not every case that fits previous requirement requires to merge more than 2 sources. However, for mid-to-large scale apps, for which java is known the most, I would say it's fairly common. I went over our codebase and found that there were at least 10+ usages of concat, and a few of them followed this kinda ugly pattern of nested concates.<br><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Separately, it's not just one method. Consider that `concat` is also implemented in specialized streams such as IntStream, DoubleStream, and LongStream.</blockquote></div><br>This is unfortunate, but I would dare to say that once Reference spliterrator is implemented, others may also be derived by analogy fairly quickly<br><br>And last but not least, answering Daniel<br><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Not immediately obvious but you can create a Stream<Stream<T>> using<br>Stream.of and reduce that using Stream::concat to obtain a Stream<T>.</blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Something along those lines:</blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">```<br>var stream = Stream.of(Stream.of(1,2,3), Stream.of(4), Stream.of(5, 6,<br>7, 8)).reduce(Stream.empty(), Stream::concat, Stream::concat);</blockquote><div><br>This is what I meant by "reduction-like" implementation, which is fairly straightforward, but just from the looks of it, one could assume that this solution will surely have performance consequences, even if using
flatmap insead of reduce. Not sure though, how often people would want to use such approach on the array of streams huge enough for the performance difference to be noticable, though I would assume that there is a non-linear scale of consumed time and resources from the length of streams array due to the implementation of concat method.<br><br>Nevertheless, this is an acceptable workaround for such cases, even though not the most readable one. Even if this approach is accepted as sufficient for such cases of n-sized array of streams merging, It would probably make some sense to put note about it in the docs of the concat method. Though, not having concat(Stream..) overload would still remain unintuitive for many developers, including me<br><br>Thanks everybody for the answers again<br><br>Best regards</div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Sep 17, 2025 at 5:15 PM Pavel Rappo <<a href="mailto:pavel.rappo@gmail.com" target="_blank">pavel.rappo@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">> this would be a great quality of life improvement<br>
<br>
Is it such a useful use case, though? I mean, it's no different from<br>
SequenceInputStream(...) or Math.min/max for that matter. I very<br>
rarely have to do Math.min(a, Math(min(b, c)) or some such. And those<br>
methods predate streams API by more than a decade.<br>
<br>
Separately, it's not just one method. Consider that `concat` is also<br>
implemented in specialized streams such as IntStream, DoubleStream,<br>
and LongStream.<br>
<br>
On Wed, Sep 17, 2025 at 2:58 PM Olexandr Rotan<br>
<<a href="mailto:rotanolexandr842@gmail.com" target="_blank">rotanolexandr842@gmail.com</a>> wrote:<br>
><br>
> Greetings to everyone on the list.<br>
><br>
> When working on some routine tasks recently, I have encountered a, seemingly to me, strange decision in design of Stream.concat method, specifically the fact that it accepts exactly two streams. My concrete example was something along the lines of<br>
><br>
> var studentIds = ...;<br>
> var teacherIds = ...;<br>
> var partnerIds = ...;<br>
><br>
> return Stream.concat(<br>
> studentIds.stream(),<br>
> teacherIds.stream(),<br>
> partnerIds.stream() // oops, this one doesn't work<br>
> )<br>
><br>
> so I had to transform concat to a rather ugly<br>
> Stream.concat(<br>
> studentIds.stream(),<br>
> Stream.concat(<br>
> teacherIds.stream(),<br>
> partnerIds.stream()<br>
> )<br>
> )<br>
><br>
> Later on I had to add 4th stream of a single element (Stream.of), and this one became even more ugly<br>
><br>
> When I first wrote third argument to concat and saw that IDE highlights it as error, I was very surprised. This design seems inconsistent not only with the whole java stdlib, but even with Stream.of static method of the same class. Is there any particular reason why concat takes exactly to arguments?<br>
><br>
> I would say that, even if just in a form of sugar method that just does reduce on array (varagrs) of streams, this would be a great quality of life improvement, but I'm sure there also may be some room for performance improvement.<br>
><br>
> Of course, there are workarounds like Stream.of + flatmap, but:<br>
><br>
> 1. It gets messy when trying to concat streams of literal elements set (Stream.of) and streams of collections or arrays<br>
> 2. It certainly has significant performance overhead<br>
> 3. It still doesn't explain absence of varagrs overload of concat<br>
><br>
> So, once again, is there any particular reason to restrict arguments list to exactly two streams? If not, I would be happy to contribute Stream.concat(Stream... streams) overload.<br>
><br>
> Best regards<br>
><br>
><br>
><br>
</blockquote></div>
</blockquote></div>