RFR: JDK-8319123 : Implementation of JEP-461: Stream Gatherers (Preview)

[Viktor Klang]

On Wed, 8 Nov 2023 16:26:56 GMT, Tagir F. Valeev <tvaleev at openjdk.org> wrote:

>> @amaembo pairMap should be straight-forward to implement. And regarding parallelization I don't think that will necessarily work since it would not be able to run incrementally (depth-first).
>
> Sorry, I'm not sure what do you mean by incrementally. But I've realized that to implement this you may need to push to downstream during combining. E.g., imagine the stream of numbers 1...1000, and you want to make `parallel().gather(pairMap((left, right) -> left+"->"+right)).collect(toList())`, creating a list of "1->2", "2->3", ..., "999->1000" strings. Now, let's assume that the stream was split into four parts during the parallelization: `1..250`, `251..500`, `501..750`, `751..1000`. So you create downstream accumulators and push the pairs there with integrator, like:
> "1->2", ..., "249->250" into first toList() accumulator, 
> "251->252", ..., "499->500" into second toList() accumulator
> "501->502", ..., "749->750" into third toList() accumulator
> "751->752", ..., "999->1000" into fourth toList() accumulator
> At this point we have four independent gatherer states, and they keep the first and last element from the upstream. Now, you combine for example, first and second gatherer states. The first one is [first=1, last=250], and the second is [first=251, last=500]. The combined gatherer should be [first=1, last=500] and at the same time during the combining it should push the "250->251" object to the downstream, which will land into the first toList() accumulator.
> 
> Hopefully my explanation is clear. Will it be possible to extend the combiner function and provide the Downstream there as well, so combiner can also work as a finisher for the first combined chunk?

Ah, now I see what you mean. In this specific case you might be able to create a combiner (which would also need a Downstream) such that you can add the missing combinations (the end of the "left" and the beginning of the "right"). But imagine something like `windowFixed(3)`, where the presence of 1 or 2 elements to the "left" will skew all the elements to the "right", potentially rippling through the entire stream.

I think the Preview is a perfect place to see if the combiner would be made even better with a Downstream reference or not. Logically, the output of said Downstream would be placed *after* the "left"'s output, and *before* the "right"'s output (so in the middle of the two).

But then someone might want to be able to inject elements outside of that order, but I guess that remains to be seen.

Great thoughts, @amaembo 👍

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16420#discussion_r1386910345