New candidate JEP: 485: Stream Gatherers

[Cay Horstmann]

Hi, I have some belated questions about the design choices in this API that I could not find addressed in the JEP.

1. Why aren't characteristics used to express "greediness/short-circuiting" or "sequentialness/parallelizability"?

I understand that for the former I use ofGreedy/of, or implement Gatherers.Integrator.Greedy/Gatherers.Integrator. And for the latter, I use ofSequential/of, or, if I implement the Gatherer interface, have the combiner return defaultCombiner() or not.

But it seems a bit complex and less familiar than the characteristics mechanism that exists for spliterators, streams, and collectors.

The original design document (https://cr.openjdk.org/~vklang/Gatherers.html) used characteristics, so I wonder what motivated the change.

2. Why wasn't the combiner() designed to allow pushing of elements to the end of the first range's sink? Then distinctBy could be parallelized without buffering the elements. More generally, with some state fiddling one can then handle the elements around range splits.

As it is, I don't see how to parallelize such computations other than to buffer all elements.

I looked at the project at https://github.com/jhspetersson/packrat that implements a number of gatherers. Only one uses a combiner, to join buffers until their contents can be pushed in the finisher.

Cheers,

Cay

On 02/09/2024 19:55, Mark Reinhold wrote:
> https://openjdk.org/jeps/485
> 
>    Summary: Enhance the Stream API to support custom intermediate
>    operations.  This will allow stream pipelines to transform data in ways
>    that are not easily achievable with the existing built-in intermediate
>    operations.
> 
> - Mark

--

Cay S. Horstmann | https://horstmann.com