java.util.stream.Stream: API for user-extensible intermediate operations

[Viktor Klang]

________________________________
From: Remi Forax <forax at univ-mlv.fr>
Sent: Thursday, 29 June 2023 10:03
To: Viktor Klang <viktor.klang at oracle.com>
Cc: core-libs-dev <core-libs-dev at openjdk.org>
Subject: [External] : Re: java.util.stream.Stream: API for user-extensible intermediate operations



________________________________
From: "Viktor Klang" <viktor.klang at oracle.com>
To: "core-libs-dev" <core-libs-dev at openjdk.org>
Sent: Tuesday, June 27, 2023 7:10:42 PM
Subject: java.util.stream.Stream: API for user-extensible intermediate operations
Hi core-libs-dev,

Over the past 6+ months I've been thinking about, and tinkering with, how we'd be able to expose a user-facing API for extensible intermediate java.util.stream.Stream operations―a feature envisioned all the way back when Streams were created.

I'm now at a point where I have a viable design and implementation, and so I'm turning to you for your feedback: on the direction taken; the API concepts; and, in particular, is there anything which I have overlooked/missed?

>I think this API is overly generic and hard to reason about it, for users and IDEs.

The API is for all intents and purposes Collector with a boolean return type for the accumulator and an added downstream handle parameter added to the accumulator and the finisher.

>The main issue is that the same API is used for both stateless and stateful operations, which means that as a user, we have no idea if a call to stream.gather() is stateful or not.

How is this different from any of the other pre-existing Stream operations?

>Which is a departure from the current API that cleanly separate stateless and staful operations. Here, we are left in the dark. In a sense, this API is too powerful, it can do too much thing, so as a user we can not reason about it.

A Gatherer encodes it input and output types, in what sense would that not be enough to reason about it?

>I like the idea of a Collector 2.0 i.e. using the Gatherer API at the end of the stream (not in the middle), but currently, the Gatherer API is not a Collector, so we now have two different APIs for doing partially the same job. I wonder if the Collector API can be retroffitted to act as a Gatherer API, avoiding to have to choose which one to use, a gatherer being the equivalent of a "flat-collector" + short-circuit.

Collector serves a very important role of being able to get information out of a Stream and deliver that information in a certain shape, a Gatherer does not provide any facility for this.

>The idea of unsupportedCombiner() seems out of place, like a patch to be able to clobble different things together. I'm not sure to understand why it's needed for a Gatherer, and why it is not needed for Collectors ?

Nothing prevents us from treating a `null` combiner the same way. My primary reason for making it a dedicated thing was to be able to differentiate a possible bug (user passing in a null reference inadvertently) from explicitly stating that a combiner does not exist from this operation.

unsupportedCombiner() as an artifact can be completely hidden if desired, as Gatherer.of() can have permutations without specifying a combiner, and the default method of Gatherer.combiner() could return unsupportedCombiner(). I opted not to do this initially, because I felt like being explicit about not having a combiner means that it is a concious decision by the implementor of the Gatherer.


>So I would prefer that API to extends the current Collector API but not the intermediary operations. Yes, it's less powerful.
It means that instead of using one stream with a collect like operation in the middle, users will have to use two streams, one after the other, but it makes the code easier to understand (also having two streams give users better control on which part should be in parallel).

That would be something completely different from the goal of providing user-extensible intermediate operations, which is something which this proposal is explicitly trying to address.

Cheers,
√

>Rémi


(If you, like myself, prefer reading pre-rendered markdown, click here<https://cr.openjdk.org/~vklang/Gatherers.html>)


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/core-libs-dev/attachments/20230629/88722009/attachment-0001.htm>