Re: RFR[8238286]: 'Add new flatMap stream operation that is more amenable to pushing’

[Paul Sandoz]

Hi Peter,

You guessed correctly about the motivation from the API perspective.

I would like to stress that short-circuiting optimizations are an implementation detail that are only partially applied. Surfacing the notion of cooperative cancellation for one operation has quite a cost, and will likely increase the expectation that it applies in all scenarios.

My sense is your example is likely more of on the edge than common. I believe it's simple to implement but would prefer to hold off and focus the current implementation and integrate, then we could reconsider afterwards if need be.

Paul.

> On Jul 5, 2020, at 1:44 AM, Peter Levart <peter.levart at gmail.com> wrote:
> 
> Hi Patric, Julia,
> 
> On 7/2/20 6:45 PM, Patrick Concannon wrote:
>> http://cr.openjdk.java.net/~pconcannon/8238286/webrevs/webrev.03/  <http://cr.openjdk.java.net/~pconcannon/8238286/webrevs/webrev.03/>
> 
> 
> Then I'd like to discuss cancelation (shortcut Streams). Currently mapMulti does not do anything to optimize processing if the downstream decides it does not need more elements but just keeps pushing. This is a decision I guess to keep the API surface clean and simple. All short-cutting ops or even intermediate (like limit) must therefore ignore the surplus elements that are emitted by mapMulti and they do so. If the intermediate operations up-stream mapMulti are respecting the cancelation, this results in at most one invocation of mapMulti function producing surplus elements, but if they don't, then multiple invocations of mapMulti function is producing surplus elements.
> 
> For example:
> 
> 
>     someStream
> 
>         .mapMulti((e, sink) -> { .... sink.accept(x); ... })
> 
>         .filter(x -> ...)
> 
>         .mapMulti((x, sink) -> { ... sink.accept(y); ...})
> 
>         .limit(10)
> 
>         ...
> 
> 
> Here the 1st mapMulti emits elements and each of them is filtered and then maybe passed to 2nd mapMulti. the 2nd mapMulti could skip calling its function if the downstream limit(10) has already passed 10 elements.
> 
> 
> WDYT?
> 
> 
> Regards, Peter
> 
>