Re: RFR[8238286]: 'Add new flatMap stream operation that is more amenable to pushing’

[forax at univ-mlv.fr]

At the same time,
mapMulti and flatMap are two sides of the same coins,
flatMap uses a pull API while mapMulti uses a push API.

so having mapMulti and flatMap doing cancellation differently is inconsistent.

Consistency is a good way to avoid to expect that all operations do short-circuiting optimizations.

Rémi 

----- Mail original -----
> De: "Paul Sandoz" <paul.sandoz at oracle.com>
> À: "Peter Levart" <peter.levart at gmail.com>
> Cc: "Patrick Concannon" <patrick.concannon at oracle.com>, "Remi Forax" <forax at univ-mlv.fr>, "Julia Boes"
> <julia.boes at oracle.com>, "core-libs-dev" <core-libs-dev at openjdk.java.net>
> Envoyé: Mardi 7 Juillet 2020 20:09:01
> Objet: Re: RFR[8238286]: 'Add new flatMap stream operation that is more amenable to pushing’

> Hi Peter,
> 
> You guessed correctly about the motivation from the API perspective.
> 
> I would like to stress that short-circuiting optimizations are an implementation
> detail that are only partially applied. Surfacing the notion of cooperative
> cancellation for one operation has quite a cost, and will likely increase the
> expectation that it applies in all scenarios.
> 
> My sense is your example is likely more of on the edge than common. I believe
> it's simple to implement but would prefer to hold off and focus the current
> implementation and integrate, then we could reconsider afterwards if need be.
> 
> Paul.
> 
>> On Jul 5, 2020, at 1:44 AM, Peter Levart <peter.levart at gmail.com> wrote:
>> 
>> Hi Patric, Julia,
>> 
>> On 7/2/20 6:45 PM, Patrick Concannon wrote:
>>> http://cr.openjdk.java.net/~pconcannon/8238286/webrevs/webrev.03/
>>> <http://cr.openjdk.java.net/~pconcannon/8238286/webrevs/webrev.03/>
>> 
>> 
>> Then I'd like to discuss cancelation (shortcut Streams). Currently mapMulti does
>> not do anything to optimize processing if the downstream decides it does not
>> need more elements but just keeps pushing. This is a decision I guess to keep
>> the API surface clean and simple. All short-cutting ops or even intermediate
>> (like limit) must therefore ignore the surplus elements that are emitted by
>> mapMulti and they do so. If the intermediate operations up-stream mapMulti are
>> respecting the cancelation, this results in at most one invocation of mapMulti
>> function producing surplus elements, but if they don't, then multiple
>> invocations of mapMulti function is producing surplus elements.
>> 
>> For example:
>> 
>> 
>>     someStream
>> 
>>         .mapMulti((e, sink) -> { .... sink.accept(x); ... })
>> 
>>         .filter(x -> ...)
>> 
>>         .mapMulti((x, sink) -> { ... sink.accept(y); ...})
>> 
>>         .limit(10)
>> 
>>         ...
>> 
>> 
>> Here the 1st mapMulti emits elements and each of them is filtered and then maybe
>> passed to 2nd mapMulti. the 2nd mapMulti could skip calling its function if the
>> downstream limit(10) has already passed 10 elements.
>> 
>> 
>> WDYT?
>> 
>> 
>> Regards, Peter
>>