Re: RFR[8238286]: 'Add new flatMap stream operation that is more amenable to pushing’

[Anthony Vanelverdinghe]

Hi Julia

Since short-circuiting sounds similar to a Subscriber cancelling its 
Subscription, I believe it might be worthwhile to consider the Flow API.
If the argument would be a `Flow.Processor<T, R>`, then the 
implementation would publish instances of T to it, subscribe to receive 
instances of R, and be able to short-circuit by cancelling its Subscription.

Disclaimer: I haven't actually prototyped this, but it makes sense at 
first thought

Kind regards,
Anthony

On 03/07/2020 11:00, Julia Boes wrote:
> Hi Tagir,
>> By the way, the proposed API allows no possibility to short-circuit
>> the pusher. So if mapMulti produces many elements and short-circuiting
>> terminal operation like anyMatch finds the match, it won't stop the
>> pusher from pushing. Have you considered to use
>> BiConsumer<Predicate<T>, T>, so Stream API may signal to the pusher
>> that it's ok to stop pushing? Note that in modern Java versions,
>> flatMap can actually short-circuit.
>
> We did prototype mapMulti with a Predicate and it brought to light 
> some limitations that led us to decide against this approach.
>
> 1) On the user side, Predicate::test or Predicate::negate would be 
> used to check if it's ok to keep pushing. The semantics of these 
> methods doesn't really fit the use case (short-circuiting requested 
> true/false) and we would probably need to come up with a new method to 
> avoid confusion.
>
> 2) Along the lines of 1), but more generally the burden of exposing an 
> implementation detail to the user. You are right that flatMap supports 
> short-circuiting, but it's hidden in the implementation and the user 
> doesn't need to know how it's implemented or how to make it work. For 
> mapMulti however, we would need to expose some of it through the use 
> of Predicate, and explain to the user how to use it correctly.
>
> 3) While the JDK implementation uses Sink, a type of Consumer that 
> supports cancellation, the default implementation uses SpinedBuffer, a 
> type of Consumer that does not support cancellation. We would need to 
> add that functionality.
>
> 4) The difficulty of signalling a short-circuit request across the 
> stream pipeline, in particular because mapMulti uses flatMap 
> internally, which creates something like an inner nested pipeline. 
> While it's possible to signal across the main outer pipeline, it's not 
> easy to signal between outer and inner pipeline.
>
>
> Regards,
>
> Julia
>
>
>