Re: RFR[8238286]: 'Add new flatMap stream operation that is more amenable to pushing’

[Julia Boes]

Hi Tagir,
> By the way, the proposed API allows no possibility to short-circuit
> the pusher. So if mapMulti produces many elements and short-circuiting
> terminal operation like anyMatch finds the match, it won't stop the
> pusher from pushing. Have you considered to use
> BiConsumer<Predicate<T>, T>, so Stream API may signal to the pusher
> that it's ok to stop pushing? Note that in modern Java versions,
> flatMap can actually short-circuit.

We did prototype mapMulti with a Predicate and it brought to light some 
limitations that led us to decide against this approach.

1) On the user side, Predicate::test or Predicate::negate would be used 
to check if it's ok to keep pushing. The semantics of these methods 
doesn't really fit the use case (short-circuiting requested true/false) 
and we would probably need to come up with a new method to avoid confusion.

2) Along the lines of 1), but more generally the burden of exposing an 
implementation detail to the user. You are right that flatMap supports 
short-circuiting, but it's hidden in the implementation and the user 
doesn't need to know how it's implemented or how to make it work. For 
mapMulti however, we would need to expose some of it through the use of 
Predicate, and explain to the user how to use it correctly.

3) While the JDK implementation uses Sink, a type of Consumer that 
supports cancellation, the default implementation uses SpinedBuffer, a 
type of Consumer that does not support cancellation. We would need to 
add that functionality.

4) The difficulty of signalling a short-circuit request across the 
stream pipeline, in particular because mapMulti uses flatMap internally, 
which creates something like an inner nested pipeline. While it's 
possible to signal across the main outer pipeline, it's not easy to 
signal between outer and inner pipeline.


Regards,

Julia