[10] RFR 8193856 takeWhile produces incorrect result with elements produced by flatMap

[Paul Sandoz]

Hi Jonathan,

> On 20 Dec 2017, at 14:47, Jonathan Bluett-Duncan <jbluettduncan at gmail.com> wrote:
> 
> Hi Paul,
> 
> It seems that some clever Googler managed to find a workaround for aggressive `flatMap` operations in the form of a so-called `MoreStreams.flatten` operation, implemented in a side project called google/mug.

Thanks. MoreStreams.flatten “lowers” a stream of streams to a spliterator of spliterators which in turn is input as a new stream source, enabling a cancellation check + tryAdvance for any following short-circuit or terminal ops.

I have a solution in the works for flatMap based on sorted operation (which has to buffer all elements, sort ‘em, then push ‘em all downstream).

Paul.

> I'm sharing the javadoc <https://google.github.io/mug/apidocs/com/google/mu/util/stream/MoreStreams.html#flatten-java.util.stream.Stream-> and GitHub project homepage <https://github.com/google/mug> with you and the rest of the mailing list in the hope that they prove to be useful.
> 
> Cheers,
> Jonathan
> 
> On 20 December 2017 at 21:28, Paul Sandoz <paul.sandoz at oracle.com <mailto:paul.sandoz at oracle.com>> wrote:
> Hi,
> 
> Please review this fix for a bug in the stream takeWhile operation:
> 
>   http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8193856-takeWhile-incorrect-results/webrev/ <http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8193856-takeWhile-incorrect-results/webrev/> <http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8193856-takeWhile-incorrect-results/webrev/ <http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8193856-takeWhile-incorrect-results/webrev/>>
> 
> The flatMap operation is currently aggressive and does not detect if a downstream operation may or has cancelled processing, and will push all of it’s elements downstream. Short-circuiting operations should be guarded against such behaviour but unfortunately takeWhile was not guarded.
> 
> —
> 
> Separately i plan to figure out a way to ensure flatMap operations become less aggressive if there are short-circuiting downstream operations. This may increase efficiency and also allow support for flat map results that are infinite.
> 
> Paul.
>