hg: lambda/lambda/jdk: Replace explode() with two forms of flatMap: flatMap(T->Stream<U>), and flatMap(FlatMapper<T, U>)

[Brian Goetz]

Thanks!  I think it was a good compromise.  But I'm not sure whether to 
be happy or apprehensive about your enthusiasm.  I think probably a 
little of both.  Here's why.

The FlatMapper form -- which is very much like the current form of 
explode that you were happy to get away from (and with good reason, its 
confusing) -- is efficient; there is very little constant overhead per 
input element processed.  If something maps to nothing, it does no work. 
  Otherwise, the elements are produced and fed directly into the 
downstream consumer -- very efficient.

The T -> Stream<U> form *looks* like "hey, that's what I want", because 
it makes the code pretty.  But the performance will range from 
marginally worse (when elements reliably map to largish collections that 
are already instantiated) to much worse.

In the worst case, where an element maps to nothing, you are creating an 
empty collection, wrapping it with a Stream, asking the Stream to 
forEach its elements, which in turn creates a Spliterator (like an 
iterator).  And each of these may have ancillary objects (e.g., an 
ArrayList has an array; a Stream has helper objects, etc.)  So that's an 
awful lot of activity to do what should be zero work for an element that 
maps to nothing.

So, if you care more about code clarity than performance (that's not a 
dig -- the vast majority of code meets this description, or at least 
should, and almost certainly lots more code than its authors think) then 
the new form will make you very happy.  The downside is that it is 
slower than it looks.  Which invariably will lead to "streams are slow" 
lore in a year or two :(


On 2/7/2013 10:51 PM, Arul Dhesiaseelan wrote:
> FlatMapper turns out to be pretty slick!
>
> List<Track> allTracks = albums.stream()
>        .explode((Downstream<Track> downstream, Album element) ->
> downstream.send(element.tracks))
>        .collect(toList());
>
> to
>
> List<Track> allTracks = albums.stream()
>        .flatMap((Album album) -> album.tracks.stream())
>        .collect(toList());
>
> This is awesome Brian.
>
> On Thu, Feb 7, 2013 at 9:53 AM, Brian Goetz <brian.goetz at oracle.com> wrote:
>
>> I pushed an update along the lines of what was discussed yesterday, so
>> people can take a look.  Summary:
>>
>>   - Eliminated "Downstream" abstraction
>>   - Added FlatMapper type (with nested specializations) in j.u.s.
>>   - Added five forms of Stream.flatMap
>>     flatMap(Function<T, Stream<U>)
>>     flatMap(FlatMapper<T,U>)
>>     flatMap(FlatMapper.To{Int,**Long,Double}<T>)
>>   - Added one form of flatMap for each primitive stream:
>>     {Int,Long,Double}Stream.**flatMap(FlatMapper.{ILD}To{**ILD})
>>
>> Check it out and see what you think.  Commit message attached.  I think
>> this is an improvement.
>>
>> Bikeshedding on naming can continue :)
>>
>>
>> -------- Original Message --------
>> Subject: hg: lambda/lambda/jdk: Replace explode() with two forms of
>> flatMap: flatMap(T->Stream<U>), and flatMap(FlatMapper<T, U>)
>> Date: Thu, 07 Feb 2013 19:37:14 +0000
>> From: brian.goetz at oracle.com
>> To: lambda-dev at openjdk.java.net
>>
>> Changeset: 3aed6b4f4d42
>> Author:    briangoetz
>> Date:      2013-02-07 14:36 -0500
>> URL:       http://hg.openjdk.java.net/**lambda/lambda/jdk/rev/**
>> 3aed6b4f4d42<http://hg.openjdk.java.net/lambda/lambda/jdk/rev/3aed6b4f4d42>
>>
>> Replace explode() with two forms of flatMap: flatMap(T->Stream<U>), and
>> flatMap(FlatMapper<T,U>)
>>
>> ! src/share/classes/java/util/**stream/DoublePipeline.java
>> ! src/share/classes/java/util/**stream/DoubleStream.java
>> + src/share/classes/java/util/**stream/FlatMapper.java
>> ! src/share/classes/java/util/**stream/IntPipeline.java
>> ! src/share/classes/java/util/**stream/IntStream.java
>> ! src/share/classes/java/util/**stream/LongPipeline.java
>> ! src/share/classes/java/util/**stream/LongStream.java
>> ! src/share/classes/java/util/**stream/ReferencePipeline.java
>> ! src/share/classes/java/util/**stream/Stream.java
>> ! test-ng/bootlib/java/util/**stream/LambdaTestHelpers.java
>> ! test-ng/boottests/java/util/**stream/SpinedBufferTest.java
>> ! test-ng/tests/org/openjdk/**tests/java/util/stream/**ExplodeOpTest.java
>> ! test-ng/tests/org/openjdk/**tests/java/util/stream/**ToArrayOpTest.java
>> ! test/java/util/**LambdaUtilities.java
>> ! test/java/util/stream/Stream/**EmployeeStreamTest.java
>> ! test/java/util/stream/Stream/**IntStreamTest.java
>> ! test/java/util/stream/Stream/**IntegerStreamTest.java
>> ! test/java/util/stream/Stream/**StringBuilderStreamTest.java
>> ! test/java/util/stream/Streams/**BasicTest.java
>>
>>
>>
>>
>>
>