FlatMap (+ compiler bug)

Mon Oct 15 02:20:21 PDT 2012

On 10/13/2012 08:13 PM, Brian Goetz wrote:
> Thanks for the suggestion.  This is something we've considered and is still on the "being considered" list.
>
> I think your suggestion is based on the assumption that you *already* have a collection lying around, and just want to return that.  While that is the case often, it is also often not.  In the case where you do not have a collection, replacing the existing FlatMap with something like you suggest would be both painful for the user (who has to create a garbage collection in the lambda) and also less efficient.  So we could not *replace* this flatMap with one taking Mapper<T, Collection<T>> without making the API worse.
>
> We could consider adding a convenience method
>
>     flatMap(Mapper<T, Collection<T>>)
>
> for the case you describe, but we have to be careful as we are bumping our heads up against erasure.  We could only have one such method (since the erasure is flatMap(Mapper)).  We would have to pick the signature carefully.  Mapper<T,Collection>?  Mapper<T,T[]>? Mapper<T,Streamable<T>>?  If we pick Streamable (which is nice because that subsumes collections), array users are hosed -- we can't later add an array version.

One way to avoid this may be to introduce a new interface:

     interface ArrayMapper<T, R> extends Mapper<T, R[]> {}

...and use flatMap(Mapper) for Streamables and flatMap(ArrayMapper) for 
arrays.

I tried this and current overloading scheme correctly picks the right 
method. For completeness and to avoid unnecessary conversions one might 
also need individual primitive array variants: IntArrayMapper, 
LongArrayMapper, etc...

Regards, Peter

P.S.

While checking this I stumbled on a compiler bug. The following code works:

public class OverloadTest
{
     interface ArrayMapper<T, R> extends Mapper<T, R[]> {}

     interface CharArrayMapper<T> extends Mapper<T, char[]> {}

     static class MyStream<T>
     {
         private final Stream<T> stream;

         public MyStream(Stream<T> stream)
         {
             this.stream = stream;
         }

// method A:
         <R> Stream<R> flatMap(FlatMapper<? super T, R> mapper)
         {
             return stream.flatMap(mapper);
         }

// method B:
         <R> Stream<R> flatMap(Mapper<? super T, Streamable<? extends 
Stream<R>>> mapper)
         {
             return this.<R>flatMap((sink, t) -> { 
mapper.map(t).stream().forEach(sink); });
         }

// method Bx:
//        <R> Stream<R> flatMap(Mapper<? super T, Streamable<? extends 
Stream<R>>> mapper)
//        {
//            return flatMap((sink, t) -> { 
mapper.map(t).stream().forEach(r -> { sink.apply(r); }); });
//        }

// method C:
         <R> Stream<R> flatMap(ArrayMapper<? super T, R> mapper)
         {
             return this.<R>flatMap((sink, t) -> { for (R r : 
mapper.map(t)) sink.apply(r); });
         }

// method D:
         Stream<Character> flatMap(CharArrayMapper<? super T> mapper)
         {
             return this.<Character>flatMap((sink, t) -> { for (char c : 
mapper.map(t)) sink.apply(c); });
         }
     }

     public static void main(String[] args)
     {
         Streamable<Stream<String>> strings = Arrays.asList("abc", "de.");
         MyStream<String> stream;

         stream = new MyStream<>(strings.stream());

         stream
           .<Character>flatMap((sink, s) -> { for (char c : 
s.toCharArray()) sink.apply(c); } )
           .forEach(c -> { System.out.println(c); });

         stream = new MyStream<>(strings.stream());

         stream
           .<Character>flatMap(s -> new Character[] {s.charAt(0), 
s.charAt(1), s.charAt(2)})
           .forEach(c -> { System.out.println(c); });

         stream = new MyStream<>(strings.stream());

         stream
           .<Character>flatMap(s -> Arrays.asList(s.charAt(0), 
s.charAt(1), s.charAt(2)))
           .forEach(c -> { System.out.println(c); });

         stream = new MyStream<>(strings.stream());

         stream
           .<Character>flatMap(s -> s.toCharArray())
           .forEach(c -> { System.out.println(c); });
     }
}

... but if I comment out method B and uncomment method Bx, the 
compilation produces:

An exception has occurred in the compiler (1.8.0-internal). Please file 
a bug at the Java Developer Connection 
(http://java.sun.com/webapps/bugreport)  after checking the Bug Parade 
for duplicates. Include your program and the following diagnostic in 
your report.  Thank you.
java.lang.AssertionError
     at com.sun.tools.javac.util.Assert.error(Assert.java:126)
     at com.sun.tools.javac.util.Assert.check(Assert.java:45)
     at 
com.sun.tools.javac.comp.DeferredAttr$DeferredTypeMap.apply(DeferredAttr.java:454)
     at 
com.sun.tools.javac.comp.DeferredAttr$SpeculativeDeferredTypeMap.apply(DeferredAttr.java:465)
     at com.sun.tools.javac.code.Type.map(Type.java:158)
     at com.sun.tools.javac.code.Type$MethodType.map(Type.java:942)
     at 
com.sun.tools.javac.comp.DeferredAttr$DeferredTypeMap.apply(DeferredAttr.java:451)
     at 
com.sun.tools.javac.comp.DeferredAttr$SpeculativeDeferredTypeMap.apply(DeferredAttr.java:465)
     at com.sun.tools.javac.code.Type$ForAll.map(Type.java:1151)
     at com.sun.tools.javac.comp.Attr.checkId(Attr.java:3166)
     at com.sun.tools.javac.comp.Attr.visitSelect(Attr.java:3047)
     at 
com.sun.tools.javac.tree.JCTree$JCFieldAccess.accept(JCTree.java:1810)
     at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:598)
     at com.sun.tools.javac.comp.Attr.visitApply(Attr.java:1819)
     at 
com.sun.tools.javac.tree.JCTree$JCMethodInvocation.accept(JCTree.java:1393)
     at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:598)
     at com.sun.tools.javac.comp.Attr.attribExpr(Attr.java:621)
     at com.sun.tools.javac.comp.Attr.visitExec(Attr.java:1566)
     at 
com.sun.tools.javac.tree.JCTree$JCExpressionStatement.accept(JCTree.java:1224)
     at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:598)
     at com.sun.tools.javac.comp.Attr.attribStat(Attr.java:641)
     at com.sun.tools.javac.comp.Attr.attribStats(Attr.java:657)
     at com.sun.tools.javac.comp.Attr.visitLambda(Attr.java:2346)
     at com.sun.tools.javac.tree.JCTree$JCLambda.accept(JCTree.java:1534)
     at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:598)
     at 
com.sun.tools.javac.comp.DeferredAttr.attribSpeculative(DeferredAttr.java:263)
     at 
com.sun.tools.javac.comp.DeferredAttr$DeferredType.check(DeferredAttr.java:209)
     at 
com.sun.tools.javac.comp.DeferredAttr$RecoveryDeferredTypeMap.recover(DeferredAttr.java:543)
     at 
com.sun.tools.javac.comp.DeferredAttr$RecoveryDeferredTypeMap.typeOf(DeferredAttr.java:527)
     at 
com.sun.tools.javac.comp.DeferredAttr$DeferredTypeMap.apply(DeferredAttr.java:455)
     at 
com.sun.tools.javac.comp.DeferredAttr$SpeculativeDeferredTypeMap.apply(DeferredAttr.java:465)
     at com.sun.tools.javac.code.Type.map(Type.java:158)
     at 
com.sun.tools.javac.comp.Resolve$5.getArgumentTypes(Resolve.java:1888)
     at com.sun.tools.javac.comp.Resolve.accessInternal(Resolve.java:1803)
     at com.sun.tools.javac.comp.Resolve.accessMethod(Resolve.java:1823)
     at com.sun.tools.javac.comp.Resolve.accessMethod(Resolve.java:1835)
     at com.sun.tools.javac.comp.Resolve.resolveMethod(Resolve.java:2001)
     at com.sun.tools.javac.comp.Attr.visitIdent(Attr.java:2640)
     at com.sun.tools.javac.tree.JCTree$JCIdent.accept(JCTree.java:1918)
     at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:598)
     at com.sun.tools.javac.comp.Attr.visitApply(Attr.java:1819)
     at 
com.sun.tools.javac.tree.JCTree$JCMethodInvocation.accept(JCTree.java:1393)
     at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:598)
     at com.sun.tools.javac.comp.Attr.visitReturn(Attr.java:1683)
     at com.sun.tools.javac.tree.JCTree$JCReturn.accept(JCTree.java:1312)
     at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:598)
     at com.sun.tools.javac.comp.Attr.attribStat(Attr.java:641)
     at com.sun.tools.javac.comp.Attr.attribStats(Attr.java:657)
     at com.sun.tools.javac.comp.Attr.visitBlock(Attr.java:1084)
     at com.sun.tools.javac.tree.JCTree$JCBlock.accept(JCTree.java:837)
     at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:598)
     at com.sun.tools.javac.comp.Attr.attribStat(Attr.java:641)
     at com.sun.tools.javac.comp.Attr.visitMethodDef(Attr.java:999)
     at com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:721)
     at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:598)
     at com.sun.tools.javac.comp.Attr.attribStat(Attr.java:641)
     at com.sun.tools.javac.comp.Attr.attribClassBody(Attr.java:3841)
     at com.sun.tools.javac.comp.Attr.attribClass(Attr.java:3760)
     at com.sun.tools.javac.comp.Attr.attribClass(Attr.java:3694)
     at com.sun.tools.javac.comp.Attr.visitClassDef(Attr.java:865)
     at com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:643)
     at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:598)
     at com.sun.tools.javac.comp.Attr.attribStat(Attr.java:641)
     at com.sun.tools.javac.comp.Attr.attribClassBody(Attr.java:3841)
     at com.sun.tools.javac.comp.Attr.attribClass(Attr.java:3760)
     at com.sun.tools.javac.comp.Attr.attribClass(Attr.java:3694)
     at com.sun.tools.javac.comp.Attr.attrib(Attr.java:3668)
     at 
com.sun.tools.javac.main.JavaCompiler.attribute(JavaCompiler.java:1206)
     at 
com.sun.tools.javac.main.JavaCompiler.compile2(JavaCompiler.java:892)
     at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:851)
     at com.sun.tools.javac.main.Main.compile(Main.java:441)
     at com.sun.tools.javac.main.Main.compile(Main.java:358)
     at com.sun.tools.javac.main.Main.compile(Main.java:347)
     at com.sun.tools.javac.main.Main.compile(Main.java:338)
     at com.sun.tools.javac.Main.compile(Main.java:76)
     at com.sun.tools.javac.Main.main(Main.java:61)

>
>
>
> On Oct 12, 2012, at 9:32 PM, Kin-man Chung wrote:
>
>> The use of the operation flatMap exposes the Block interface.  For
>> instance, if one wants to get the list of all orders from all customers,
>> one writes
>>
>>      customers.stream().flatMap((s,c) ->
>> c.getOrders().forEach(o->s.apply(o))
>>
>> In this case, the purpose of the parameter s is for buffering and
>> streaming the elements and can really be hidden from the user.  ( BTW,
>> if we have "yield", we can write (c->c.getOrders().forEach(o->yield o)),
>> but that's for another discussion.)
>>
>> I admit that the current flatMap is very general and powerful, but I
>> think it is more common to flatten elements that are Streamable.  Is it
>> possible to add another flavor of flaMap that takes a function that
>> returns a Streamable?  If so, the above example can be simplified to
>>
>>      customers.stream().flatMap(c -> c.getOrders())
>>
>> which is much more readable.
>>
>