Virtual Extension By Inversion of Responsibility (Second Draft)

[Collin Fagan]

Here is a plain text version of my proposal. Also if it maters, I've signed
the Sun agreement to contribute to OpenJDK.

Thanks,

Collin

-----------------


Virtual Extension By Inversion of Responsibility
Collin Fagan
Second draft, Jun 2010

Problem statement

    Once an API is published and in common use, it is difficult to evolve
the very top layers of an abstraction without breaking existing code.
Changes that would introduce new methods to an interface or new abstract
methods to an abstract class will break both source and binary
compatibility.
The addition of lambdas to the Java language in JDK 7 presents API designers
with this same problem on a much larger scale.

    Currently in Java the most straightforward approach would be to add new
interfaces to the API. These new interfaces can then be implemented by
concrete or abstract classes. The downside is that to use this new
functionality you must either program to this new interface or play the game
of "cast and hope". Neither immediately presents itself as a desirable
option.

Virtual Extension

    This document proposes a special "cast with fallback" mechanism. This
replaces the "hope" of the "casting and hoping" with the certainty of a
default implementation. This inverts the responsibility of providing this
new functionality to the extender and not the base class.

Example Syntax:

List<String> planets = Arrays.asList("Earth","Mars","Venus");
String result = planets->join(“,”);
assert(result == "Earth,Mars,Venus");
    Here join looks like a method call on the List interface using a funny
operator. The -> operator is syntactic sugar for the inconvenient
alternative code. The important part however is that the extension method is
a separate entity from the interface.

Other sugared examples:

// groovy style GString.execute()
"cp file1.txt file2.txt"->execute();

// log a severe exception
ex->severe();

// Null safe isEmpty from Commons Lang
String x = null;
assert(x->isEmpty());

// Lambda sort:
planets->sort(#(String a, String b)(a.length() - b.length()));
Possible Desugarings

Static Methods:

    One possible way to translate the -> operator is through delegation to a
static method.

    // Sugared
        String result = planets->join(“,”);

    // Unsugared
        String result = JoinUtils.join(planets, “,”);


    Lets assume we have the following interface.

    public interface Joinable{
        String join(String sep);
    }

    There needs to exist a mapping between the join method of this interface
and the static join method available in JoinUtils. Here is one way one could
declare such a relationship.

    @ExtentionSupport(supports=Joinable.class)
    public final class JoinUtils{
        ...
    }

    This could be a new keyword but I've chosen to use an annotation. Next
there needs to be a static method whose name and signature matches the join
signature with one extra parameter. This extra parameter is the "target" of
the method. By convention this could be the very first parameter., otherwise
another marker is required. This marker might also be useful for adapting
existing static methods.

    public static String join(@ExtentionTarget List<String> list, String
sep){
        ...
    }

    Now that we've indicated which parameter we want the -> operator to
apply to, we need a way to let the compiler know where to look for the join
method. Static methods already have a system for including them in the
current scope, static import.

    //Sugared

    import static JoinUtils.*;
    ...
    List<String> planets = Arrays.asList("Earth","Mars","Venus");
    String result = planets->join(“,”);
    assert(result == "Earth,Mars,Venus");
    ...

    // Unsugared

    import static Utils.*;
    ...
    List<String> planets = Arrays.asList("Earth","Mars","Venus");
    String result = join(planets, “,”);
    assert(result == "Earth,Mars,Venus");
    ...

    Finally there is the possibility that the "target" object might already
be an instance of Joinable. To support this case the final unsugared code
would need to check.

    // Unsugared

    import static Utils.*;
    ...
    List<String> planets = Arrays.asList("Earth","Mars","Venus");
    String result = null;

    if(o instanceof Joinable){
        result = ((Joinable)o).join(",");
    }else{
        result = join(planets, “,”);
    }
    assert(result == "Earth,Mars,Venus");
    ...

Pros
- No change required to any interface or class.
- Adding a keyword is not required.
- Many currently static methods could be retrofitted to work.
Cons
- Encourages static methods.
- Requires static import.
- Possibly not a huge savings in typing for the conceptual weight.
- Requires an instanceof and a cast

Delegating to a supporting object:

    Another possible desugaring of the -> operator is to delegate to a
supporting object. Below is an example of a "support object" for the
Joinable interface.

@ExtentionSupport(supports=Joinable.class)
public class JoinableSupport<E> implements Joinable{

    private Collection<E> targetCollection;

    public JoinableSupport(Collection<E> targetCollection) {
        this.targetCollection = targetCollection;
    }

    @Override
    public String join(String sep) {
        .. join logic ..
    }
}

Each support object would be required to have a single argument constructor.
This constructor argument would be a reference to the object this
functionality is being "applied" to.

// Sugared

List<String> planets = Arrays.asList("Earth","Mars","Venus");
String result = planets->Join(“,”);
assert(result == "Earth,Mars,Venus");

// Unsugared

List<String> planets = Arrays.asList("Earth","Mars","Venus");
String result = new JoinableSupport(planets).join(planets, “,”);
assert(result == "Earth,Mars,Venus");


// Sugared

List<String> planets = Arrays.asList("Earth","Mars","Venus");

planets->ForEach(#(String target){
    target->Print();
})

// Unsugared (with an SAM style lambda expansion *)

new ForEachSupport<String>().invoke(planets, new Block<String>(){
    @Override
    public void invoke(String target) {
        new Print().invoke(target, Void);
    }
});
* I'm not advocating that lambdas be implemented like this. I just need to
translate them to something remotely familiar to talk about extension
methods.

    Again there is the possibility that the "target" object might already be
an instance of Joinable. To support this case the final unsugared code would
need to check.

    // Unsugared
    ...
    List<String> planets = Arrays.asList("Earth","Mars","Venus");
    String result = null;

    if(o instanceof Joinable){
        result = ((Joinable)o).join(",");
    }else{
        result = new JoinableSupport(planets).join(“,”);
    }
    assert(result == "Earth,Mars,Venus");
    ...

Pros
- No change required to any interface or class.
- Adding a keyword is not required.
- Does not encourages static methods.
- Does not requires static import.

Cons
- Need to manage number of objects created.
- Assumes that importing a "support class" makes it available to the ->
operator.

Lambda Delegation

    If lambdas exist then another desurgaring becomes possible, an Extension
Lambda.

// Sugared

List<String> planets = Arrays.asList("Earth","Mars","Venus");

planets->forEach(#(String target){
    target->print();
});

// Unsugared (to lambda):

List<String> planets = Arrays.asList("Earth","Mars","Venus");

forEach.(planets, #(String target){
    print.(target);
});

// Lambda declaration
#void(@ExtentionTarget Collection<T>, #(T)) forEach<T> =
    #(Collection<T> collection, #(T) lambda){
    for(T item: collection){
        lambda.(item);
    }
};

    We again use an annotation on the parameter we want the -> operator to
bind to. One thing not explicitly stated in the straw man proposal is
whether a lambda is allowed as a top level construct. Currently Java
supports class, interfaces and enum. If lambdas are treated as top level
constructs then they could be imported directly. If not then we might have
to go back to statically importing some public static final fields that
refer to lambdas. Also not specified in the straw man proposal is the
interaction of lambdas and polymorphism. Is it possible for a subclass to
override the default implementation of a lambda?

Pros
  No change required to any interface or class.
  Adding a keyword is not required.
  Creates a similar amount of overhead as a lambda.
Cons
    May require static import?
    How can you override a lambda in a subclass?

Final thoughts

    All three transformations are an attempt to remove the need to evolve
interfaces. Each try to push the responsibility into more flexible
constructs. I consider nothing final, not the syntax, not the details of the
de-sugaring, not anything. I hope this document will spur positive
discussion in this direction and I welcome all comments.



On Fri, Jun 18, 2010 at 1:58 PM, Alex Buckley <alex.buckley at oracle.com>wrote:

> Hi Collin,
>
> If lambda-dev is to discuss your proposal, you will need to send the text
> of the proposal to lambda-dev, rather than pointing to an external site.
> Thanks.
>
> Alex
>
>
> On 6/17/2010 8:09 PM, Collin Fagan wrote:
>
>> Hi Everyone,
>>
>> I've put together a second draft of my proposal for virtual extension
>> methods.
>>
>> http://www.box.net/shared/r24vvgma5p
>>
>> And yes *this* time they are really virtual. I welcome all comments.
>>
>> Thanks,
>>
>> Collin
>>
>>