"Model 2" prototype status

[Brian Goetz]

Hi John;

The prototype in the valhalla repo is fairly functional (though by no 
means complete).  So I'll answer these with some code.  Of course, this 
is still a prototype, so everything will probably change.

If I have:

class Box<any T> {
     public T t;

     public T get() { return t; }
}

Compilation (currently) produces a combination erased class / template 
file (Box.java), and a wildcard interface (Box$$any):

brian:tmp$ javap -c Box\$\$any.class
Compiled from "Box.java"
interface Box$$any {
   public abstract java.lang.Object get();

   public abstract java.lang.Object t$get();

   public abstract java.lang.Object t$set(java.lang.Object);
}

We see that methods, and accessors for the getter and setter for the 
field 't', are lifted to the interface.  Box<any> (represented at 
runtime by Box$$any) becomes the common supertype between all 
instantiations of Box<T>.

Note that the signatures have some mangled types, which looks like 
erasure here (but is not exactly the same as erasure, and hence needs a 
new name -- call it "anyrasure".)  Ideally, the anyrasure of a type 
would be a common supertype of all possible instantiations of that type, 
but there's no common supertype between "Object" and "int", so we'll 
settle for a common conversion target.

The anyrasure of a bare type variable is its erasure; a reference 
instantiation need make no conversions, and a value instantiation needs 
to apply a boxing conversion to implement the methods of Box$$any. 
Clients accessing members through a concretely typed receiver bypass the 
boxing path since both client and implementation will already agree on 
member types.

What about when Foo<T> shows up in a signature?  Compiling this:

class Box<any T> {
     Box<T> me() { return this; }
}

yields

interface Box$$any {
   public abstract Box$$any me();
}

showing that the anyrasure of Box<any T> is the wildcard type Box<any> 
(which is a common supertype).  As I mentioned in my talk, we are 
working on a story for the anyrasure of T[] as well (but is not yet 
implemented.)

Mapping wildcards to interfaces is imperfect.  It works great for 
capturing subtyping using tools the VM already understands (which means 
lots of things Just Work).  It also leverages invokeinterface, which is 
good.  However, there are some obvious imperfections in the story, which 
we're currently working on:

  - What about protected and package members?  (In theory we could 
support these accessibilities on interfaces -- the "consistency brigade" 
would certainly applaud this -- but these have issues too (i.e., what 
does it mean for a public interface to have a 'package' member?  Only 
classes in that package can implement the interface?  That's kind of 
weird.))

  - Even if we did the above, what about private members?  Private 
members are not inherited, so dispatching through invokeinterface won't 
work, so that needs another story regardless.

  - What about reflection?  Currently, the way to reflect over "all 
instantiations of Foo" is to reflect over Foo.class.  But Foo.class is 
only the erased instantiation, so people will want the equivalent of 
reflecting over Foo<any>.  (This is one place where lifting the members 
to the interface really shines; an indy-centric approach would require 
getting there another way.)

  - How do we enforce finality?  There *are* multiple legitimate classes 
that implement Foo<any T> -- Foo<int> is a different class than 
Foo<erased>.  How do we enforce that only "legitimate" instantiations 
are allowed to implement Foo$$any?

  - Even ignoring finality, how do we prevent impersonation?  If we 
represent Foo<any> by Foo$$any at runtime, how do we keep someone from 
simply implementing Foo$$any but not respecting the invariants of Foo, 
and passing the fake to a method expecting a Foo<any>?

  - What about wildcard capture?  We need a story for this; we're not 
working on this quite yet.

  - Non-anyfied superclasses.  The case "class Foo<any T> extends Moo" 
is an annoying corner case, which messes up our story for subtyping 
checks (member access is easily dealt with in one of several ways.)

In the happy cases, many operations on anyfied types proceed quite 
nicely, leaning on existing VM mechanisms (subtyping, invokeinterface, 
reflection), so they "just work".  All the work is dealing with the 
unhappy cases.

With a big enough crowbar(s), each of these issues can be dealt with, 
but we're mindful of the risk of ending up with a pile of unrelated hacks.

OK, back to work!


On 8/31/2015 10:04 PM, John Altidor wrote:
> Hi Brian,
>
> I am new to this mailing list.  My PhD dissertation covered subtyping with
> variance and Java wildcards extensively, so the questions you raised in
> this thread are very interesting to me.  I was wondering how you are
> handling the translational aspects of wildcards and specialized generic
> methods.
>
> Your earlier post asked how to represent List<any> in bytecode.  Since
> List<any> is a supertype of both List<int> and List<double>, for example,
> type List<any> should only support operations that can be applied to both
> List<int> and List<double>.  One such operation is counting the number of
> elements using method List.size().  Which byte representation would support
> being able to dispatch List.size() on both an instance of List<int> and an
> instance of List<double>?
>
> It seems such a byte representation would need to be independent of fields.
>    The number of bytes needed to represent an instance differs among
> primitive types (e.g. int and double).  As a result, it seems List<int> and
> List< double> may differ in the number of bytes needed for their fields.
> In that case, one could not know the number of bytes in the instance of
> List<any> returned from the following method:
>
> List<any> func(int input_num) {
>    if(input_num is odd)
>      return new List<int>();
>    else
>      return new List<double>();
> }
>
> In addition to type-independent methods such as List.size(), another
> operation that is type safe to allow on an instance of List<any> is
> wildcard capture.  Consider the generic method, swapFirstTwo, below that
> just swaps the order of the first two elements in the input list.  It is
> type safe to pass an instance of List<any> to this method (because no
> runtime type error would occur).
>
> <any T> void swapFirstTwo(List<T> list) {
>    T first = list.getAndRemoveFirst();
>    T second = list.getAndRemoveFirst();
>    list.addToBeginning(first);
>    list.addToBeginning(second);
> }
>
> Would two calls to method swapFirstTwo, one with a List<int> as input and
> the other method call with a List<double> as input, result in two
> specialized copies of method swapFirstTwo in byte code?  If that is the
> case, what is the byte representation of method swapFirstTwo when the input
> is an instance of List<any>?
>
> Thank you,
> John Altidor
> http://jgaltidor.github.io/
>