"Model 2" prototype status

[Brian Goetz]

> Thanks for the hard work on this.
> The Foo<ref> looks like a nice parallel to Foo<any> to my eyes.

Thanks!

Some more on where we're going with this.  You should be able to say 
"ref" wherever you can say "any", so:

   class Foo<T> { ... }

becomes a shorthand for

   class Foo<ref T> { ... }

(just as Foo<?> has always been shorthand for Foo<? extends Object?>.)

Also, the details of translation mean that the "all any" and "all ref" 
instantiations of a type with multiple tvars are somewhat "special", so 
for a type Foo<any X, any Y, any Z>, we'll probably accept

   Foo<any>

as shorthand for Foo<any, any, any>

and

   Foo<ref>

as shorthand for Foo<ref, ref, ref>

Since the arity is fixed and known at compile time, there's no problem 
with this; there's no other logical meaning to instantiating with the 
wrong number of type args.

(Note: The compiler doesn't yet support these yet.)

The details of "partial wildcards" (Foo<int, any>) are messy and we're 
working on a story for them.

>
> Stephen
> On 31 Jul 2015 20:51, "Brian Goetz" <brian.goetz at oracle.com> wrote:
>
>> Over the past few months, we've been building what we call our "Model 2"
>> prototype, and incrementally checking it in.  It now seems stable enough
>> for brave early adopters to check out.  (You'll need to build the JDK from
>> source.)
>>
>> The previous version ("Model 1") was focused on exploring the practicality
>> of specialization on the JVM architecture we have today. The compiler
>> augmented classes with type metadata that otherwise would be erased away,
>> which could be ignored by the JVM but consumed by the specializer.
>> Specialized classes were identified using a name mangling scheme (strictly
>> an expedience for prototyping, not a long-term plan.) The class loader
>> recognizes the name mangling scheme and, if not found in the class path,
>> the class loader invokes the specializer to generate the specialized class
>> on the fly.
>>
>> With these many hacks (name mangling, abuse of class loaders), the result
>> was mixed.  On the good side, it worked!  It was possible to write
>> specializable generic classes and run them on an only-lightly-hacked JVM.
>> On the bad, the resulting language had a significant usability issue -- the
>> lack of a nontrivial common supertype between Foo<int> and Foo<String>.
>>
>> Of course, we didn't pursue this approach because we thought half-killing
>> wildcards was a great idea; we pursued it because it was what worked on the
>> JVM we had.  So with the Model 1 prototype in hand, we set out to see what
>> could be done about providing a reasonable common supertype for all
>> instantiations of an any-generic type.  (We explored a number of possible
>> mechanisms and approaches, including several that are more radical than
>> where we landed.  Hopefully we will find time to write up some of these
>> roads-not-taken.)
>>
>>
>> Type variables.  Type variables are divided into two categories; "ref"
>> (ordinary) and "any".  Any tvars ("avars") are identified by the keyword
>> "any" at their declaration site (for both generic classes and generic
>> methods).  If a type variable is not modified by "any", it is implicitly an
>> ordinary tvar, and treated just as in Java 8.
>>
>>      class Foo<T> { ... }      // T is an ordinary tvar
>>      class Bar<any T> { ... }  // T is an avar
>>
>> Class hierarchies can be any-fied from the top down.  So it's OK to do:
>>
>>      class A<any T> { ... }
>>      class B<T> extends A<T> { ... } // T is an ordinary tvar here
>>
>> but not OK to do:
>>
>>   class A<T> { ... }
>>   class B<any T> extends A<T> { ... }
>>
>> The rationale for this should be clear enough; specializing a class
>> entails specializing its superclasses, and if the superclass is not
>> specializable, this won't work.  (Alternately, you can interpret "any T" as
>> a union bound ("T extends Object | value"), and its OK to use a narrower
>> bound than your supertype, but not a wider one.)
>>
>>
>> Restrictions on avars.  Some operations that are allowed on ordinary tvars
>> are not allowed on avars, such as assigning a T to an Object, assigning
>> null to a T, etc.  These have not changed from Model 1.
>>
>>
>> Wildcards.  The big change in Model 2 is the addition of support for
>> wildcards over avars.  The problem with wildcards has two facets;
>> translational (how do we represent a wildcard type over avars in bytecode?)
>> and interpretation (Foo<?> has always been a shorthand for Foo<? extends
>> Object>; on the other hand, the "intuitive" intepretation of Foo<?> is "any
>> instantiation of Foo.")  The translational issues require some help from
>> the JVM to solve (not yet implemented in the prototype.)  The interpretive
>> issues are subtle.  While we explored trying to automatically interpret
>> Foo<?> according to the common user intuition, this ran afoul of numerous
>> compatibility issues.
>>
>> So, where we landed is: just as one must specify any-ness at the
>> declaration site for a type variable, one must do the same for a wildcard
>> (which is essentially declaring an anonymous type variable.) So there are
>> two forms of wildcard:
>>
>>     Foo<ref> -- describes any reference instantiation
>>     Foo<any> -- describes any instantiation
>>
>> and Foo<?> will be retconned to mean Foo<ref>.
>>
>> Raw types. Raw types have not changed at all from Java 8. As such, they
>> are limited to reference instantiations.
>>
>> The upshot of this is we can achieve perfect source compatibility; the raw
>> type Foo and the wildcard type Foo<?> continues to mean exactly what they
>> always did.
>>
>>
>> As a proof of concept, I've checked in a limited version of Streams
>> (java.anyutil.Stream) that has been ported to the new model.  (Some things
>> are still not hooked up yet, but the basic functionality for Stream<any T>
>> works.)
>>
>>
>> The Model 2 approach needs some (targeted) help from the VM, which is not
>> yet in place.  (Specifically, protected/package methods in interfaces, and
>> some help for access to private members across specializations of the same
>> class.)  Until we have this, protected/package methods in anyfied classes
>> will be problematic.
>>
>> There are also some compiler limitations, the most significant of which is
>> that inner classes don't yet work as instantiations of anyfied classes; you
>> have to refactor the inner class to a static named class. (There's no
>> fundamental issue here, its just not done yet.)
>>
>> Most helpfully, the latest IntelliJ has some early support for anyfied
>> generics!  (It's not perfect, but its been really helpful -- thanks guys.)
>> Select language level "X" (for experimental) and it will mostly deal with
>> any-tvars and any-wildcards, instead of painting your whole program red.
>>
>>
>> We'd like to get feedback from people *using the prototype* to anyfy their
>> own code.  (I'm sure people have a zillion "why did you / why didn't you"
>> questions; we'll try to get to those in a writeup.)  I'll also be putting
>> together a writeup on the translation strategy (though brave explorers can
>> and surely will reverse engineer this with javap before this happens.)
>>
>>
>>