Fwd: Proposal for generics over primitives needs a rethink

[Gavin King]

Inline:


On Wed, Dec 31, 2014 at 11:33 PM, Brian Goetz <brian.goetz at oracle.com> wrote:

> *Obviously* it would be more desirable to integrate primitives and values
> into generics by leaning on the existing notion of type bound, rather than
> introducing all the additional complexity that we're considering.

Great! So then if everyone agrees that this is the best and most
correct solution, then the only problem that remains is: how do we
implement it!

Excellent :-)

> Where the wheels start to come off the wagon is: how do we represent a
> variable of type 'Any" in bytecode (a field, local variable, or method
> parameter or return type)?  If we can't answer that, we can't allow use of
> Any in these places.  And solving this problem amounts to only slightly less
> than a total redesign of the JVM and bytecode architecture.  So this
> harmless-seeming question (couched in claims of "simpler" and "more
> elegant") amount to "Why not just redesign the VM completely."

Well, it seems to me that there's at least two reasonable solutions to
this problem, as I'll outline below.

> Languages that have attempted to unify primitives and references on the JVM,
> with the existing bytecode architecture, while retaining some sort of
> compatibility with existing Java idioms, have failed at doing so. (And I am
> thankful to have those experiments to inform our work here!) As a concrete
> example, I point you to Paul Philips' excellent "Scala War Stories" talk
> from JVM Language Summit 2013, which covers the failure of such unifictions,
> and more:
>   http://medianetwork.oracle.com/video/player/2623635250001

Yeah look, I don't want to get hung up on the problems of Scala since
Scala's problems are already well known and I was a skeptic of Scala
before it was fashionable. Fortunately it's pretty much a strawman in
this context since I'm about the last person on earth to advocate
Scala as a model for Java.

> But you might say "Wait a second, C# managed to pull off this redesign of
> the VM to support polymorphism over objects and primitives".  And indeed
> they did, and overall their solution is quite elegant.  And obviously, we
> must have known about this example, so why wouldn't we explore this?

I'm not an expert on C#, and my suggestion isn't influenced by C#. So
let's also put C# aside, and focus on what is best and most correct
for _Java_.

> We're concerned with it too, as I think we've made quite clear.

Great! So let's fix it.

> Here's the
> position we're in: if we wait until we have a complete, 100% solution before
> sharing anything, people throw rocks at us for doing everything behind
> closed doors, but if we share our working thoughts in progress, people throw
> rocks at us for being half-baked.  We've chosen the latter poison, so by all
> means, throw your rocks, but don't kid yourself that you've spotted
> something that no one else has.  (And please, check the attitude at the
> door, it's just not helpful.)

Look, if this document was merely a thought bubble, then I'll go back
to sleep. But AFAICT, and correct me if I'm wrong, but that's actually
not the case. This is, if I'm not mistaken, the second public revision
of an actual proposal, and it looks like you're actually defending the
approach proposed in the document. I mean, the only reason for
publishing a proposal in public is to solicit feedback, and so,
therefore, I'm giving you my well-intentioned feedback. Whatever you
perceive my attitude to be is utterly irrelevant; either I'm right or
I'm wrong, and my perceived "attitude", whatever that might be, is not
an issue of interest to the Java community. So let's just set that
distraction to one side, please.

>> What this proposal does is introduce parametric polymorphism over
>> primitive types, while leaving it impossible to abstract over
>> primitives and reference types with subtype polymorphism. Thus, at the
>> intersection of the two systems of abstraction, namely, *variance*, we
>> get the broken behavior that a List<int> isn't a List<?>.
>
>
> Indeed, we've already pointed this out, and its not pretty.  All
> constructive suggestions accepted.  But implicitly dropping a key
> requirement (like gradual migration compatibility), and then claiming
> there's an obvious answer, is not really helpful.

I have not proposed to drop any requirement for backward compatibility.

>> I therefore suggest a different, simpler, and much more natural
>> starting point for this work: stop pretending that there is no type
>> Any.
>
>
> This is a particularly funny way to put it, as it is the notion that there
> *is* an Any type which requires pretending!  There is simply no way (without
> boxing) to represent this on the JVM as it currently stands.

So what I'm going to do is offer you *two* solutions. One works with
the JVM with a fairly small modification, and one requires that the
JVM be enhanced to be more useful and more powerful.

>  But if boxing
> were good enough, then we wouldn't need to do anything --we'd just write
> ArrayList<Integer> and be done with it!  But obviously boxing isn't good
> enough, since we're having this conversation.  ...  Where
> the data hits the heap is where the boxing story (and therefore the Any
> story) falls apart.

So, the first possibility, as you've intuited, is that assignment of a
primitive to Any results in boxing, as it does today when you assign a
primitive to Object.

*However*, this approach actually does not result in the problem
you've described. Consider the possible concrete instantiations of
List:

- List<Any> - boxing, like today with List<Object>
- List<Object> - same as today
- List<int> - no boxing, since we can use the same specialization
approach used in the paper

The only case which involves boxing is a case which you disallow in
the current proposal, which means that this is not worse than the
current proposal.

Now, the downside of this is that it does motivate a relatively small
change to the VM where invokeinterface would be applicable to both the
reference and primitive specialization of List, since, for an
invocation of the instantiation List<? extends Any>, we have something
that might be an instance of List<Object> or might be an instance of
List<int>. I don't sounds like an unreasonable thing to contemplate.

(Of course, there are also more complicated solutions which don't
require a change to the VM here, but they're uglier, and I think the
best place to solve the problem is at the VM level.)

The second solution is harder to implement but more useful. Since
about the 1970's, some programming languages have had the notion of a
"tagged" or "discriminated" union type. If you look closely, Any is
just a special case of a discriminated union. So if the VM were to add
an Any type, implemented as a discriminated union, no boxing would be
required when working with Any.

> (but please, constructively).

Do you a deal: I'll be constructive, as usual, and you'll stop being
defensive? OK?

> And, feel free to prove us wrong!  Try implementing the changes you are
> envisioning in the JVM, and show how they can get us to the goal!

No, I'm not going to implement this myself. I already have much more
than a fulltime job with Ceylon. And, frankly, it's unreasonable for
Oracle to demand that any criticism/feedback of its proposals for
enhancements to the Java platform be accompanied by implementation. I
don't expect my users to submit a patch with every bug report /
feature request, and neither should you.



--
Gavin King
gavin at ceylon-lang.org
http://profiles.google.com/gavin.king
http://ceylon-lang.org
http://hibernate.org
http://seamframework.org