An alternative to explicit template specialization: explicit versioning

[Brian Goetz]

Hi Jonathan;

I totally understand your reaction of "why does any of this have to be 
explicit, wouldn't it be great if it could be implicit?"

First, let me reassure you that we have the same concerns.  *OF COURSE* 
it would be nicer if you could magically take existing generics and 
extend the domain of their type arguments to include primitives and 
values.  If we could, we would.  And maybe we'll be able to; right now, 
we don't see a way to do that without introducing worse complexity, but 
we're just getting started on a long road and I predict that almost 
everything that we've got on the table now will have changed in some way 
by the time we're done.

There's quite a tangle of issues here, so let me try to quickly untangle 
some of them.

  - The goal of the project is to have efficient generics over 
primitives and values.  (If we didn't care about efficiency, we'd 
already be done; generics over boxed primitives are acceptably 
expressive now, and do not have these corner cases we're concerned about.)

  - The migration potholes like remove(T)/remove(int) are just that -- 
migration potholes.  They will need to be paved over.  If we can do so 
compatibly without introducing language features, so much the better; I 
am no fan of the complexity of conditional methods, nor do I like the 
idea of adding language features that we hope will be very infrequently 
used.  So we're looking for the minimal-complexity way to pave over 
these potholes in a responsible way.

  - We did explore an explicit versioning approach in the early days of 
default methods.  Once you take into account all the real requirements, 
including separate compilation and version skew between compiler and 
runtime, this gets quite complex indeed, and we abandoned this direction 
as being more complexity than was needed to address the real goal -- 
which was interface evolution.  I can't say for sure whether we'd reach 
the same conclusion with respect to the current problem, but it seems 
kind of likely.

  - There are places where user control over specialization is in fact a 
positive thing.  In our discussions with .NET users, for example, we 
found a significant pain point with their generics was that it was 
impossible to provide a different implementation for foo() in Foo<T> 
based on whether the receiver had T as a reference or value type. 
Auto-specialization is nice, but I can imagine a better implementation 
of ArrayList<boolean> than the obvious specialization, and it might be 
nice to let API implementors do so in a way that's transparent to users. 
  Etc etc.

I do think your assumption that the complexity of fine-grained 
versioning is less than that of specialization is not likely to survive 
deep exploration; I think its one of those situations where its easy to 
write the easy cases on the board and conclude its easy enough, and 
still be several PhD theses away from a good answer.  We're just farther 
along in the specialization analysis so the warts of that approach are 
more evident than the warts of a less-explored approach.

I do sympathize with your "why should the user even have to know about 
this" reaction; we hope we can lower the maintenance burden.  Right now, 
though, we're a long way from making any of those decisions -- we're 
mostly trying to figure out if and how the approach works.

Cheers,
-Brian

On 10/30/2014 12:43 PM, Jonathan C. Ross wrote:
> Hi all,
>
> Regarding the discussion on explicit template specialization, I find myself to be in the all-or-nothing camp: either we introduce the general mechanism, or we don’t expose any explicit form to the end user.
>
> Assuming that the project is not aiming to redo generics and introduce language support for specializing templates for arbitrary types, I find myself wondering whether there are in fact any compelling usecases for exposing template specialization to java programmers, and not to do our upmost best to keep it hidden?  The only reason I can come up with (and the example discussed in the state of the specialization document) is the usual one: preserving backwards compatibility. Assuming that the motivation is indeed primarily the preservation of backwards compatibility, has anyone explored any other mechanisms for doing that?
>
> One possibility mechanism might be a beefed-up, explicit deprecation mechanism that couples the visibility of methods to the class file version of the client code.  (Dr. Deprecation - Stuart Marks - might like this. It could be used as a mechanism for actually removing code that was marked deprecated in previous versions of the JVM.)  So, taking the example from the state of the specialization document:
>
>      interface ListLike<T> {
>          @Removed(“1.9”) // pay no attention to exact syntax
>          void remove(int i);
>          @Removed(“1.9")
>          void remove(T t);
>
>          @New(1.9") // maybe implied by overloading the previous declaration?
>          void remove(T t);
>          void removeByIndex();
>      }
>
> The methods annotated @Removed(“1.9”), would only be visible to classes with class version 34 or lower, the methods annotated with @New(“1.9”) only from 35 and up.  (Please don’t pay any attention to syntax, or indeed whether we will get this shipped with 1.9...) This mechanism should be easy enough to extend to classes and member variables too.  It would also be applicable to any other project where one would rather redesign the API.  From my perspective as a developer, it would be more intuitive than the template specialization alternative presented in the ‘state of the specialization’ document.
>
> The mechanism would allow library developers to ship class files that would support different multiple versions of the JVM at once, or they could focus on just a single JVM if they wanted.  If I want to implement ListLike<T> for just 1.9 and up in the example above, I only have to implement two methods from the interface; same if I want to stick to 1.8 and below.  IDEs could help a hand by (optionally) only showing available API’s, in particular when offering autocomplete suggestions.
>
> I am no byte code or class file layout expert, but I am sure it would be possible to implement this idea using attributes that mirror the Removed/New annotations. Perhaps having multiple versions of methods/fields/classes with exactly the same signature, differing only by the supported class file versions (like in the example above) would be a tougher nut to crack, but I would imagine that it would be a usable mechanism even with the constraint in place that one cannot replace a method/class/field with exactly the same signature.
>
> As an ‘end user’, I would find it more appealing and intuitive to work with versioned API’s than with the - for any given JDK version - much larger API that would be required to retrofit API’s to work for <any T>.  As a stakeholder in the further development of the Java programming language, I would be excited by a mechanism that would allow JDK engineers to throw out the old, and be able to innovate without worrying about breaking existing code.
>
> Thoughts?
>
>       Jonathan Ross
>
>
> ________________________________
>
> The information in this e-mail is intended only for the person or entity to which it is addressed.
>
> It may contain confidential and /or privileged material. If someone other than the intended recipient should receive this e-mail, he / she shall not be entitled to read, disseminate, disclose or duplicate it.
>
> If you receive this e-mail unintentionally, please inform us immediately by "reply" and then delete it from your system. Although this information has been compiled with great care, neither IMC Financial Markets & Asset Management nor any of its related entities shall accept any responsibility for any errors, omissions or other inaccuracies in this information or for the consequences thereof, nor shall it be bound in any way by the contents of this e-mail or its attachments. In the event of incomplete or incorrect transmission, please return the e-mail to the sender and permanently delete this message and any attachments.
>
> Messages and attachments are scanned for all known viruses. Always scan attachments before opening them.
>