Defender methods and compatibility

[Neal Gafter]

On Tue, Nov 30, 2010 at 2:48 AM, Reinier Zwitserloot <
reinier at zwitserloot.com> wrote:

> On Mon, Nov 29, 2010 at 11:28 PM, Neal Gafter <neal at gafter.com> wrote:
>
>  To be (hopefully) more clear, here are the costs of the current approach:
>> o New syntax for extension methods (as opposed to using the existing
>> method body syntax)
>>
>
> Not related. If at some point the ability to specify the default directly
> as a method body in the interface is added, there's still a default of some
> sort.
>

It is related in the sense that the new syntactic footprint adds nothing
else than the ability to take the identity of the default into account in
the semantics.  Without doing that, there is no point in adding the new
syntactic form to the language.  I consider it a disadvantage to add to the
language's syntactic footprint without any purpose.


>
> o Changing the default of an extension method is not source compatible (in
>> the sense that it can cause some existing clients to no longer compile)
>>
>
> This is a trivial detail that's not worth holding back progress for
>

Was somebody proposing to hold back progress?  This is actually a critical
detail, the resolution of which will advance the purpose of this language
change.

, for two reasons. (1) Such a change leading to an actual compile error for
> existing code is exceedingly rare.
>

Either that or perhaps it is exceedingly common.  We don't have enough
experience with the feature to really know.  The nature of the
incompatibility is unlike any existing incompatibility, among other reasons
that overload resolution is used to find the matching method.

(2) In the few cases where the maintainer of the interface realizes this
> could become a problem, he can change the default implementation instead.
>

There are three problems with this.  First, the maintainer is likely not to
realize there is a problem.  Changing an extension method's default is
nearly *always* source incompatible in the sense that there can exist
clients that would break.  Second, it might not be possible to change the
implementation of the default's body because the programmer doesn't control
the sources of the default's body.  See my next paragraph for a simple
example of this.  Third, the new default may need to have a more specific
receiver type, which is very commonly the case when improving the
performance of an API.

That would be an entirely source (and binary) compatible change. This is not
> of course possible if this maintainer does not control the default method,
> but I again ask for practical situations where that would be the case. For
> example, List will possibly get a sort() method, defaulted to
> Collections.sort. Both List and Collections are in the same "module" -
> whomever has the ability to change the one has the ability to change the
> other.
>

The simplest example would be Google releasing extended collection
interfaces that use as defaults static methods that are part of the JDK.

Or consider a more elaborate example: First, Google provides the
third-generation "collections" APIs with static methods for operations that
Oracle did not include as extension methods.

Then, someone (say, you) defines extension interfaces that use these as
defaults.

Later, Google improves the implementation of these methods by providing
strictly more specific, behaviorally compatible static overloads.  That is
normally a source and binary compatible change.  However, because the
defaults are specified in your sources by name only, when you recompile you
get the new, improved implementation as your default in only those
interfaces that are subtypes of the new first argument (receiver) type.
This can cause your sources to compile just fine if the conflict doesn't
arise in your source.  But it can break the source code of third parties
that implement your extension interfaces.

Who do we blame for the problem?  Clients of your interfaces would blame
you, but you'd rightfully point out that there was no potential issue when
you first created your interfaces.  You might blame Google for adding
overloads to existing methods, but Google would rightfully point out that
the new overloads are strictly more specific than the old ones and
behaviorally compatible (only more efficient).  You might blame clients of
your interfaces for inheriting in a diamond pattern, but the ability to do
so is one of the principal benefits of interfaces over classes.

The root of the problem is the language specification that enables such
incompatibilities to arise so late in the game for seemingly innocuous
reasons.  It would have been better for the defaults method bodies to appear
inline, so that any client who inherits in a diamond pattern will be forced
to explicitly disambiguate potentially conflicting default methods on the
very first compile.

o Under the current draft spec, not binary compatible either, but there have
>> been promised (as yet unexplained) complexities to be added to method
>> resolution (and therefore the JVM) that may be introduced to reduce this
>> effect.
>>
>>
> Not relevant either:
>

It is relevant, because keeping the platform simpler is better, all other
things being equal.


> Reason 3: Unless you're talking about doing away with the "default X"
> mechanism entirely and going with "supply body inline", how do you propose
> to fix this? You're doing the same thing: You insinuate a way to get rid of
> the complexities without explaining how.
>

I would prefer to see the body inline, but it could also be fixed by
removing reference to the identity of the default in the specification.


>  Here are the benefits:
>> o Certain compile-time ambiguities might be avoided when extension methods
>> in unrelated classes have the same signature and implementation.
>>
>
> Surely you mean unrelated /interfaces/, not classes. That's a rather
> creative definition of "unrelated". If the default implementation is the
> same, odds are the 2 interfaces are related (such as: are from the same
> "module", i.e. List and Map).
>

I'm not suggesting we design the language based on the odds, but based on
sound software engineering and language design principles.  In any case, I
believe you're wrong, as my example above suggests.


>  The default of an extension method is not a part of the programmer-visible
>> API in any useful, observable sense *except to the extent that it can
>> introduce this incompatibility.*
>>
>>
>  Vs. not introducing it, in which case the potential incompatibility is
> turned into a sure thing.
>

Actually, without this special treatment there is never be an ambiguity
created by changing an extension method's implementation.