invokespecial of default methods in unrelated interfaces

[Dan Smith]

Thanks for taking the time to explore this in detail.  See my comments below.

On Jun 20, 2013, at 8:34 PM, Daniel Heidinga <Daniel_Heidinga at ca.ibm.com> wrote:

> Regarding the invokespecial to an unrelated interface, I agree this should be illegal.  At a minimum, it should be required that the interface targeted by the invokespecial must be compatible with the invoker as this prevents invoking methods in unrelated interfaces.
> 
> While thinking about this, I've also taken a look at how the VM can prevent "level skipping" while preserving both binary compatibility and programmer intent.  The picture is not pretty.  
> 
> After reading both your and Dan's emails, there is a subtle difference in the terms both of you are using:
> * "immediate supertype of the invoker" - Immediate supertype is not defined by the JVM spec but intuitively, a superinterface ("I") is not an immediate supertype if it is also a parent of another superinterface (J, J extends I).
> 
> * "direct superinterface" (Dan's term) - This is defined by the JVM spec 4.1 to be any interface declared in the classfile's interfaces[].  Using this definition, but I and J would be considered direct superfaces if they were declared in the interfaces[].  Using this definition means that redundant interface declarations are suddenly meaningful information. 

FWIW, I read "immediate supertype of the invoker" as "direct superinterface".  In any case, yes, there are two different ways we could define the set of valid invokespecial target interfaces.

> Starting with this simple example:
> interface J { default void m() {...}}
> interface I { default void m() {...}}
> interface K extends I, J { default void m() { I.super.m(); }
> K is a valid class that resolves the conflicting defenders by choosing to dispatch to I.m().
> If J is changed to extend I, a binary compatible change, then the question becomes "is I still a direct / immediate superinteface?" and a valid target for invokespecial?
> If I is not a valid target because it allows K to skip J.m(), then K will throw an AbstractMethodError and violate the binary compatibility requirements.

1) Let me point out that, inherent with multiple inheritance of behavior, there are certain refinements that have to be made to the concept of binary compatibility.  I don't know exactly what they look like, but the status quo will not do.

Example:
interface I { default void m() { ... } }
interface J {}
class C implements I, J {}
new C; invokevirtual C.m()V

Runs successfully.  Now add a default m()V method to J, or add a new superinterface to C that declares m()V (both binary compatible changes, per JLS 7), and there's an ambiguity error.

The details of exactly what JVM 8 binary compatibility looks like need to be explored in depth; I'm happy to consider any suggestions you may have.

2) That said, I do not think we should be trying to enforce the full language-level concept of no level skipping.  I'm happy to allow the named class to be any direct superinterface of the calling class.

> Enforcing "no level skipping" can also break programs if new defender implementations are added, which decreases our ability to evolve interfaces.
> interface A { default void m() {...}}
> interface B { }
// B extends A?
> interface C { }
// C extends A?
> interface D extends B, C { default void m() { C.super.m(); }
> Adding a new default implementation of m() to B will also result in AbstraceMethodError as D's super send would skip the new B.m().
> If resolution started at the targeted class 'C', as stated in the current 335 spec, then this would continue to call A.m() matching the intent at the time the code was written.  Unfortunately, this makes 'C' a "bypass interface" which allows level skipping by using dummy subinterfaces. 
> 
> (Note, a clever programmer can always target any particular interface's default implementation by adding a "bypass interface" that implements the required method using a super send to the actual desired interface - this is just an abuse of the standard way for users to fix defender conflicts.)

Again, bypassing B.m doesn't bother me.  I'm happy to leave this restriction to the language and allow the call to proceed in the VM.

> This seems to require the VM to ignore level skipping.  Unfortunately, there's holes here with respect to programmer intent because the targeted interface must be named:
> interface S { default void m() { } }
> class T implements S { }
> class U extends T implements S { void m() { S.super.m(); } }
> The programmer than refines T to target a interface V:
> interface V extends S { default void m() {...} }
> class T implements V { }
> And now class U will throw an AbstractMethodError.  This is the simple case, a single inheritance path, that might argue for some kind of interface analog to the ACC_SUPER search rules.
> 
> If the goal is to preserve binary compatibility - that pre-existing binaries continue to link without error whenever possible - then preventing level skipping in the VM is impossible.

That's okay with me.  As long as U directly implements S, it doesn't bother me to allow free access to the (accessible) methods of S.

> The following is a resolution algorithm for invokespecial that attempts to limit level skipping while preserving binary compatibility:
> 
> * Perform interface resolution using the set of interfaces implemented by the invoker of the invokespecial as defined in the JSR 335 JVM spec 5.4.3.4.  
> * Examine the set of candidate methods:
> ** If there is only 1 method, then resolution succeeds with that method (This assumes any super implementation meets the required contract)
> ** If there are more than 1 method, remove any method that is not provided by the the interface named in the invokespecial or subclass of it.
> *** If this results in a single method, resolution succeeds with that method.
> * Otherwise, redo method resolution starting at the interface named in the invokespecial and succeed or fail with the result of this resolution.
> 
> This uses two-passes to limit level skipping in cases when possible, such as the first example in this email, while preserving binary compatibility when the first pass would otherwise of failed.

I gather the effect you're after is to, where there is an unambiguous choice, select the "most specific method" that overrides the named method.  But this is seriously weakened (and rendered fairly unpredictable) by the fact that, where there is an ambiguous choice, the named method is chosen anyway.  If I've added a method in what I hope is a compatible manner, but that actually messes with resolution/selection of a method, I don't think I'll be happier with an unintuitive behavioral change than with a new error.

At the level of the VM, I think we should be making hard guarantees, or not trying to do anything at all.  Fuzzy "this is probably what you meant" logic doesn't belong here.

—Dan