type variables from enclosing types?

[Stephan Herrmann]

Hi,

I couldn't find how type inference would handle type variables
of an enclosing type.

Looking at this example:

   class Outer<O> {
     class Inner {}

     static <I> void method(Outer<I>.Inner x) {}
   }

   public class ExtendedOuter<E> extends Outer<E> {
     class ExtendedInner extends Inner {
       void foo() {
         method(this);
       }
     }
   }

Current compilers accept this program, but playing strictly
by the inference spec, I get funny results:

1) We have an initial constraint ⟨this -> Outer<I#0>.Inner⟩
where a naive implementation might consider "Inner"
as a proper type. Should a note be added to the definition
of "proper type", to the effect that enclosing types
(explicit or implicit) should also be considered when looking
for a mention of any inference variables?
Otherwise the initial constraint directly reduces to FALSE.

2) If we interpret Outer<I#0>.Inner as an improper type
the inference variable I#0 will be inferred to Object,
but that seems to be the wrong outcome:
  - "this" is an instance of ExtendedOuter<E>.ExtendedInner
  - thus I#0 should be inferred to be E

3) Interestingly, when I modify the program so that the
inference of I#0 to Object would trigger an error, the
usage of I forces the inference into better results.
Does this mean, the "wrong" inference result is acceptable
because it is irrelevant to the current inference task?

4) I made experiments with pulling E into the inference
by augmenting 18.2.3 bullet 5.2: I added corresponding
constraints also for the type variables of enclosing types
of C<A1, ..., An> and C<B1, ..., Bn>. This directed the
inference into the expected solution, but then it caused
regressions in situations of nested diamonds like
   new <>Outer.<>Inner()
   new <X>Outer.<>Inner()
I haven't yet analyzed the details of these regressions,
though.


Should some rules be added for handling type variables from
enclosing types, or can we rely on the assumption that type
variables that are only implicitly referenced need not be
resolved to their optimal solution?


thanks,
Stephan