RFR: 8334121: Anonymous class capturing two enclosing instances fails to compile

[Vicente Romero]

On Wed, 26 Jun 2024 10:04:18 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> This PR contains a fix for an issue when translating local/anon classes. Currently, javac assumes that a local/anon class C must have an enclosing instance whose type is the innermost enclosing class of C. But this is not always true - there are cases where the innermost enclosing instance is in early construction context, so we can't really refer to its members from the local class. Moreover, even if the innermost class _was_ accessible, there could be cases where the enclosing instance of that innermost enclosing class is _not_ accessible.
> 
> In other words, we cannot rely on the assumption that, given a local/anonymous class, there exist a chain of enclosing instances from the innermost to the outermost. Some chain exists, but it might not start from the innermost enclosing class, and it might have holes.
> 
> The core of the fix is the addition of a new function in `Symbol`, namely `Symbol::innermostAccessibleEnclosingClass`. As the name implies, the goal of this function is to return the class symbol corresponding to the innermost enclosing type **that is accessible** from the symbol's class. This is used by `Lower` to determine the type of `this$0`, which allows us to construct a chain of enclosing instances that skips over all the inaccessible types.
> 
> I've also updated the `Symbol::hasOuterInstance` method, so that it, too, will skip over inaccessible enclosing type (and only return `true` if some accessible enclosing type is found).
> 
> This cleanup brings a lot of simplifications to `Lower.FreeVarCollector`. That class is used to compute the set of captured variables inside a local/anon class. There is a workaround in this class so that if a local/anon class occurs in a pre-construction context (where the enclosing `this` is not accessible), we say that the local/anon class captures the enclosing constructor parameter of the place where the local/anon class occurs (e.g. the enclosing `'this`'s enclosing `this`). This adds complexity to the code, as now the set of captured variables depends on the state of `Lower`. With this PR, all that logic is now gone, and `FreeVarCollector` is effectively a stateless visitor.

lgtm

-------------

Marked as reviewed by vromero (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19900#pullrequestreview-2148601137