RFR: 8336786: VerifyError with lambda capture and enclosing instance references

[Maurizio Cimadamore]

On Fri, 19 Jul 2024 16:46:53 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> I came up with something slightly different:
> https://github.com/openjdk/jdk/compare/master...mcimadamore:jdk:lambda_encl_capture?expand=1

Looking closer, IMHO dropping the `isMemberOf` from `LambdaToMethod` and replacing it with a simpler subclassing test on the captured symbol owner seems like it should work: after all, `Lower` has already decided whether to resolve via an enclosing `this` or not. 

If it did, it already has replaced the access to use some `this$0`. At which point, what check `LambdaToMethod` shouldn't matter much, as the synthetic `this$0` will always be accessed and preferred over just `this`.

If it didn't, well, then just `this` is fine. As your patch fixes the type of `this` to be the current class (which is correct). But then, no matter whether `Lower` used `isMemberOf` or `isSubclass`, the innermost enclosing type should always act as a capturing frame for `LambdaToMethod` (since we really want the current `this`). So again, using a simpler subclassing check would be fine.

In other words, I'm wondering whether the `isMemberOf` check in `LambdaToMethod` is also a relic of a time where `LambdaToMethod` used to run _before_ `Lower`, so it needed to make a choice of how to resolve the enclosing this.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20259#issuecomment-2239699673