API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[David Holmes]

Hi Karen,

On 16/02/2018 8:01 AM, Karen Kinnear wrote:
> David,
> 
> I think that is a much better solution. Let the description of each 
> Lookup mode be precise, and you have already updated PRIVATE mode to
> include nestmates.

Okay I've deleted that sentence.

Unfortunately something has broken specdiff so I can't regenerate the 
docs at the moment - and I've lost the ones I had generated.

> 
> I brought this concern up in the EG meeting  yesterday and wanted to 
> clarify the difference
> between handling of inner/outer classes for backward compatibility and 
> general nestmate handling.

Many thanks for that detailed walk through.

David
-----


> Assumptions:
> 1. MethodHandle/VarHandle behavior is modeled on bytecode behavior.
> 2. Nestmates have the added capability of access to private members of 
> their nestmates. Period.
> 3. In future we expect to use nestmates for more than inner/outer classes.
> 4. Inner/outer classes will continue to have the InnerClasses attribute, 
> and starting in JDK11, javac
> will also generate NestHost and NestMember attributes.
> 5. With Nestmates, javac will not generate the default (package) 
> trampolines to allow inner/outer
> classes to access each other’s private members. Note that today this is 
> only done for members that
> have compile time accesses.
> (6. With Nestmates, bridges for protected members will still be 
> generated unchanged.)
> 
> For nestmates in general, the modifications you have made below allow a 
> nestmate to access private
> members of their nestmates to match the bytecode behavior.
> 
> Prior to nestmates, there is a special workaround in 
> MethodHandles.Lookup.in() to allow inner/outer
> classes to access any member of any class that shares its top level 
> class to emulate the generated trampolines.
> 
> https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/MethodHandles.Lookup.html#lookupModes--
> 
> In some cases, access between nested classes is obtained by the Java 
> compiler by creating an wrapper method to access a private method of 
> another class in the same top-level declaration. For example, a nested 
> class |C.D| can access private members within other related classes such 
> as |C|, |C.D.E|, or |C.B|, but the Java compiler may need to generate 
> wrapper methods in those related classes. In such cases, a 
> |Lookup| object on |C.E| would be unable to those private members. A 
> workaround for this limitation is the |Lookup.in| 
> <https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/MethodHandles.Lookup.html#in-java.lang.Class-> method, 
> which can transform a lookup on |C.E| into one on any of those other 
> classes, without special elevation of privilege.
> 
> This workaround will continue to be supported going forward explicitly 
> for inner/outer classes.
> A side-effect of this workaround is the ability of the returned Lookup 
> to access not only private methods in
> the “related” class, but also protected and inherited members of that 
> class which are defined in other packages.
> So this workaround will continue to work for inner/outer classes that 
> are also nestmates for backward
> compatibility.
> 
> Going forward, for nestmates in general, the goal is to provide access 
> to private members, which
> can be done via the access check to match bytecode behavior, and does 
> not require special Lookup.in() workarounds.
> 
> If at some point in the future we decide we want increased access for 
> nestmates, we can widen
> this. Let’s just say that the complexity there is challenging and that 
> it is better to err on the side of
> starting out more restrictive.
> 
> Summary - I agree with David. We can leave the documentation as is, with 
> the explicit changes to
> access checking modified below for private members accessible to nestmates.
> 
> thanks David!
> Karen
> 
>> On Feb 14, 2018, at 8:36 PM, David Holmes <david.holmes at oracle.com 
>> <mailto:david.holmes at oracle.com>> wrote:
>>
>> Hi Karen,
>>
>> Thanks for looking at this.
>>
>> On 15/02/2018 1:16 AM, Karen Kinnear wrote:
>>> David,
>>> Re-reading these I had one suggestion:
>>>> - java/lang/invoke/MethodHandles.java
>>>>       * <p>
>>>> -     * In some cases, access between nested classes is obtained by 
>>>> the Java compiler by creating
>>>> -     * an wrapper method to access a private method of another class
>>>> -     * in the same top-level declaration.
>>>> +     * Since JDK 11 the relationship between nested types can be 
>>>> expressed directly through the
>>>> +     * {@code NestHost} and {@code NestMembers} attributes.
>>>> +     * (See the Java Virtual Machine Specification, sections 4.7.28 
>>>> and 4.7.29.)
>>>> +     * In that case, the lookup class has direct access to private 
>>>> members of all its nestmates, and
>>>> +     * that is true of the associated {@code Lookup} object as well.
>>>> +     * Otherwise, access between nested classes is obtained by the 
>>>> Java compiler creating
>>>> +     * a wrapper method to access a private method of another class 
>>>> in the same nest.
>>>>       * For example, a nested class {@code C.D}
>>>>
>>>> Updated the nested classes description to cover legacy approach and 
>>>> new nestmate approach.
>>>>
>>>> -     * {@code C.E} would be unable to those private members.
>>>> +     * {@code C.E} would be unable to access those private members.
>>>>
>>>> Fixed typo: "access" was missing.
>>>>
>>>>       * <em>Discussion of private access:</em>
>>>>       * We say that a lookup has <em>private access</em>
>>>>       * if its {@linkplain #lookupModes lookup modes}
>>>> -     * include the possibility of accessing {@code private} members.
>>>> +     * include the possibility of accessing {@code private} members
>>>> +     * (which includes the private members of nestmates).
>>>>       * As documented in the relevant methods elsewhere,
>>>>       * only lookups with private access possess the following 
>>>> capabilities:
>>>>       * <ul style="font-size:smaller;">
>>>> -     * <li>access private fields, methods, and constructors of the 
>>>> lookup class
>>>> +     * <li>access private fields, methods, and constructors of the 
>>>> lookup class and its nestmates
>>>>
>>>> Clarify that private access includes nestmate access.
>>>>
>>>> -     *  access all members of the caller's class, all public types 
>>>> in the caller's module,
>>>> +     *  access all members of the caller's class and nestmates, all 
>>>> public types in the caller's module,
>>> For the above, I would change this to
>>> * access all members of the caller’s class, all private members of 
>>> nestmates, all types in the caller’s package, all public …
>>> Specifically, we are extended the PRIVATE mode as above to include 
>>> access to all private members of nestmates
>>> and this description is trying to summarize access when all possible 
>>> bits are set.
>>> None of the settings give you access to default (package-private) 
>>> members that a nestmate inherits from a
>>> different package - since the Lookup model is based on the 
>>> JVMS/bytecode behavior.
>>
>> True - you don't get access to a nestmates inherited protected members 
>> declared in a different package. I certainly didn't intend to somehow 
>> imply that.
>>
>>> I added the types in the caller’s package since PACKAGE gives you 
>>> that but it was missing from the existing list.
>>
>> Backing up ... in 8 this method doc simply said:
>>
>> "A freshly-created lookup object on the caller's class has all 
>> possible bits set, since the caller class can access all its own members."
>>
>> It doesn't try to say what a class can access, it just makes the 
>> obvious statement that it can access all its own members. If that was 
>> the current text I would not have needed to make any adjustment for 
>> nestmates.
>>
>> But for 9/10 it states:
>>
>> "A freshly-created lookup object on the caller's class has all 
>> possible bits set, except UNCONDITIONAL. The lookup can be used to 
>> access all members of the caller's class, all public types in the 
>> caller's module, and all public types in packages exported by other 
>> modules to the caller's module."
>>
>> This is quite a different formulation as it now tries to enumerate the 
>> set of accessible things - or at least gives that impression to me! 
>> But it is not complete as it doesn't mention non-public types in the 
>> current package, nor does it mention package-accessible members of 
>> types (whether public or not).
>>
>> With nestmates we have only expanded to include private member access, 
>> but the original text doesn't touch on nestmates directly at all. If 
>> we were to say only "all private members of nestmates" then we seem to 
>> suggest no access to the all the other members (public, directly 
>> declared protected, package). But if we say "nestmates" then that may 
>> imply more than intended as you point out. If we say nothing then we 
>> again imply by omission that there is no nestmate access.
>>
>> This seems to be a bit of an unwravelling thread started by the 
>> changes in 9. I would suggest we simply delete this sentence altogether:
>>
>> "The lookup can be used to access all members of the caller's class, 
>> all public types in the caller's module, and all public types in 
>> packages exported by other modules to the caller's module."
>>
>> as each of the modes, together with the "Access Checking" section of 
>> the class docs, define what is accessible under which mode. I don't 
>> think lookupModes() needs to try and restate that.
>>
>> What do you think?
>>
>> Thanks,
>> David
>>
>>> thanks,
>>> Karen
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> David
>