API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[David Holmes]

Specdiffs now available again:

http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang/java/lang/Class.html

http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.invoke/java/lang/invoke/MethodHandle.html

http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.invoke/java/lang/invoke/MethodHandles.Lookup.html

http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.reflect/java/lang/reflect/AccessibleObject.html

http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.reflect/java/lang/reflect/Method.html

David

On 16/02/2018 7:20 PM, David Holmes wrote:
> Hi Karen,
> 
> On 16/02/2018 8:01 AM, Karen Kinnear wrote:
>> David,
>>
>> I think that is a much better solution. Let the description of each 
>> Lookup mode be precise, and you have already updated PRIVATE mode to
>> include nestmates.
> 
> Okay I've deleted that sentence.
> 
> Unfortunately something has broken specdiff so I can't regenerate the 
> docs at the moment - and I've lost the ones I had generated.
> 
>>
>> I brought this concern up in the EG meeting  yesterday and wanted to 
>> clarify the difference
>> between handling of inner/outer classes for backward compatibility and 
>> general nestmate handling.
> 
> Many thanks for that detailed walk through.
> 
> David
> -----
> 
> 
>> Assumptions:
>> 1. MethodHandle/VarHandle behavior is modeled on bytecode behavior.
>> 2. Nestmates have the added capability of access to private members of 
>> their nestmates. Period.
>> 3. In future we expect to use nestmates for more than inner/outer 
>> classes.
>> 4. Inner/outer classes will continue to have the InnerClasses 
>> attribute, and starting in JDK11, javac
>> will also generate NestHost and NestMember attributes.
>> 5. With Nestmates, javac will not generate the default (package) 
>> trampolines to allow inner/outer
>> classes to access each other’s private members. Note that today this 
>> is only done for members that
>> have compile time accesses.
>> (6. With Nestmates, bridges for protected members will still be 
>> generated unchanged.)
>>
>> For nestmates in general, the modifications you have made below allow 
>> a nestmate to access private
>> members of their nestmates to match the bytecode behavior.
>>
>> Prior to nestmates, there is a special workaround in 
>> MethodHandles.Lookup.in() to allow inner/outer
>> classes to access any member of any class that shares its top level 
>> class to emulate the generated trampolines.
>>
>> https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/MethodHandles.Lookup.html#lookupModes-- 
>>
>>
>> In some cases, access between nested classes is obtained by the Java 
>> compiler by creating an wrapper method to access a private method of 
>> another class in the same top-level declaration. For example, a nested 
>> class |C.D| can access private members within other related classes 
>> such as |C|, |C.D.E|, or |C.B|, but the Java compiler may need to 
>> generate wrapper methods in those related classes. In such cases, a 
>> |Lookup| object on |C.E| would be unable to those private members. A 
>> workaround for this limitation is the |Lookup.in| 
>> <https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/MethodHandles.Lookup.html#in-java.lang.Class-> method, 
>> which can transform a lookup on |C.E| into one on any of those other 
>> classes, without special elevation of privilege.
>>
>> This workaround will continue to be supported going forward explicitly 
>> for inner/outer classes.
>> A side-effect of this workaround is the ability of the returned Lookup 
>> to access not only private methods in
>> the “related” class, but also protected and inherited members of that 
>> class which are defined in other packages.
>> So this workaround will continue to work for inner/outer classes that 
>> are also nestmates for backward
>> compatibility.
>>
>> Going forward, for nestmates in general, the goal is to provide access 
>> to private members, which
>> can be done via the access check to match bytecode behavior, and does 
>> not require special Lookup.in() workarounds.
>>
>> If at some point in the future we decide we want increased access for 
>> nestmates, we can widen
>> this. Let’s just say that the complexity there is challenging and that 
>> it is better to err on the side of
>> starting out more restrictive.
>>
>> Summary - I agree with David. We can leave the documentation as is, 
>> with the explicit changes to
>> access checking modified below for private members accessible to 
>> nestmates.
>>
>> thanks David!
>> Karen
>>
>>> On Feb 14, 2018, at 8:36 PM, David Holmes <david.holmes at oracle.com 
>>> <mailto:david.holmes at oracle.com>> wrote:
>>>
>>> Hi Karen,
>>>
>>> Thanks for looking at this.
>>>
>>> On 15/02/2018 1:16 AM, Karen Kinnear wrote:
>>>> David,
>>>> Re-reading these I had one suggestion:
>>>>> - java/lang/invoke/MethodHandles.java
>>>>>       * <p>
>>>>> -     * In some cases, access between nested classes is obtained by 
>>>>> the Java compiler by creating
>>>>> -     * an wrapper method to access a private method of another class
>>>>> -     * in the same top-level declaration.
>>>>> +     * Since JDK 11 the relationship between nested types can be 
>>>>> expressed directly through the
>>>>> +     * {@code NestHost} and {@code NestMembers} attributes.
>>>>> +     * (See the Java Virtual Machine Specification, sections 
>>>>> 4.7.28 and 4.7.29.)
>>>>> +     * In that case, the lookup class has direct access to private 
>>>>> members of all its nestmates, and
>>>>> +     * that is true of the associated {@code Lookup} object as well.
>>>>> +     * Otherwise, access between nested classes is obtained by the 
>>>>> Java compiler creating
>>>>> +     * a wrapper method to access a private method of another 
>>>>> class in the same nest.
>>>>>       * For example, a nested class {@code C.D}
>>>>>
>>>>> Updated the nested classes description to cover legacy approach and 
>>>>> new nestmate approach.
>>>>>
>>>>> -     * {@code C.E} would be unable to those private members.
>>>>> +     * {@code C.E} would be unable to access those private members.
>>>>>
>>>>> Fixed typo: "access" was missing.
>>>>>
>>>>>       * <em>Discussion of private access:</em>
>>>>>       * We say that a lookup has <em>private access</em>
>>>>>       * if its {@linkplain #lookupModes lookup modes}
>>>>> -     * include the possibility of accessing {@code private} members.
>>>>> +     * include the possibility of accessing {@code private} members
>>>>> +     * (which includes the private members of nestmates).
>>>>>       * As documented in the relevant methods elsewhere,
>>>>>       * only lookups with private access possess the following 
>>>>> capabilities:
>>>>>       * <ul style="font-size:smaller;">
>>>>> -     * <li>access private fields, methods, and constructors of the 
>>>>> lookup class
>>>>> +     * <li>access private fields, methods, and constructors of the 
>>>>> lookup class and its nestmates
>>>>>
>>>>> Clarify that private access includes nestmate access.
>>>>>
>>>>> -     *  access all members of the caller's class, all public types 
>>>>> in the caller's module,
>>>>> +     *  access all members of the caller's class and nestmates, 
>>>>> all public types in the caller's module,
>>>> For the above, I would change this to
>>>> * access all members of the caller’s class, all private members of 
>>>> nestmates, all types in the caller’s package, all public …
>>>> Specifically, we are extended the PRIVATE mode as above to include 
>>>> access to all private members of nestmates
>>>> and this description is trying to summarize access when all possible 
>>>> bits are set.
>>>> None of the settings give you access to default (package-private) 
>>>> members that a nestmate inherits from a
>>>> different package - since the Lookup model is based on the 
>>>> JVMS/bytecode behavior.
>>>
>>> True - you don't get access to a nestmates inherited protected 
>>> members declared in a different package. I certainly didn't intend to 
>>> somehow imply that.
>>>
>>>> I added the types in the caller’s package since PACKAGE gives you 
>>>> that but it was missing from the existing list.
>>>
>>> Backing up ... in 8 this method doc simply said:
>>>
>>> "A freshly-created lookup object on the caller's class has all 
>>> possible bits set, since the caller class can access all its own 
>>> members."
>>>
>>> It doesn't try to say what a class can access, it just makes the 
>>> obvious statement that it can access all its own members. If that was 
>>> the current text I would not have needed to make any adjustment for 
>>> nestmates.
>>>
>>> But for 9/10 it states:
>>>
>>> "A freshly-created lookup object on the caller's class has all 
>>> possible bits set, except UNCONDITIONAL. The lookup can be used to 
>>> access all members of the caller's class, all public types in the 
>>> caller's module, and all public types in packages exported by other 
>>> modules to the caller's module."
>>>
>>> This is quite a different formulation as it now tries to enumerate 
>>> the set of accessible things - or at least gives that impression to 
>>> me! But it is not complete as it doesn't mention non-public types in 
>>> the current package, nor does it mention package-accessible members 
>>> of types (whether public or not).
>>>
>>> With nestmates we have only expanded to include private member 
>>> access, but the original text doesn't touch on nestmates directly at 
>>> all. If we were to say only "all private members of nestmates" then 
>>> we seem to suggest no access to the all the other members (public, 
>>> directly declared protected, package). But if we say "nestmates" then 
>>> that may imply more than intended as you point out. If we say nothing 
>>> then we again imply by omission that there is no nestmate access.
>>>
>>> This seems to be a bit of an unwravelling thread started by the 
>>> changes in 9. I would suggest we simply delete this sentence altogether:
>>>
>>> "The lookup can be used to access all members of the caller's class, 
>>> all public types in the caller's module, and all public types in 
>>> packages exported by other modules to the caller's module."
>>>
>>> as each of the modes, together with the "Access Checking" section of 
>>> the class docs, define what is accessible under which mode. I don't 
>>> think lookupModes() needs to try and restate that.
>>>
>>> What do you think?
>>>
>>> Thanks,
>>> David
>>>
>>>> thanks,
>>>> Karen
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> David
>>