[nestmates] validation of ClassFile for hidden classes

[Mandy Chung]

On 11/13/19 9:01 PM, John Rose wrote:
> On Nov 13, 2019, at 8:49 PM, David Holmes <david.holmes at oracle.com 
> <mailto:david.holmes at oracle.com>> wrote:
>>
>> We would want the programmatic nesthost to take precedence, then try 
>> to resolve the static NH etc.
>
> Yes, determining the dynamic nest host of a HC must happen before the 
> attrbutes
> are examined.  In this sense the attributes will be ignored for HCs. 
>  The JVM has
> a rule for computing the host H or H'.
>
> I think it’s harmless (in practice) that the derivation of the dynamic 
> nest host H’ or H
> for the non-HC part of a resolution request might throw because a 
> non-hidden
> class has a broken attribute.  That’s just the cost of doing business 
> with the
> bidirectional relations of the NH/NMs attributes.
>
>> Not sure if Mandy has the reflection aspect documented outside of 
>> email at the moment.
>
> I think she has that covered in the API spec for jl.Class.
>
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/8171335/specdiff/java.base/java/lang/Class.html

This is what we have in the nestmates branch:

Class::getNestHost returns the runtime nest host.   Note that JDK 13 
does not have the runtime nest host concept but Class::getNestHost 
returns this Class if it fails the nest membership validation.

Class::getNestMembers will first determine the runtime nest host and 
then returns all members of the nest of this class.

Lookup::defineHiddenClass with NESTMATE option first will determine the 
runtime nest host of the lookup class (that should be the same as what 
LC.getNestHost() returns) and the runtime nest host of the lookup class 
will be the dynamic nest host of HC.    In other words, HC's NH and NM 
attributes are effectively ignored.

Then this will give consistent views:
     HC.getNestHost == LC.getNestHost
     HC.getNestMembers == LC.getNestMembers

I think JVMS should mention these reflection APIs along with 
Lookup::defineHiddenClass that will compute the runtime nest host.

If HC is not defined as a nestmate (i.e. Lookup::dHC is called without 
NESTMATE option), if a symbolic reference triggers the nestmate access 
control test before the runtime nest host of HC is not set, any error 
will be thrown due to the bad nest membership during access check.

Mandy