Nest host validation vs NestHost attribute performed by Lookup::defineHiddenClass
mandy.chung at oracle.com
Wed Sep 25 00:27:21 UTC 2019
This is regarding an incompatibility concern when LambdaMetaFactory and
any other framework library converts the use of
`Unsafe::defineAnonymousClass` to `Lookup::defineHiddenClass` when a
caller caller `C` whose nest host `H` indicated by `NestHost` attribute
`NestHost` and `NestMembers` attributes are consulted by JVM access
control (JVMS 5.4.4). If `H` cannot be loaded, or is not in the same
run-time package as `C`, or does not authorize nest membership (`H` has
a `NestMembers attribute but there is no entry named `C`), then a
linkage error occurs.
OTOH, `Class::getNestHost` may return a Class that is not the named
class indicated by `NestHost` attribute. Specifically:
If there is a linkage error accessing the nest host, or if this class
or interface is not enumerated as a member of the nest by the nest host,
then it is considered to belong to its own nest and this is returned
as the host.
If `C` has a bad nest host indicated by `NestHost` attribute, then
effectively `NestHost` attribute is ignored and `C` is the host of its
own nest as if `C` lacked of `NestHost` attribute. No error occurs
until `C` attempts to access a private member of `H` - consistency for
VM and core reflection.
Now `Lookup::defineHiddenClass` can be used to add a hidden class to a
nest of the lookup class.
If a framework uses C's full-power lookup object to call
`Lookup::defineHiddenClass` to add a hidden class `X` into C's nest.
What is the nest host of `X`?
If C has a good nest host, of course X will have the same nest host as
C. If C has a bad nest host `H`, the lookup object on C is used to call
`Lookup::defineHiddenClass` - should it fail with illegal nest host?
Should it succeed to be consistent with core reflection
(asC.class::getNestHost returns C - see more)?
That is the current proposed behavior of `Lookup::defineHiddenClass` to
validate that the lookup class must be the host of a nest. If the
lookup class has `NestHost` attribute, it is a member of some other
class's nest and so fails. The design principle is that the lookup
class and the proper permission together authorizes the lookup object to
add a new member into a nest. In addition, C's nest host is H and
adding a class with C as the nest host should be disallowed as this
conceptually creates a "nested nest" (H is C's nest host, C is X's nest
A framework library can't tell whether C is the class indicated by C's
NestHost attribute. Take an example in practice. C uses lambda
expression or method reference expression but H cannot be found while C
never accesses H's private member (If C accesses H's private member,
then IAE must occur since it fails the access check). The first
invocation of indy LMF for C class unfortunately throws
BootstrapMethodError because LambdaMetaFactory
calls`Lookup::defineHiddenClass` to define a lambda proxy class for C
and IAE occurs as C is a member of `H` statically.
It is expected that this error case should rarely occur but it does have
an incompatibility risk that impacts existing applications.
As Class::getNestHost returns C as the nest host, a framework library
calls `defineHiddenClass` with the returned nest host but surprisely it
throws an exception that nothing it can do.
If C has a bad nest host `H`, it will not get an error until C attempts
to access a private member of H. I think we should revisit this
validation and relax `defineHiddenClass` validation to check that the
lookup class is the same class as Class::getNestHost returns. The
hidden class X has C as the nest host (but NOT H). The VM access check
will ensure that X will fail when accessing H's private members. No
nested nest is created since at runtime C is the host of its own nest
and X is added to C's nest (where the static nestmate relationship is
not in effect).
`Lookup::defineHiddenClass` performs the nest host validation but won't
consultthe `NestHost` attribute of the lookup class.
More information about the valhalla-dev