Alternative mechanism for reflective access control (#ReflectiveAccessToNonExportedTypes / #AwkwardStrongEncapsulation)

[Peter Levart]

Hi David,

I'm not the one to answer your queries, but may I ask you some questions 
about your requirements and also give some comments...

On 09/28/2016 02:13 PM, David M. Lloyd wrote:
> Hi all, I've been requested to ask if the OpenJDK development team 
> have had a chance to review this email, and when we might expect a 
> response.
>
> Thanks!
>
> On 09/21/2016 11:39 AM, David M. Lloyd wrote:
>> In our internal discussion of the proposal for
>> #ReflectiveAccessToNonExportedTypes, we discussed the ins and outs of
>> various behaviors and have come up with a few ideas or starting points
>> for solutions that we think would be more workable in conjunction with
>> existing middleware (ours and others').
>>
>> For reasons previously explained, we do not think that weak modules are
>> a good way forward; I won't go into that again here.  But the logical
>> re-starting point is: If not weak modules, then what?
>>
>> I will boil it down to a few basic requirements that we have
>> established.  This list is probably non-exhaustive but hopefully
>> complete enough to go on for now:
>>
>> • A module definition must be able to establish that a dependent has (or
>> all modules have) access to one or more (or all) packages for public
>> reflection only.
>> • A module definition must be able to establish that a dependent has (or
>> all modules have) access to one or more (or all) packages for public or
>> private reflection only.

Why do you think is important to have a mode that allows reflective 
access but disallows compilation/linkage? Because frameworks that use 
reflection usually have ability to dynamically adjust to changes of code 
that is exposed in that way while compilation/linkage hard-codes access 
to the code and therefore breaks when such code is changed?

I have a concern that this will not prevent breakages. If reflective 
access is allowed, you can never be sure whether someone might be using 
it and just hard-coding such access in the consuming module. So while 
not providing "easy" hard-coded access to such code, it is still 
provided, just discouraged by making it more awkward.

So I'm wondering if there might be an alternative way of "discouraging" 
hard coded access without complicating the module system with this 
aspect. A standard compile-time TYPE, PACKAGE or even MODULE annotation 
comes to mind that would trigger compilation warning if such annotated 
type or type in annotated package or type in annotated module would be 
referenced in compiled code. A warning can always be turned into an 
error for the conscious. I think this could even be implemented as an 
annotation processor, outside of javac. There just has to be a standard 
annoation to agree upon. What do you think?

>> • A module definition must be able to establish that a dependent has (or
>> all modules have) access to one or more (or all) packages for public
>> reflection and compilation/linkage (i.e. it's an export by today's
>> terminology).
>> • A module definition must be able to establish that a dependent has (or
>> all modules have) access to one or more (or all) packages for public or
>> private reflection and compilation/linkage (i.e. it's a "private" export
>> by today's terminology).
>> • As today, any packages not declared in one or more of the above
>> categories is inaccessible outside of the module in any way (note that
>> as I showed previously we have also concluded that it should continue to
>> be impossible to export a package for compilation/linkage without public
>> reflection, as we have not discovered any use for such a mode).

So given above "alternative" for discouraging hard-coded access, current 
jigsaw already covers your requirements up to this point. Except maybe a 
desire to more easily "export" multiple packages with a single 
directive. I don't know why a "wildcard" export couldn't be allowed. I 
haven't heard a strong argument against it yet besides the fact that 
packages do have hierarchical naming structure, but are otherwise 
independent entities so using wildcard exports would be a precedent 
which would group a subtree of packages in a hierarchy of names and give 
them equal accessibility. But that would just accept the reality which 
is that developers usually do group packages in naming hierarchies 
according to some property. This might not be THE property which exports 
would give them (accessibility) so such feature might in fact push 
developers into grouping packages into hierarchies by the accessibility 
constraints instead of giving them freedom of unbalanced choice. So this 
might be an argument against wildcard exports...

>>
>> More generally:
>>
>> • The syntax for all of the above has no particular constraint (in fact
>> I will try to actively avoid touching what could be a very
>> bikeshedding-rich discussion), except that it should not be construable
>> as being pejorative against the usage of reflective frameworks; rather,
>> it should be clear what level of trust is being established without
>> raising undue warning.
>> • Applications should not need gratuitous amounts of declarations in
>> their module(s) in order to utilize frameworks.
>> • As previously established, it should not be possible for one
>> declaration to reduce the scope of access of another declaration in a
>> module definition.

I have no questions or comments about above three requirements as I 
think are not in conflict currently. Are they?

>> • Access to a module (for reflective purposes only) must not cause
>> conflicts if multiple such modules which contain identical packages are
>> accessible to a single consumer; in other words, reflection-only access
>> into non-dependency modules is not bound by duplicate package
>> restrictions as long as each package is unique per class loader, as per
>> the current (Java 8) class loader rules.

I think this is more or less not a problem. Access to a module for 
reflective purposes is usually performed by frameworks and framework 
modules do not "require" modules that expose classes for reflective 
access (it's usually the other way around). Frameworks can reflectively 
access classes in modules that they don't "require" because reflection 
does not need or implies readability. The only concern is when two 
independent modules, loaded by different class loaders, export same 
packages and they are both "required" by the same module. Maybe I don't 
see it, but is this a use case that arises in practice?

>>
>> The above cover the useful access modes that we have identified.  This
>> is _nearly_ adequate to cover the use cases that we are currently
>> concerned about; for example, I could export all packages for public
>> reflection only to a specific framework, if only I know the module name
>> of the implementation.
>>
>> Unfortunately, this does not work well in the case where a module may
>> consume a framework whose specification is separate from the
>> implementation.  An application module may need to use (say) EJB and
>> JPA; there is presently no clean way to do so without either (a) relying
>> on a container environment to rewrite the descriptor or (b) opening up
>> the module and defeating the security mechanism (e.g. "weak"). Without
>> either of these workarounds, the application developer must have a good
>> deal of knowledge about what modules provide what services within a
>> framework-rich environment, possibly resulting in a very verbose (and
>> error-prone) descriptor; none of these options is really satisfactory.

This, I think, is a real concern that needs addressing and has its 
weight so that complicating jigsaw because of it might be warranted.

Maybe there is an API only solution. If we look at the use-cases, they 
are all about a consuming module using some API which is implemented by 
a module which is not known in advance and needs reflective access back 
to classes of consuming module.

For example, we have a consuming module, say "app", an API module that 
just defines the API, say "jpa" and a module that implements the API and 
is somehow dynamically resolved at runtime (by ServiceLoader or 
similarly), say "hibernate", which name is not known in advance to the 
"app" module. What "app" module does know is the name of the API module, 
"jpa", so the "app" module could pretend that the "jpa" module is the 
module to be performing reflective access to its classes:

module app {
     requires jpa;
     exports private app.entity to jpa;
}

This, by itself, would only enable accessing app.enity classes from jpa 
module which is just a bunch of interfaces without actual code.

Now if there was a SecurityManager checked method in java.lang.Module:

      public void assumeAccessOf(Module module) throws SecurityException 
{ ... }

...which would somehow merge the reflective access privileges of 'this' 
module with privileges of given 'module'. A central SecurityManager 
privileged "arbiter" (a container or launcher, for example) would then 
invoke:

     hibernameModule.assumeAccessOf(jpaModule);

at appropriate time. You could ask why wouldn't hibernate module just 
declare that it assumes access of jpa module in its module descriptor? 
Because any module could do that and gain access. It has to be an 
independent privileged entity that decides and establishes trust towards 
'hibernate' in this case.

  Simple. Would that satisfy your requirements?

>>
>> Thus, apart from the option of redesigning (to an extent) the security
>> mechanism (thereby eliminating the need to seal off access to public
>> reflection, which is definitely still an attractive option for various
>> reasons from our perspective, but which is also a very different
>> discussion), we need some sort of mechanism which decouples the literal
>> dependency system from access permission (much like uses/provides does).
>>
>> For example if I could declare that my module uses "javax.ejb", and, in
>> so doing, automatically grants public and private reflective access to
>> the module that provides that service, this would be a good outcome.  A
>> module which answers to that service name could be responsible for
>> reflective access to the application module, providing that information
>> privately to any other framework modules which require it.

As said in above alternative solution, this is not safe as any module 
can declare that it provides any service. There has to be an "arbiter". 
Also conflating service uses/provides with access privileges seems 
messy. At least from the "syntactical" point of view. You would then 
have to conflate declaration of service uses with packages that you wish 
to export to the providers of that service. I think it's better to 
express access in plain exports directives that are not explicitly tied 
to service uses and leave the "binding" to the 3rd party.

>>
>> The migration story looks much better in this light: module descriptors
>> still can be quite terse and specific.  Applications which use
>> reflective frameworks do not need gratuitous exports; in fact it's much
>> more fluid for a user to say "I require these helper libraries; I use
>> EJB; that's it" which means they don't have to worry about the details
>> of whatever particular environment they run in.  This also has the
>> advantage of allowing new Java 9-generation specifications to stipulate
>> standard service names for each specification (e.g. "javax.ejb",
>> "javax.cdi", that sort of thing).
>>
>> While this doesn't cover 100% of our remaining issues with Jigsaw (of
>> course; we'll all continue moving through the issues list as we have
>> been to get us there), meeting these requirements would go a long way
>> towards at least having a reflection story that is more practical for
>> present-day frameworks to move forward with.  So the last requirement
>> would be:
>>
>> • A module definition must be able to establish that an "indirect"
>> dependency exists on an otherwise unknown module providing a capability,
>> wherein that module may require public or public+private reflection
>> access to some or all packages without compile/link access. This could
>> possibly exist in conjunction with, or as an evolution of, the current
>> services mechanism, however a complicating factor is that the current
>> mechanism is based specifically on types, whereas a purely symbolic
>> relationship might be better for this purpose (this is not a requirement
>> though if it can be made to work as-is).  Note that any symbolic
>> relationship system would need some in-code discovery mechanism such
>> that consumers of the capability are made available to the provider
>> and/or vice-versa, in order to make practical use of the relationship.
>>
>> The following example syntax is meant to be unambiguous and
>> illustrative; no specific attempt is made to reuse existing keywords
>> (for example), or even to imply an endorsement of the current descriptor
>> mechanism at all, but to clarify how this might look in practice and
>> provide a practical application of the ideas herein.
>>
>> Example 1: A contrived provider of the fictional framework
>> "javax.fictional.orm" illustrating provides/uses-based access granting
>>
>> module org.foo.orm.provider {
>>
>>       // Require a module dependency, and give it private reflection
>> access to everything
>>       requires org.apache.commons.beanutils with private reflection 
>> on *;
>>
>>       // Require a module dependency with no reflection
>>       requires org.apache.commons.logging;
>>
>>       // Provide the framework
>>       provides javax.fictional.orm.ORM
>>           using private reflection
>>           with org.foo.orm.provider.ORMImpl1,
>>                org.foo.orm.provider.ORMImpl2;
>> }
>>
>> Example 2: A contrived consumer of #1
>>
>> module com.mycompany.application {
>>       uses javax.fictional.orm.ORM; // automatically gives private
>> reflection
>> }

...as said, giving access on types to unknown modules, letting any 
module decide for itself whether it is "qualified" is IMO not a safe 
solution.

>>
>> Example 3: Grant reflection access to a couple of packages to a named
>> non-dependency module
>>
>> module com.mycompany.application {
>>       grant public reflection on
>>           com.mycompay.application.package1,
>>           com.mycompay.application.package2
>>       to org.foo.framework;
>> }

...would be alternatively expressed as:

// module-info.java
module com.mycompany.application {
       exports com.mycompay.application.package1 to org.foo.framework;
       exports com.mycompay.application.package2 to org.foo.framework;
}

//  package-info.java: trigger compiler warnings for referencing types 
in package
@DynamicallyAccessed package com.mycompay.application.package1;

//  package-info.java: trigger compiler warnings for referencing types 
in package
@DynamicallyAccessed package com.mycompay.application.package2;

>>
>> Example 4: Behave like Java 8
>>
>> module com.mycompany.application {
>>       grant private reflection on * to *;
>> }

...alternatively:

// trigger compiler warnings for referencing types in module
@DynamicallyAccessed
module com.mycompany.application {
     exports private *; // but see above why wildcard exports might not 
be a good idea
}

or:

// trigger compiler warnings for referencing types in module
@DynamicallyAccessed
weak module com.mycompany.application {
}

>>
>> Example 5: Behave like Java 8, but restrict private access without
>> requiring a security manager
>>
>> module com.mycompany.application {
>>       grant public reflection on * to *;
>> }

...alternatively:

// trigger compiler warnings for referencing types in module
@DynamicallyAccessed
module com.mycompany.application {
     exports *; // but see above why wildcard exports might not be a 
good idea
}

there's no alternative without using widcards here.

>>
>> Example 6: An example of using CDI and EJB with symbolic capabilities
>>
>> module com.mycompany.application {
>>       uses capability javax.ejb, javax.cdi
>> }
>>
>> Example 7: An example of providing EJB with symbolic capabilities
>>
>> module org.foo.ejb.provider {
>>       [...]
>>       provides capability javax.ejb using private reflection;
>> }

The language of module-info is getting very complicated, hm... Why not 
simply:

@DynamicallyAccessed
module com.mycompany.application {
     exports private * to javax.ejb;
     exports private * to javax.cdi;
}

module javax.ejb {
     ...
}

module javax.cdi {
     ...
}

module org.foo.ejb.provider {
       requires javax.ejb;
}


Then use a launcher on class-path or in a separate module that requires 
all of above like:

public class Launcher {
     public static void main(String ... args) {
org.foo.ejb.provider.SomeClass.class.getModule().assumeAccessOf(javax.ejb.SomeClass.class.getModule());
         ....
     }
}


Module::assumeAccessOf could even have a corresponding java launcher 
option, in the style of --add-exports etc...


>>
>>
>> -- 
>> - DML
>


Regards, Peter