Alternative mechanism for reflective access control (#ReflectiveAccessToNonExportedTypes / #AwkwardStrongEncapsulation)

[GREGG WONDERLY]

I still, like others seem to, find it amazingly odd, that the security manager, existing basis for access control is not still what would count.  I understand that the JDK itself is not deployed with a security manager impl in most cases, and thus there would be no access context for the security manager to be used against.  What’s odd, is that you are still trying to block access to reflective access to the “open JDK”.  If it’s really open and it’s really something that the community contributes to, why do we need to block access, hide details and otherwise obfuscate access details?  Modularization should just be about separating pieces not needed should it not?  Why has this degenerated into such a huge bit of access restriction too?

Could people who think that modularity should solve these problems and the security manager not be part of the solution please provide some details about how security is the modularization mechanisms responsibility and not the security managers?  For deployment, I’ve always used the security manager to limit/control access when I take some 3rd party code/jar and deploy it into a production environment.  That helps me restrict its access to resources in a manageable and maintainable way.

What I see happening, is that this single issue of “access” to pieces of software, from other pieces of software is degenerating into huge amounts of complexity and specification and release control, which isn’t really what makes “open software” work together.

The conversation below, I feel, illustrates how big of a freight train of focus this single issue has degenerated into.  David tries hard here to be brief it seems, but arguably, there is so many specifics of variability and lack of ability to fully specify detail, that it’s hard to imagine, at least for me, how this path can result in anything more than “pseudo science” of specification and detail which will then result in a really hard to manage path forward, because there will still be no clean, clear and fully specified solution.

This is stuff that’s hard to detail, and even harder to managed with a “fixed” specification.  In the end, I am still concerned that what is happening, is something very specific to what the JDK team feels they need to do, to keep people out of private implementation.  It seems like that would best be done with a runtime security implementation, specific to the JVM, instead of a modularization specification which provides so many restrictions for open software systems that we end up only being able to say “all” or “nothing” is what we need to support.  The JDK needs more than that, but it’s only because developers at Oracle are being obstructionist to use of software that they don’t feel should be used.  

For a long time, I have had conversations with a lot of different Sun, and now Oracle employees.   I think that they all want to enable great things to happen with the Java platform.  But the attempts at perfection and control involved in many of the JSRs (NIO2’s failure to provide a working, dependable filesystem observer for all platforms for example) which have resulted in only partial improvements and in many cases reduced penetration (java.util.logging still doesn’t get improvements to align it’s apis better with varargs and more brief API designs) into new projects demonstrates for me that there is not always the right focus on what needs to happen in actuality, and instead a shotgun approach that destroys a lot of opportunity and splatters things around making them visible without great, ongoing improvement.

How can we make sure that this work doesn’t just result in a complete melt down of open software, causing huge migration to Harmony or Wine or some other Java platform abandonment?  I know that seems far fetched, but software developers are never really interested in friction, going slower, or working harder to deal with new features.   

Gregg

> On Sep 21, 2016, at 11:39 AM, David M. Lloyd <david.lloyd at redhat.com> wrote:
> 
> In our internal discussion of the proposal for #ReflectiveAccessToNonExportedTypes, we discussed the ins and outs of various behaviors and have come up with a few ideas or starting points for solutions that we think would be more workable in conjunction with existing middleware (ours and others').
> 
> For reasons previously explained, we do not think that weak modules are a good way forward; I won't go into that again here.  But the logical re-starting point is: If not weak modules, then what?
> 
> I will boil it down to a few basic requirements that we have established.  This list is probably non-exhaustive but hopefully complete enough to go on for now:
> 
> • A module definition must be able to establish that a dependent has (or all modules have) access to one or more (or all) packages for public reflection only.
> • A module definition must be able to establish that a dependent has (or all modules have) access to one or more (or all) packages for public or private reflection only.
> • A module definition must be able to establish that a dependent has (or all modules have) access to one or more (or all) packages for public reflection and compilation/linkage (i.e. it's an export by today's terminology).
> • A module definition must be able to establish that a dependent has (or all modules have) access to one or more (or all) packages for public or private reflection and compilation/linkage (i.e. it's a "private" export by today's terminology).
> • As today, any packages not declared in one or more of the above categories is inaccessible outside of the module in any way (note that as I showed previously we have also concluded that it should continue to be impossible to export a package for compilation/linkage without public reflection, as we have not discovered any use for such a mode).
> 
> More generally:
> 
> • The syntax for all of the above has no particular constraint (in fact I will try to actively avoid touching what could be a very bikeshedding-rich discussion), except that it should not be construable as being pejorative against the usage of reflective frameworks; rather, it should be clear what level of trust is being established without raising undue warning.
> • Applications should not need gratuitous amounts of declarations in their module(s) in order to utilize frameworks.
> • As previously established, it should not be possible for one declaration to reduce the scope of access of another declaration in a module definition.
> • Access to a module (for reflective purposes only) must not cause conflicts if multiple such modules which contain identical packages are accessible to a single consumer; in other words, reflection-only access into non-dependency modules is not bound by duplicate package restrictions as long as each package is unique per class loader, as per the current (Java 8) class loader rules.
> 
> The above cover the useful access modes that we have identified.  This is _nearly_ adequate to cover the use cases that we are currently concerned about; for example, I could export all packages for public reflection only to a specific framework, if only I know the module name of the implementation.
> 
> Unfortunately, this does not work well in the case where a module may consume a framework whose specification is separate from the implementation.  An application module may need to use (say) EJB and JPA; there is presently no clean way to do so without either (a) relying on a container environment to rewrite the descriptor or (b) opening up the module and defeating the security mechanism (e.g. "weak").  Without either of these workarounds, the application developer must have a good deal of knowledge about what modules provide what services within a framework-rich environment, possibly resulting in a very verbose (and error-prone) descriptor; none of these options is really satisfactory.
> 
> Thus, apart from the option of redesigning (to an extent) the security mechanism (thereby eliminating the need to seal off access to public reflection, which is definitely still an attractive option for various reasons from our perspective, but which is also a very different discussion), we need some sort of mechanism which decouples the literal dependency system from access permission (much like uses/provides does).
> 
> For example if I could declare that my module uses "javax.ejb", and, in so doing, automatically grants public and private reflective access to the module that provides that service, this would be a good outcome.  A module which answers to that service name could be responsible for reflective access to the application module, providing that information privately to any other framework modules which require it.
> 
> The migration story looks much better in this light: module descriptors still can be quite terse and specific.  Applications which use reflective frameworks do not need gratuitous exports; in fact it's much more fluid for a user to say "I require these helper libraries; I use EJB; that's it" which means they don't have to worry about the details of whatever particular environment they run in.  This also has the advantage of allowing new Java 9-generation specifications to stipulate standard service names for each specification (e.g. "javax.ejb", "javax.cdi", that sort of thing).
> 
> While this doesn't cover 100% of our remaining issues with Jigsaw (of course; we'll all continue moving through the issues list as we have been to get us there), meeting these requirements would go a long way towards at least having a reflection story that is more practical for present-day frameworks to move forward with.  So the last requirement would be:
> 
> • A module definition must be able to establish that an "indirect" dependency exists on an otherwise unknown module providing a capability, wherein that module may require public or public+private reflection access to some or all packages without compile/link access.  This could possibly exist in conjunction with, or as an evolution of, the current services mechanism, however a complicating factor is that the current mechanism is based specifically on types, whereas a purely symbolic relationship might be better for this purpose (this is not a requirement though if it can be made to work as-is).  Note that any symbolic relationship system would need some in-code discovery mechanism such that consumers of the capability are made available to the provider and/or vice-versa, in order to make practical use of the relationship.
> 
> The following example syntax is meant to be unambiguous and illustrative; no specific attempt is made to reuse existing keywords (for example), or even to imply an endorsement of the current descriptor mechanism at all, but to clarify how this might look in practice and provide a practical application of the ideas herein.
> 
> Example 1: A contrived provider of the fictional framework "javax.fictional.orm" illustrating provides/uses-based access granting
> 
> module org.foo.orm.provider {
> 
>      // Require a module dependency, and give it private reflection access to everything
>      requires org.apache.commons.beanutils with private reflection on *;
> 
>      // Require a module dependency with no reflection
>      requires org.apache.commons.logging;
> 
>      // Provide the framework
>      provides javax.fictional.orm.ORM
>          using private reflection
>          with org.foo.orm.provider.ORMImpl1,
>               org.foo.orm.provider.ORMImpl2;
> }
> 
> Example 2: A contrived consumer of #1
> 
> module com.mycompany.application {
>      uses javax.fictional.orm.ORM; // automatically gives private reflection
> }
> 
> Example 3: Grant reflection access to a couple of packages to a named non-dependency module
> 
> module com.mycompany.application {
>      grant public reflection on
>          com.mycompay.application.package1,
>          com.mycompay.application.package2
>      to org.foo.framework;
> }
> 
> Example 4: Behave like Java 8
> 
> module com.mycompany.application {
>      grant private reflection on * to *;
> }
> 
> Example 5: Behave like Java 8, but restrict private access without requiring a security manager
> 
> module com.mycompany.application {
>      grant public reflection on * to *;
> }
> 
> Example 6: An example of using CDI and EJB with symbolic capabilities
> 
> module com.mycompany.application {
>      uses capability javax.ejb, javax.cdi
> }
> 
> Example 7: An example of providing EJB with symbolic capabilities
> 
> module org.foo.ejb.provider {
>      [...]
>      provides capability javax.ejb using private reflection;
> }
> 
> 
> --
> - DML