Alternative mechanism for reflective access control (#ReflectiveAccessToNonExportedTypes / #AwkwardStrongEncapsulation)
David M. Lloyd
david.lloyd at redhat.com
Thu Sep 29 13:30:05 UTC 2016
On 09/29/2016 07:31 AM, Peter Levart wrote:
> Hi David,
>
> I'm not the one to answer your queries, but may I ask you some questions
> about your requirements and also give some comments...
>
> On 09/28/2016 02:13 PM, David M. Lloyd wrote:
>> Hi all, I've been requested to ask if the OpenJDK development team
>> have had a chance to review this email, and when we might expect a
>> response.
>>
>> Thanks!
>>
>> On 09/21/2016 11:39 AM, David M. Lloyd wrote:
>>> In our internal discussion of the proposal for
>>> #ReflectiveAccessToNonExportedTypes, we discussed the ins and outs of
>>> various behaviors and have come up with a few ideas or starting points
>>> for solutions that we think would be more workable in conjunction with
>>> existing middleware (ours and others').
>>>
>>> For reasons previously explained, we do not think that weak modules are
>>> a good way forward; I won't go into that again here. But the logical
>>> re-starting point is: If not weak modules, then what?
>>>
>>> I will boil it down to a few basic requirements that we have
>>> established. This list is probably non-exhaustive but hopefully
>>> complete enough to go on for now:
>>>
>>> • A module definition must be able to establish that a dependent has (or
>>> all modules have) access to one or more (or all) packages for public
>>> reflection only.
>>> • A module definition must be able to establish that a dependent has (or
>>> all modules have) access to one or more (or all) packages for public or
>>> private reflection only.
>
> Why do you think is important to have a mode that allows reflective
> access but disallows compilation/linkage? Because frameworks that use
> reflection usually have ability to dynamically adjust to changes of code
> that is exposed in that way while compilation/linkage hard-codes access
> to the code and therefore breaks when such code is changed?
No, not at all. It's just that the two concerns (linkage versus
reflection access permission) are not really coupled.
For example if I have a library that uses reflection to examine
arbitrary objects that are handed to it, I definitely don't need to (or
want to) depend on the object class's source module or package for
linkage purposes. The code of the given object class might be non-API
for example, and I may not want to give a permanent compatibility
guarantee to my consumers, thus I do not want to export it.
> I have a concern that this will not prevent breakages. If reflective
> access is allowed, you can never be sure whether someone might be using
> it and just hard-coding such access in the consuming module. So while
> not providing "easy" hard-coded access to such code, it is still
> provided, just discouraged by making it more awkward.
Of course not; anyway in this scenario the deployer could als simply
modify the descriptor to suit their needs. However it's obvious that in
doing these things that breakage may occur if the implementation changes.
There is a lot of focus on the security aspect of encapsulation, but
using encapsulation also encourages clean engineering by presenting
clear contracts (much like public and non-public types do today).
Though this is no proof against complaint in the event of breakage -
especially where the JDK is concerned - out "in the wild" of real-world
applications and frameworks, these mechanisms really work.
> So I'm wondering if there might be an alternative way of "discouraging"
> hard coded access without complicating the module system with this
> aspect. A standard compile-time TYPE, PACKAGE or even MODULE annotation
> comes to mind that would trigger compilation warning if such annotated
> type or type in annotated package or type in annotated module would be
> referenced in compiled code. A warning can always be turned into an
> error for the conscious. I think this could even be implemented as an
> annotation processor, outside of javac. There just has to be a standard
> annoation to agree upon. What do you think?
It seems pretty weird to have a package be exported so you can use it
*and* annotated that you *shouldn't* use it. It seems cleanest to
export what you want exported, grant what you want granted, and leave
the rest closed off. And this talks about javac/compile time, but it is
our position that in the real world, this is completely irrelevant as
build tools can and will do whatever they need to do to access what they
want to access, with complete disregard to what is packaged in a
descriptor. The descriptors are more likely to be assembled by a
distributor or deployer than to have a module developer drop a module
which is expected to work in any module environment (which requires,
among other things, completely universal agreement on module namespacing
- something that has been shown time and again to be an unrealistic
expectation).
Therefore all these requirements are from the perspective of load-link
time to run time.
>>> • A module definition must be able to establish that a dependent has (or
>>> all modules have) access to one or more (or all) packages for public
>>> reflection and compilation/linkage (i.e. it's an export by today's
>>> terminology).
>>> • A module definition must be able to establish that a dependent has (or
>>> all modules have) access to one or more (or all) packages for public or
>>> private reflection and compilation/linkage (i.e. it's a "private" export
>>> by today's terminology).
>>> • As today, any packages not declared in one or more of the above
>>> categories is inaccessible outside of the module in any way (note that
>>> as I showed previously we have also concluded that it should continue to
>>> be impossible to export a package for compilation/linkage without public
>>> reflection, as we have not discovered any use for such a mode).
>
> So given above "alternative" for discouraging hard-coded access, current
> jigsaw already covers your requirements up to this point. Except maybe a
> desire to more easily "export" multiple packages with a single
> directive. I don't know why a "wildcard" export couldn't be allowed. I
> haven't heard a strong argument against it yet besides the fact that
> packages do have hierarchical naming structure, but are otherwise
> independent entities so using wildcard exports would be a precedent
> which would group a subtree of packages in a hierarchy of names and give
> them equal accessibility. But that would just accept the reality which
> is that developers usually do group packages in naming hierarchies
> according to some property. This might not be THE property which exports
> would give them (accessibility) so such feature might in fact push
> developers into grouping packages into hierarchies by the accessibility
> constraints instead of giving them freedom of unbalanced choice. So this
> might be an argument against wildcard exports...
I think that a single '*' wildcard which applies to all packages is
definitely required, however I don't have an opinion about mixing
wildcards with partial package names.
>>> More generally:
>>>
>>> • The syntax for all of the above has no particular constraint (in fact
>>> I will try to actively avoid touching what could be a very
>>> bikeshedding-rich discussion), except that it should not be construable
>>> as being pejorative against the usage of reflective frameworks; rather,
>>> it should be clear what level of trust is being established without
>>> raising undue warning.
>>> • Applications should not need gratuitous amounts of declarations in
>>> their module(s) in order to utilize frameworks.
>>> • As previously established, it should not be possible for one
>>> declaration to reduce the scope of access of another declaration in a
>>> module definition.
>
> I have no questions or comments about above three requirements as I
> think are not in conflict currently. Are they?
Well, "weak" modules are in direct conflict with the first one, as I
described previously. To use non-"weak" modules seems to mean requiring
gratuitous declarations, in the absence of "exports dynamic". Only the
third point is not currently in conflict; mainly I'm just reaffirming
that to head off any proposals that would be rejected out of hand for
that reason.
>>> • Access to a module (for reflective purposes only) must not cause
>>> conflicts if multiple such modules which contain identical packages are
>>> accessible to a single consumer; in other words, reflection-only access
>>> into non-dependency modules is not bound by duplicate package
>>> restrictions as long as each package is unique per class loader, as per
>>> the current (Java 8) class loader rules.
>
> I think this is more or less not a problem. Access to a module for
> reflective purposes is usually performed by frameworks and framework
> modules do not "require" modules that expose classes for reflective
> access (it's usually the other way around). Frameworks can reflectively
> access classes in modules that they don't "require" because reflection
> does not need or implies readability. The only concern is when two
> independent modules, loaded by different class loaders, export same
> packages and they are both "required" by the same module. Maybe I don't
> see it, but is this a use case that arises in practice?
No, this is not something that happens with our module system and I
don't expect that it will be an expected use case in the future.
>>> The above cover the useful access modes that we have identified. This
>>> is _nearly_ adequate to cover the use cases that we are currently
>>> concerned about; for example, I could export all packages for public
>>> reflection only to a specific framework, if only I know the module name
>>> of the implementation.
>>>
>>> Unfortunately, this does not work well in the case where a module may
>>> consume a framework whose specification is separate from the
>>> implementation. An application module may need to use (say) EJB and
>>> JPA; there is presently no clean way to do so without either (a) relying
>>> on a container environment to rewrite the descriptor or (b) opening up
>>> the module and defeating the security mechanism (e.g. "weak"). Without
>>> either of these workarounds, the application developer must have a good
>>> deal of knowledge about what modules provide what services within a
>>> framework-rich environment, possibly resulting in a very verbose (and
>>> error-prone) descriptor; none of these options is really satisfactory.
>
> This, I think, is a real concern that needs addressing and has its
> weight so that complicating jigsaw because of it might be warranted.
>
> Maybe there is an API only solution. If we look at the use-cases, they
> are all about a consuming module using some API which is implemented by
> a module which is not known in advance and needs reflective access back
> to classes of consuming module.
>
> For example, we have a consuming module, say "app", an API module that
> just defines the API, say "jpa" and a module that implements the API and
> is somehow dynamically resolved at runtime (by ServiceLoader or
> similarly), say "hibernate", which name is not known in advance to the
> "app" module. What "app" module does know is the name of the API module,
> "jpa", so the "app" module could pretend that the "jpa" module is the
> module to be performing reflective access to its classes:
>
> module app {
> requires jpa;
> exports private app.entity to jpa;
> }
>
> This, by itself, would only enable accessing app.enity classes from jpa
> module which is just a bunch of interfaces without actual code.
>
> Now if there was a SecurityManager checked method in java.lang.Module:
>
> public void assumeAccessOf(Module module) throws SecurityException
> { ... }
>
> ...which would somehow merge the reflective access privileges of 'this'
> module with privileges of given 'module'. A central SecurityManager
> privileged "arbiter" (a container or launcher, for example) would then
> invoke:
>
> hibernameModule.assumeAccessOf(jpaModule);
>
> at appropriate time. You could ask why wouldn't hibernate module just
> declare that it assumes access of jpa module in its module descriptor?
> Because any module could do that and gain access. It has to be an
> independent privileged entity that decides and establishes trust towards
> 'hibernate' in this case.
>
> Simple. Would that satisfy your requirements?
I don't like that this would (seemingly) leave the check to run time. I
agree that, ideally, if I require or use an API which requires
reflective access, if we want to ensure that the usage is not illicit,
then user/requester should somehow have to acknowledge that the grant is
allowed. I'd prefer using a simple syntax (but again I'm not going to
get into that) - something that says "use this service, and yes I give
it permission to access me".
This way the module resolve would actually fail instead of encountering
a much later run time failure.
It doesn't necessarily preclude a more specific export like you describe
though. That would be a requirement beyond what we need but not in
conflict as far as I can see. I don't want to necessitate large amounts
of declarations though. The more succinct, the better - though I'd take
a fairly verbose, working solution over a terse solution which doesn't
meet our other requirements.
Requiring a manual activation of such a grant seems undesirable (in the
same way that adding reads is) because it creates a large (and seemingly
unnecessary) burden on maintainers of existing frameworks to support
modules.
>>> Thus, apart from the option of redesigning (to an extent) the security
>>> mechanism (thereby eliminating the need to seal off access to public
>>> reflection, which is definitely still an attractive option for various
>>> reasons from our perspective, but which is also a very different
>>> discussion), we need some sort of mechanism which decouples the literal
>>> dependency system from access permission (much like uses/provides does).
>>>
>>> For example if I could declare that my module uses "javax.ejb", and, in
>>> so doing, automatically grants public and private reflective access to
>>> the module that provides that service, this would be a good outcome. A
>>> module which answers to that service name could be responsible for
>>> reflective access to the application module, providing that information
>>> privately to any other framework modules which require it.
>
> As said in above alternative solution, this is not safe as any module
> can declare that it provides any service. There has to be an "arbiter".
I agree, this seems necessary.
> Also conflating service uses/provides with access privileges seems
> messy. At least from the "syntactical" point of view. You would then
> have to conflate declaration of service uses with packages that you wish
> to export to the providers of that service. I think it's better to
> express access in plain exports directives that are not explicitly tied
> to service uses and leave the "binding" to the 3rd party.
Leaving the syntax stuff alone.
>>> The migration story looks much better in this light: module descriptors
>>> still can be quite terse and specific. Applications which use
>>> reflective frameworks do not need gratuitous exports; in fact it's much
>>> more fluid for a user to say "I require these helper libraries; I use
>>> EJB; that's it" which means they don't have to worry about the details
>>> of whatever particular environment they run in. This also has the
>>> advantage of allowing new Java 9-generation specifications to stipulate
>>> standard service names for each specification (e.g. "javax.ejb",
>>> "javax.cdi", that sort of thing).
>>>
>>> While this doesn't cover 100% of our remaining issues with Jigsaw (of
>>> course; we'll all continue moving through the issues list as we have
>>> been to get us there), meeting these requirements would go a long way
>>> towards at least having a reflection story that is more practical for
>>> present-day frameworks to move forward with. So the last requirement
>>> would be:
>>>
>>> • A module definition must be able to establish that an "indirect"
>>> dependency exists on an otherwise unknown module providing a capability,
>>> wherein that module may require public or public+private reflection
>>> access to some or all packages without compile/link access. This could
>>> possibly exist in conjunction with, or as an evolution of, the current
>>> services mechanism, however a complicating factor is that the current
>>> mechanism is based specifically on types, whereas a purely symbolic
>>> relationship might be better for this purpose (this is not a requirement
>>> though if it can be made to work as-is). Note that any symbolic
>>> relationship system would need some in-code discovery mechanism such
>>> that consumers of the capability are made available to the provider
>>> and/or vice-versa, in order to make practical use of the relationship.
>>>
>>> The following example syntax is meant to be unambiguous and
>>> illustrative; no specific attempt is made to reuse existing keywords
>>> (for example), or even to imply an endorsement of the current descriptor
>>> mechanism at all, but to clarify how this might look in practice and
>>> provide a practical application of the ideas herein.
>>>
>>> Example 1: A contrived provider of the fictional framework
>>> "javax.fictional.orm" illustrating provides/uses-based access granting
>>>
>>> module org.foo.orm.provider {
>>>
>>> // Require a module dependency, and give it private reflection
>>> access to everything
>>> requires org.apache.commons.beanutils with private reflection
>>> on *;
>>>
>>> // Require a module dependency with no reflection
>>> requires org.apache.commons.logging;
>>>
>>> // Provide the framework
>>> provides javax.fictional.orm.ORM
>>> using private reflection
>>> with org.foo.orm.provider.ORMImpl1,
>>> org.foo.orm.provider.ORMImpl2;
>>> }
>>>
>>> Example 2: A contrived consumer of #1
>>>
>>> module com.mycompany.application {
>>> uses javax.fictional.orm.ORM; // automatically gives private
>>> reflection
>>> }
>
> ...as said, giving access on types to unknown modules, letting any
> module decide for itself whether it is "qualified" is IMO not a safe
> solution.
>
>>>
>>> Example 3: Grant reflection access to a couple of packages to a named
>>> non-dependency module
>>>
>>> module com.mycompany.application {
>>> grant public reflection on
>>> com.mycompay.application.package1,
>>> com.mycompay.application.package2
>>> to org.foo.framework;
>>> }
>
> ...would be alternatively expressed as:
>
> // module-info.java
> module com.mycompany.application {
> exports com.mycompay.application.package1 to org.foo.framework;
> exports com.mycompay.application.package2 to org.foo.framework;
> }
>
> // package-info.java: trigger compiler warnings for referencing types
> in package
> @DynamicallyAccessed package com.mycompay.application.package1;
>
> // package-info.java: trigger compiler warnings for referencing types
> in package
> @DynamicallyAccessed package com.mycompay.application.package2;
>
>>>
>>> Example 4: Behave like Java 8
>>>
>>> module com.mycompany.application {
>>> grant private reflection on * to *;
>>> }
>
> ...alternatively:
>
> // trigger compiler warnings for referencing types in module
> @DynamicallyAccessed
> module com.mycompany.application {
> exports private *; // but see above why wildcard exports might not
> be a good idea
> }
>
> or:
>
> // trigger compiler warnings for referencing types in module
> @DynamicallyAccessed
> weak module com.mycompany.application {
> }
>
>>>
>>> Example 5: Behave like Java 8, but restrict private access without
>>> requiring a security manager
>>>
>>> module com.mycompany.application {
>>> grant public reflection on * to *;
>>> }
>
> ...alternatively:
>
> // trigger compiler warnings for referencing types in module
> @DynamicallyAccessed
> module com.mycompany.application {
> exports *; // but see above why wildcard exports might not be a good
> idea
> }
>
> there's no alternative without using widcards here.
>
>>>
>>> Example 6: An example of using CDI and EJB with symbolic capabilities
>>>
>>> module com.mycompany.application {
>>> uses capability javax.ejb, javax.cdi
>>> }
>>>
>>> Example 7: An example of providing EJB with symbolic capabilities
>>>
>>> module org.foo.ejb.provider {
>>> [...]
>>> provides capability javax.ejb using private reflection;
>>> }
>
> The language of module-info is getting very complicated, hm... Why not
> simply:
>
> @DynamicallyAccessed
> module com.mycompany.application {
> exports private * to javax.ejb;
> exports private * to javax.cdi;
> }
>
> module javax.ejb {
> ...
> }
>
> module javax.cdi {
> ...
> }
>
> module org.foo.ejb.provider {
> requires javax.ejb;
> }
>
>
> Then use a launcher on class-path or in a separate module that requires
> all of above like:
>
> public class Launcher {
> public static void main(String ... args) {
>
> org.foo.ejb.provider.SomeClass.class.getModule().assumeAccessOf(javax.ejb.SomeClass.class.getModule());
> ....
> }
> }
>
>
> Module::assumeAccessOf could even have a corresponding java launcher
> option, in the style of --add-exports etc...
Assuming one regards the necessity of --add-exports as a good thing ;-)
--
- DML
More information about the jigsaw-dev
mailing list