Proposal: #ReflectiveAccessToNonExportedTypes (revised) & #AwkwardStrongEncapsulation: Weak modules & private exports

Tue Oct 11 15:15:14 UTC 2016

2016/9/16 8:30:41 -0700, david.lloyd at redhat.com:
> After a fairly detailed review of this proposal, we have determined that 
> it is not acceptable to Red Hat in its present form.  I will list the 
> primary problems here, and then I'll start up discussion on jigsaw-dev 
> of several possible solutions that could work for us.  I'll number these 
> in case anyone wants to respond piece-wise.
> 
> #1) The "weak" designation appears to be pejorative
> 
> Under this solution, many existing frameworks and/or the modules which 
> consume them must be marked "weak" in order to work correctly, or else a 
> much more complex module descriptor must be used.  While this seems to 
> be a stepping stone for migration, we find that it rather makes 
> middleware appear as a second-class citizen.

No insult is intended; "weak" is used here purely in a technical sense,
as the obvious antonym of "strong".  Suggestions for a better word for
this concept are welcome.

> #2) Weak modules are, in fact, weak
> 
> The idea of retaining the Java 8 rules for transition modules appears 
> good on the surface, however the problem is that users may be forced to 
> use weak modules indefinitely in order to interoperate with certain 
> existing middleware and libraries, or else be faced with a potentially 
> complex migration.  If the security benefits of "strong" modules are 
> real, then users will be rightly inclined to move from weak modules to 
> strong modules.  This can be severely problematic if existing 
> middleware, libraries, and large applications cannot easily operate as 
> or with "strong" modules or cannot do so in an "easy to learn"/"easy to 
> use" fashion.  This also means that "weak" modules exist only for the 
> purposes of maintaining a transition period - at some point they will 
> inevitably transition from "convenience" to "attractive nuisance".

Weak modules do fall between automatic and strong modules on the overall
migration path but they're not intended, necessarily, to be transitory.
There's nothing wrong with evolving an existing JAR file into a weak
module and stopping there.  There's also nothing wrong with creating a
new weak module from scratch, as one might do for a module that contains
nothing but JPA model classes, and never evolving it into a strong
module -- there's no benefit in doing that, so don't do it.

> ...
> 
> #3) Controlling reflection access is always bound up with controlling 
> exports
> 
> The proposed exports system is simplistic but not exactly simple, yet 
> several useful modes remain left unaccounted-for.  Internally we 
> examined the usefulness of controlling access to public and so-called 
> "deep" reflection in combination with whether or not a member was 
> exported, and we came up with this logical table:
> 
>           +------------------+---------------------------+
>           |                  |        Module is...       |
>           |  Reflection is... +------------+--------------+
>           |                  |  Exported  | Not Exported |
>           +----------------------------------------------+
>           |        Forbidden | Not Useful |    Useful    |
>           +----------------------------------------------+
>           |      Public Only |   Useful   |    Useful    |
>           +----------------------------------------------+
>           | Public & Private |   Useful   |    Useful    |
>           +------------------+------------+--------------+

I'm not sure how to read this table.  Should the cell containing the text
"Module is ..." instead contain "Member is ..."?

By "exported" I'll assume that you mean non-reflective access, i.e., from
the language at compile time and bytecodes at run time.

Do you have example use cases for each of the "Useful" cells?  Some are
obvious, but others are not.

> ...
> 
> #4) There isn't always a container
> 
> The justification for not providing an easy mechanism to restrict 
> private reflective access to a trusted framework is that a container can 
> always rewrite a module or inject classes.  However there isn't always a 
> container present - something which the existence of weak modules 
> acknowledges.  Some frameworks and middleware are meant to be usable 
> with or without a container, and it is undesirable to force these use 
> cases into a weak module situation or to require a strong coupling in 
> the module descriptor in this case.  Some other solution should be found 
> that works both for containers and "regular" applications.

If you're running outside of a container then how important is it to
precisely restrict access to packages meant for deep reflection?  If
you're outside of a container then you have to trust the person (you?)
who sets up your module path and/or class path.  To take the canonical
case of a module that just contains JPA model classes, if you make it a
weak module or else `exports private` the relevant packages without
qualification so that the hibernate.core (or whatever) module can reflect
into it, then the scope for trouble is limited.  Sure, some other module
on the module path or even code on the class path could reflect into your
module when it's not supposed to, but is it worth complicating the module
system just to handle this scenario?

Don't get me wrong: Strong encapsulation is generally a very good thing.
It's not always absolutely necessary, however, and we need to balance the
number of ways in which it can be used against the overall complexity of
the design.

(It looks like there was no #5 in your list.)

> #6) Not all reflection is 100% "friendly", and that's OK
> 
> Serialization frameworks generally need private reflective access to the 
> module which contains the classes being serialized.  For some 
> frameworks, this access will already be adequate.  Some existing 
> frameworks utilize Unsafe to create uninitialized instances.  However 
> other frameworks, including frameworks that seek to comply with the 
> Serialization Specification [1], use ReflectionFactory, as that class 
> provides (exclusively, as far as I know) the ability to acquire a 
> constructor that produces a _partially_-initialized class where only the 
> constructors of superclasses to a certain depth are called.  It is our 
> view that it would be a regression to force such frameworks to use 
> --add-exports or JVMTI, and using Unsafe for this purpose, or continuing 
> to use it, (as I understand it) puts at risk the ability to perform 
> optimizations which depend on entry point control, which I am given to 
> understand are a strong motivator for many of the design constraints 
> that exist on the Jigsaw implementation.  It should be possible for 
> these frameworks to continue to use ReflectionFactory (unsupported 
> though it may be) if suitably privileged (in the security manager 
> sense), as long as all of the module(s) upon which the framework 
> reflects have granted private reflective access to it (in the module 
> declaration sense).

So far as I know, this is already the case.  The `ReflectionFactory` API
is specifically retained in JDK 9 for this very purpose, per JEP 260 [1].
Do you have an example for which this does not work?

> #7) More loose coupling seems necessary and useful
> 
> In order for typical applications to function with modern middleware as 
> modules, without compromising security, it may be necessary to enhance 
> the loose coupling mechanism (uses/provides), or to provide an 
> additional, similar mechanism, which allows a symbolic coupling which 
> would allow modules to declare (in an abstract manner) modules which 
> need to have reflective access to it in a "friendly" manner. 
> Implementation ideas are forthcoming on jigsaw-dev.
> 
> Providing a way within the system to grant public and/or private 
> reflection access in a specific manner, and doing so in a manner which 
> is easy to use and not prejudicial against existing or future middleware 
> and small or large applications which consume such middleware in the 
> Java SE or EE space, is our most basic requirement for satisfactory 
> resolution of this issue.

To confirm my understanding, would it be correct to say that your main
concern here is scenarios in which some of the classes in a user module
are intended to be manipulated reflectively by some trusted framework
module, but that framework module is either not known at compile time or
is otherwise inconvenient to name in the declaration of the user module?

>                            I will follow up on jigsaw-dev with some 
> implementation thoughts and more specific requirements.

Discussions of issues in the proposed design, and specific proposals
for changing the design, belong here on the EG list.  The jigsaw-dev
list is meant primarily for the discussion of issues outside the scope
of the JSR such as implementation details, the design and implementation
of JDK-specific APIs and tools, and feedback and questions arising from
practical use of the prototype implementation.

- Mark

[1] http://openjdk.java.net/jeps/260