Proposal: #ReflectiveAccessToNonExportedTypes (revised) & #AwkwardStrongEncapsulation: Weak modules & private exports

[David M. Lloyd]

On 10/11/2016 10:15 AM, mark.reinhold at oracle.com wrote:
> 2016/9/16 8:30:41 -0700, david.lloyd at redhat.com:
>> After a fairly detailed review of this proposal, we have determined that
>> it is not acceptable to Red Hat in its present form.  I will list the
>> primary problems here, and then I'll start up discussion on jigsaw-dev
>> of several possible solutions that could work for us.  I'll number these
>> in case anyone wants to respond piece-wise.
>>
>> #1) The "weak" designation appears to be pejorative
>>
>> Under this solution, many existing frameworks and/or the modules which
>> consume them must be marked "weak" in order to work correctly, or else a
>> much more complex module descriptor must be used.  While this seems to
>> be a stepping stone for migration, we find that it rather makes
>> middleware appear as a second-class citizen.
>
> No insult is intended; "weak" is used here purely in a technical sense,
> as the obvious antonym of "strong".  Suggestions for a better word for
> this concept are welcome.

It's the concept itself that is problematic, I think.  Really, if I 
understand correctly, the "weak" notion exists merely as shorthand to 
say "grant permission on all packages to all modules for private 
reflection", which is the sort of thing that a user ought to be saying 
explicitly.  Many users will use weak modules without understanding what 
they are; some will come to understand it and immediately press for best 
practices to get rid of them.  I think that regardless of what the 
concept is named, it's going to spell trouble in the end.

Our position is that grants of reflective access could (and perhaps 
should) be separated completely from exports, though as stated 
previously, exporting should imply public reflection access merely 
because it doesn't really make any sense not to do so.

>> #2) Weak modules are, in fact, weak
>>
>> The idea of retaining the Java 8 rules for transition modules appears
>> good on the surface, however the problem is that users may be forced to
>> use weak modules indefinitely in order to interoperate with certain
>> existing middleware and libraries, or else be faced with a potentially
>> complex migration.  If the security benefits of "strong" modules are
>> real, then users will be rightly inclined to move from weak modules to
>> strong modules.  This can be severely problematic if existing
>> middleware, libraries, and large applications cannot easily operate as
>> or with "strong" modules or cannot do so in an "easy to learn"/"easy to
>> use" fashion.  This also means that "weak" modules exist only for the
>> purposes of maintaining a transition period - at some point they will
>> inevitably transition from "convenience" to "attractive nuisance".
>
> Weak modules do fall between automatic and strong modules on the overall
> migration path but they're not intended, necessarily, to be transitory.
> There's nothing wrong with evolving an existing JAR file into a weak
> module and stopping there.  There's also nothing wrong with creating a
> new weak module from scratch, as one might do for a module that contains
> nothing but JPA model classes, and never evolving it into a strong
> module -- there's no benefit in doing that, so don't do it.

I definitely understand the intent, but I don't think intent will truly 
matter for this case as I've said above.

>> #3) Controlling reflection access is always bound up with controlling
>> exports
>>
>> The proposed exports system is simplistic but not exactly simple, yet
>> several useful modes remain left unaccounted-for.  Internally we
>> examined the usefulness of controlling access to public and so-called
>> "deep" reflection in combination with whether or not a member was
>> exported, and we came up with this logical table:
>>
>>           +------------------+---------------------------+
>>           |                  |        Module is...       |
>>           |  Reflection is... +------------+--------------+
>>           |                  |  Exported  | Not Exported |
>>           +----------------------------------------------+
>>           |        Forbidden | Not Useful |    Useful    |
>>           +----------------------------------------------+
>>           |      Public Only |   Useful   |    Useful    |
>>           +----------------------------------------------+
>>           | Public & Private |   Useful   |    Useful    |
>>           +------------------+------------+--------------+
>
> I'm not sure how to read this table.  Should the cell containing the text
> "Module is ..." instead contain "Member is ..."?

Right.

> By "exported" I'll assume that you mean non-reflective access, i.e., from
> the language at compile time and bytecodes at run time.

Yes.

> Do you have example use cases for each of the "Useful" cells?  Some are
> obvious, but others are not.

Sure, I'll go over each one by one with some example use cases.

Forbidden/Not Exported: Obviously the foundation case for 
access-oriented encapsulation, this is the default mode.

Exported modes:

Public Only/Exported: It doesn't make sense to disallow reflection on 
public types and members that can already be accessed directly, so this 
would be the default access mode to imported members when one module 
imports another.

Public & Private/Exported: This is the JDK 8 compatibility mode, but a 
module should have to opt into this mode, and it should be restricted to 
specific targets whenever possible, else the mechanism is essentially 
defeated.

Not exported modes:

Public Only/Not Exported: This is the mode that would be used (for 
example) by DI and ORM frameworks and containers which operate on public 
members.  Normally a module which imports such a framework would grant 
the necessary reflective access to it as well (though a full export is 
in this case unnecessary); the framework wouldn't contain requires 
clauses for modules which consume it.  Like I suggested in my first 
email, a way to "handshake" this agreement seems like it would be 
necessary: the framework would not allow itself to be imported without 
the importing module acknowledging the access in its configuration.

Public & Private/Not Exported: This is the mode that would be used by 
serialization and DI/ORM frameworks and containers which operate on 
private members (e.g. field-level injection).  Like the above though, 
this level of access is usually granted by the module importing the 
framework in question and should be explicitly opted into if possible.

The better situation however is to separate the grant from the export.

>> #4) There isn't always a container
>>
>> The justification for not providing an easy mechanism to restrict
>> private reflective access to a trusted framework is that a container can
>> always rewrite a module or inject classes.  However there isn't always a
>> container present - something which the existence of weak modules
>> acknowledges.  Some frameworks and middleware are meant to be usable
>> with or without a container, and it is undesirable to force these use
>> cases into a weak module situation or to require a strong coupling in
>> the module descriptor in this case.  Some other solution should be found
>> that works both for containers and "regular" applications.
>
> If you're running outside of a container then how important is it to
> precisely restrict access to packages meant for deep reflection?  If
> you're outside of a container then you have to trust the person (you?)
> who sets up your module path and/or class path.  To take the canonical
> case of a module that just contains JPA model classes, if you make it a
> weak module or else `exports private` the relevant packages without
> qualification so that the hibernate.core (or whatever) module can reflect
> into it, then the scope for trouble is limited.  Sure, some other module
> on the module path or even code on the class path could reflect into your
> module when it's not supposed to, but is it worth complicating the module
> system just to handle this scenario?

In either situation, the person (deployer) who sets up the class path or 
installation can currently do whatever they want.  Nothing in the new 
(or existing) security framework truly makes it possible to run 
untrusted code safely in this regard, nor to safely run trusted code in 
an untrusted environment, nor is it a sufficient solution for secure 
sandboxing even with a security manager present, with or without a 
container.  At best it's a mitigation strategy.

That said: the rise of microservices and reactive programming suggests 
that such standalone (containerless) deployments are, if anything, more 
popular than ever, but they may suffer from security problems which are 
very similar to, if not identical to, those which applications running 
in containers are subject to.  In fact jlink (and the new AOT work for 
that matter) seem particularly and specifically suited to this type of 
use case.

Finally: a JPA model class module is indeed probably harmless in any 
practical security sense, and locking it down is probably not useful 
from a security perspective, but even in this case it may yet be useful 
to restrict package exports such that only _supported_ classes are 
visible (especially when a module is published within an organization 
and contracts must be maintained); i.e. encapsulation is not solely a 
security concern, it also establishes intent as clearly as any interface 
declaration does.

> Don't get me wrong: Strong encapsulation is generally a very good thing.
> It's not always absolutely necessary, however, and we need to balance the
> number of ways in which it can be used against the overall complexity of
> the design.
>
> (It looks like there was no #5 in your list.)

Another genius stroke of... um, well, yeah.

>> #6) Not all reflection is 100% "friendly", and that's OK
>>
>> Serialization frameworks generally need private reflective access to the
>> module which contains the classes being serialized.  For some
>> frameworks, this access will already be adequate.  Some existing
>> frameworks utilize Unsafe to create uninitialized instances.  However
>> other frameworks, including frameworks that seek to comply with the
>> Serialization Specification [1], use ReflectionFactory, as that class
>> provides (exclusively, as far as I know) the ability to acquire a
>> constructor that produces a _partially_-initialized class where only the
>> constructors of superclasses to a certain depth are called.  It is our
>> view that it would be a regression to force such frameworks to use
>> --add-exports or JVMTI, and using Unsafe for this purpose, or continuing
>> to use it, (as I understand it) puts at risk the ability to perform
>> optimizations which depend on entry point control, which I am given to
>> understand are a strong motivator for many of the design constraints
>> that exist on the Jigsaw implementation.  It should be possible for
>> these frameworks to continue to use ReflectionFactory (unsupported
>> though it may be) if suitably privileged (in the security manager
>> sense), as long as all of the module(s) upon which the framework
>> reflects have granted private reflective access to it (in the module
>> declaration sense).
>
> So far as I know, this is already the case.  The `ReflectionFactory` API
> is specifically retained in JDK 9 for this very purpose, per JEP 260 [1].
> Do you have an example for which this does not work?

No, I had just interpreted previous messages as raising the possibility 
of closing this off so I wanted to capture the requirement.  Since this 
is not the case though, that's one less thing to worry about.  The 
recent enhancements to this class have also gone a long way to ease my 
concerns on this.

>> #7) More loose coupling seems necessary and useful
>>
>> In order for typical applications to function with modern middleware as
>> modules, without compromising security, it may be necessary to enhance
>> the loose coupling mechanism (uses/provides), or to provide an
>> additional, similar mechanism, which allows a symbolic coupling which
>> would allow modules to declare (in an abstract manner) modules which
>> need to have reflective access to it in a "friendly" manner.
>> Implementation ideas are forthcoming on jigsaw-dev.
>>
>> Providing a way within the system to grant public and/or private
>> reflection access in a specific manner, and doing so in a manner which
>> is easy to use and not prejudicial against existing or future middleware
>> and small or large applications which consume such middleware in the
>> Java SE or EE space, is our most basic requirement for satisfactory
>> resolution of this issue.
>
> To confirm my understanding, would it be correct to say that your main
> concern here is scenarios in which some of the classes in a user module
> are intended to be manipulated reflectively by some trusted framework
> module, but that framework module is either not known at compile time or
> is otherwise inconvenient to name in the declaration of the user module?

Correct.

We have observed over the past several years that the arrangement and 
structure of modules at deployment/run time generally diverges 
substantially from the structure of modules at compile time.  In order 
for compilation to have any kind of regularity, decoupling API from 
implementation wherever possible seems like a necessity, especially when 
you consider that JSR and other standards will want to be able to 
publish a simple contract for interfacing which is agnostic to 
implementation.  So really you can extend this notion beyond reflection; 
it's just that reflection is a particularly interesting case of the more 
general requirement.

In a practical sense, the most useful unit of coupling between modules 
is ABI.  When you depend on a module, that is really what you're relying 
on: an ABI which adheres to some contract.  Therefore it is useful to 
have names which correspond to ABIs as well as implementations (whether 
this is the same or a separate namespace doesn't really matter from this 
perspective).

>>                            I will follow up on jigsaw-dev with some
>> implementation thoughts and more specific requirements.
>
> Discussions of issues in the proposed design, and specific proposals
> for changing the design, belong here on the EG list.  The jigsaw-dev
> list is meant primarily for the discussion of issues outside the scope
> of the JSR such as implementation details, the design and implementation
> of JDK-specific APIs and tools, and feedback and questions arising from
> practical use of the prototype implementation.
>
> - Mark
>
>
> [1] http://openjdk.java.net/jeps/260
>

-- 
- DML