Proposal (revised^2): #ReflectiveAccessToNonExportedTypes & #AwkwardStrongEncapsulation: Open modules & open packages

Wed Nov 16 16:39:10 UTC 2016

2016/11/7 7:08:15 -0800, david.lloyd at redhat.com:
> I was waiting on a response to an earlier question about readability, 
> but I'm pretty sure I know the answer so I'm going to plow ahead.
> 
> On 10/27/2016 10:53 AM, mark.reinhold at oracle.com wrote:
>> ...
>> 
>> To open specific packages in normal module declarations we introduce a
>> new per-package directive, `opens`.  A package in a normal module can be
>> opened, or exported, or both, as follows:
>> 
>>  - If a package is only opened (`opens p;`) then properties (1), (2),
>>    and (3) hold, i.e., its types are not accessible at compile time,
>>    it does not affect resolution, and all of its elements are available
>>    for deep reflection at run time.
>> 
>>  - If a package is only exported (`exports p;`) then those properties
>>    do not hold, and it is exported exactly as it is today.
>> 
>>  - If a package is both exported and opened (`exports p; opens p;`)
>>    then it is exported as today and is, additionally, available for
>>    deep reflection at run time (3).
> 
> I'd like to stop here and ask a bit about deep reflection, based on some 
> discussion happening on jigsaw-dev.
> 
> It was my assumption that "deep reflection" implies any indirect access 
> to nonpublic members - which includes MethodHandle style access as well 
> as classical reflection.  So here are some questions:
> 
> 1) I assume that one will continue to be able to unreflect any 
> AccessibleObject which has had accessibility enabled and get 
> MethodHandles which have deep reflective access.  Is this assumption
> true?

Yes.  This proposal does not change how method handles work.

> 2) Is it a goal to ultimately eliminate setAccessible()?  In other 
> words, is this the direction that Java security is moving?  (No wrong 
> answer here, I just haven't been able to glean a clear yes/no answer to 
> this and it's definitely important to guide my thinking on this issue 
> since I'm mentally trying to follow two paths at once on this.)

There is no specific plan as yet.  In the long run we (as in, "many of us
who work on the JDK") do want to move away from the difficult-to-secure
identity-based core-reflection API and toward the easier-to-secure
capability-based method-handle API, or perhaps to some new mirror-based
API as yet undefined.

> 3) Given nonpublic member M in an opened package; will/could/should it 
> be possible for Lookups (belonging to classes which observe the 
> "openness" of M's package) to be able to directly create or acquire 
> MethodHandles for M without using reflection (especially if the answer 
> to #2 is "yes") and without relying on donated Lookups?

As things stand today, no.  To support this would require defining a new
`MethodHandles.Lookup` factory that grants deep access to packages that
are open to the caller.  We could do that or, as I mentioned earlier [1],
we could take the simpler approach of requiring the client of a framework
to pass (i.e., "donate") an appropriate Lookup object when initializing
the framework.  I gather from your comments on jigsaw-dev [2] that you'd
prefer the former approach, and I agree that it'd require much less
change on the part of framework users.  I have an idea for how to do
this which I'll write up soon, for #IndirectQualifiedReflectiveAccess.

>> ...
>> 
>> Both the `exports` and `opens` directives can be qualified, so that a
>> package is exported or opened only to certain other named modules.  As
>> before, duplicate directives are not permitted in order to ensure easy
>> readability.  At most one `exports` directive is relevant to any given
>> package, and at most one `opens` directive is relevant to any given
>> package.
> 
> Do module names specified in qualified "opens" directives need to be 
> present at compile or run time, or will absent module names be ignored?

The latter, as noted in [3]:

  It is permitted for the `to` clause of an `exports` or `opens`
  statement to specify a module which is not observable.

This is true in all phases.  (If it weren't then we couldn't compile
java.base, or run an image that contains just that module, since it
has qualified exports to many other JDK modules.)

>> ...
>> 
>> The existing syntax of `requires public` has long been confusing, so we
>> here take the opportunity to fix that problem by renaming the `public`
>> modifier in `requires` directives to `transitive`.  Thus the declaration
>> 
>>    module foo.bar {
>>        exports com.foo.bar;
>>        requires public java.sql;
>>    }
>> 
>> is now written
>> 
>>    module foo.bar {
>>        exports com.foo.bar;
>>        requires transitive java.sql;
>>    }
> 
> Just to clarify, since this isn't really clearly spelled out in SOTMS or 
> in any other doc at present (I think?), "requires transitive" means 
> exactly that all packages exported by "java.sql" are also exported by 
> "foo.bar"

No, those packages are not in any sense exported by `foo.bar`.

Remember that `requires`, and hence `requires transitive`, only affect
readability.  In the above example, any module that reads `foo.bar` will
also be made to read `java.sql`, whether or not it does so explicitly
itself.

Readability, together with `exports` and `opens` directives, governs
access control.  A module that reads `foo.bar` will also read `java.sql`,
and will thus have access to all of the exported packages of `java.sql`.

>          , and that there is no way to narrow the set of packages in 
> this case, correct?

`requires transitive` isn't a re-export operator, so there's no way to
narrow the set of packages conveyed in this manner.

> Also, I assume that requiring with "transitive" does not imply that 
> "openness" is also transitively exposed, i.e. openness is always 
> non-transitive.  Is that a correct assumption?

No, though the details differ depending upon whether you start with core
reflection or with method handles.

Core reflection assumes readability (cf. #ReflectionWithoutReadability),
so `requires` (and hence `requires transitive`) isn't really relevant.
If a package is open to some module then code in that module can use core
reflection and `setAccessible` to access private members regardless of
readability relationships.  The same would be true of a method handle
unreflected from a core-reflection object upon which `setAccessible` has
been invoked.

A pure method-handles approach would, by contrast, respect readability.
With the method-handl API as it stands today, if module A opens package
P, and some other module B reads A, and code in B creates a `Lookup`
object, then B will have reflective method-handle access to public (but
not private) members of package P in A.  B will read A if one of the
following conditions is true:

  - B `requires` A
  - B `requires` C, and C `requires transitive` A, or
  - B does not `requires` A but somehow obtains a reads edge to A
    at run time (e.g., via `Module::addReads`).

In the second case one can say that "openness" is conveyed transitively.

> Overall this is a very good proposal, and I think (unless the answer to 
> one of the above questions is very surprising) that we (Red Hat) are 
> close to agreement on it.

Glad to hear it.

- Mark

[1] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-October/000431.html
[2] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-November/009894.html
[3] http://cr.openjdk.java.net/~mr/jigsaw/spec/lang-vm.html