Proposal: #ReflectiveAccessToNonExportedTypes (revised) & #AwkwardStrongEncapsulation: Weak modules & private exports

[mark.reinhold at oracle.com]

2016/9/16 6:44:17 -0700, forax at univ-mlv.fr:
> Hi all,
> the goals of this proposal is
> - to explicitly mark packages that allow 'deep' reflection,
> - disable 'deep' reflection by default even for exported packages
> - ease the transition by adding a new migration state between automatic
>   module and strong module (weak module).
> 
> so 3 goals with one stone :)

Indeed!

> Have i miss something obvious but the last goal, easing migration is not
> listed in the issue summary ?  While adding a new kind of module to ease
> migration is maybe a good idea, it also add a new kind of modules when
> thinking about the modules and their relationship, which seems a terrible
> idea. We are loosing the KISS principle of the current module system.

I didn't list ease of migration in the issue summary but I didn't think
that necessary, since it's an overarching goal of this entire effort.

I understand your reluctance to introduce what appears to be another kind
of module, but semantically one can read

    weak module foo.bar {
        requires hibernate.core;
        requires hibernate.entitymanager;
    }

as a simple shorthand for the more verbose

    module foo.bar {
        exports private com.foo.bar.model;
        exports private ...;                // for other packages as needed
        requires hibernate.core;
        requires hibernate.entitymanager;
    }

or, if we were to add wildcards, then

    module foo.bar {
        exports private *;
        requires hibernate.core;
        requires hibernate.entitymanager;
    }

The advantage of introducing weak modules as an explicit concept is that
they nicely capture the reliable-configuration part of our story and, by
their very name, explicitly do not promise strong encapsulation.

I think weak modules are, moreover, easier to explain to developers just
starting to work with modules.  Telling someone that they have to type

    module foo.bar {
        exports private *;
    }

in order to make a simple module's packages available for deep reflection
raises all kinds of questions.  What does `exports` mean?  What does
`private` mean?  What does `*` range over?  It's all a bit like telling
them that they need to type

    public class X {
        public static void main(String... args) { ... }
    }

in order to print "Hello, world!".  With weak modules you can simply
explain that a weak module has no strong encapsulation and so will work
whether it has a programmatic API or, as is the case here, it just
contains model classes to be manipulated by a reflective framework.
Strong encapsulation can be added later.

> I fully agree that 'deep' reflection should be disallowed by default and can
> be allowed package by package.  But as Stephen Colebourne, i fail to see why
> enabling 'deep' reflection on a package is related to the export
> keyword. While 'deep' reflection is somewhat related to visibility, it's not

(I assume you mean accessibility here, rather than visibility.)

> only about visibility of a class/method/field only, by example you can change
> the value of a public final field of a public class using setAccessible. In
> my opinion, exposing the API to 'deep' reflection is a different concern as
> exporting a package, thus using 'private' as a modifier on export is a bad
> idea (it also doesn't work well with the notion of restricted export (export
> to ...)).  Adding a new keyword 'expose' as Stephen propose to enable 'deep'
> reflection is a better way to answer to the goal 1 and 2.

Reflective access to the elements of packages, deep or otherwise, is
intimately associated with language/bytecode-level access because the
reflection API has always aimed to be an API-level simulation of what's
possible at the language/bytecode level.  This is an extremely valuable
property, since it makes it easy to reason about the correctness of
reflective code.  (There are, obviously, some exceptions to this analogy,
namely the long-standing `setAccessible` method and, more recently, the
changes we made for #ReflectionWithoutReadability.)  Both the language
and the reflection API are subject to the access-control rules of the
module system, so using different keywords to express how they're made
subject to those rules strikes me as confusing.

I've read Colebourne's proposal [1], and I can see its intuitive appeal
to developers (like us!) who are used to thinking about all these things.
To a new developer it might be easier to explain his `exposes *` than the
above `exports private *`, but I think explaining weak modules is even
easier.  A new developer is likely to hear the "reliable configuration +
strong encapsulation" mantra fairly early on, and weak modules are simply
the former, without the latter.

A couple of other problems I see with Colebourne's approach:

  - The only way to allow an exported API package to be used reflectively
    is to expose it for deep reflection.  That requires typing another
    directive and, worse, makes the internals of that package available
    for deep reflection, which is probably not what was intended.

  - The interactions between `exports`, `exposes`, and the qualified
    forms of these directives turns out to be pretty complicated once you
    work out all the cases.  (We've done this internally, for a somewhat
    similar proposal that allows both `private` and `dynamic` modifiers
    on the `exports` directive.)  They wind up being nearly as bad as the
    existing rules for protected members [2].  I'm not convinced that
    this relatively small bit of expressive power is worth the additional
    complexity.

> About the other proposed items:
>   - using transitive instead of public for require, i fully agree.

Good.

>   - about removing the dynamic exports feature, a package declared as non
>     exported but declared as exposed will have the same meaning, no ? if yes,
>     i agree that the notion dynamic export should be removed.

I'm not sure if you're thinking of my proposal here, or Colebourne's.

In my proposal you can't "expose" a package for reflection without also
exporting it.  If you export a package then it's available for shallow
reflection at run time; if you add the `private` modifier then it's also
available for deep reflection via `setAccessible`.

As I mentioned in one of the notes in the proposal, removing `dynamic`
does remove some expressive power, since you can no longer export a
package only at run time.  This means that an IDE or a compiler cannot
(as easily) prevent you from writing code against a module that contains,
e.g., JPA POJOs that are not also APIs (which most aren't).  I understand
the use case, but the cost of supporting it by retaining the `dynamic`
modifier (or its semantic equivalent) is rather high, as I noted above.
If experience shows that it's really necessary, we could add it in a
future release.

- Mark


[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-September/009370.html
[2] http://docs.oracle.com/javase/specs/jls/se8/html/jls-6.html#jls-6.6.2