Proposal: #ReflectiveAccessToNonExportedTypes (revised) & #AwkwardStrongEncapsulation: Weak modules & private exports

[Tim Ellison]

mark.reinhold at oracle.com wrote on 11/10/2016 16:14:14:
> 2016/9/16 6:44:17 -0700, forax at univ-mlv.fr:
> > Hi all,
> > the goals of this proposal is
> > - to explicitly mark packages that allow 'deep' reflection,
> > - disable 'deep' reflection by default even for exported packages
> > - ease the transition by adding a new migration state between 
automatic
> >   module and strong module (weak module).
> > 
> > so 3 goals with one stone :)
> 
> Indeed!
> 
> > Have i miss something obvious but the last goal, easing migration is 
not
> > listed in the issue summary ?  While adding a new kind of module to 
ease
> > migration is maybe a good idea, it also add a new kind of modules when
> > thinking about the modules and their relationship, which seems a 
terrible
> > idea. We are loosing the KISS principle of the current module system.
> 
> I didn't list ease of migration in the issue summary but I didn't think
> that necessary, since it's an overarching goal of this entire effort.

Overall the concept of weak module does not provide an attractive 
migration 
step for existing module systems because it requires each module to 
"export 
private" all of its packages in order to retain existing behavior with 
respect to reflection.

This will effectively expose all of the internals of the existing module 
as 
if it were API at runtime and at compile/tool time.

It can be argued that even in existing modular systems there is a limited 
set of packages that must be allowed for deep reflection and the wholesale 

enablement of all packages concealed and exported for deep reflection is 
not needed.  But this is not the case when mapping an existing modular 
system's custom class loaders into a JPMS Layer (e.g. OSGi).  In that case 

we are motivated to preserve the existing behavior for the modules that 
represent the pre-existing modules/bundles.

> I understand your reluctance to introduce what appears to be another 
kind
> of module, but semantically one can read
> 
>     weak module foo.bar {
>         requires hibernate.core;
>         requires hibernate.entitymanager;
>     }
> 
> as a simple shorthand for the more verbose
> 
>     module foo.bar {
>         exports private com.foo.bar.model;
>         exports private ...;                // for other packages as 
needed
>         requires hibernate.core;
>         requires hibernate.entitymanager;
>     }
> 
> or, if we were to add wildcards, then
> 
>     module foo.bar {
>         exports private *;
>         requires hibernate.core;
>         requires hibernate.entitymanager;
>     }
> 
> The advantage of introducing weak modules as an explicit concept is that
> they nicely capture the reliable-configuration part of our story and, by
> their very name, explicitly do not promise strong encapsulation.
> 
> I think weak modules are, moreover, easier to explain to developers just
> starting to work with modules.  Telling someone that they have to type
> 
>     module foo.bar {
>         exports private *;
>     }
> 
> in order to make a simple module's packages available for deep 
reflection
> raises all kinds of questions.  What does `exports` mean?  What does
> `private` mean?  What does `*` range over?  It's all a bit like telling
> them that they need to type
> 
>     public class X {
>         public static void main(String... args) { ... }
>     }
> 
> in order to print "Hello, world!".  With weak modules you can simply
> explain that a weak module has no strong encapsulation and so will work
> whether it has a programmatic API or, as is the case here, it just
> contains model classes to be manipulated by a reflective framework.
> Strong encapsulation can be added later.

The declaration of a package as exported for tool-time linkage/compilation 

and normal runtime class loader delegation is orthogonal to the 
declaration 
of a package as available for 'deep' and/or 'shallow' reflection. 

The proposal mandates that 'shallow' reflection is only allowed on 
exported
packages, and any concealed packages are not available for any type of
reflection, so the use of the keyword 'private' also is confusing. 

Here the 'private' keyword is used to allow broader access to an exported 
package which is counter intuitive.  One would think a private export 
would 
be less accessible than a 'public' export, but in general the proposed 
semantics are troublesome regardless of the keywords used.

> > I fully agree that 'deep' reflection should be disallowed by default 
and can
> > be allowed package by package.  But as Stephen Colebourne, i fail to 
see why
> > enabling 'deep' reflection on a package is related to the export
> > keyword. While 'deep' reflection is somewhat related to visibility, 
it's not
> 
> (I assume you mean accessibility here, rather than visibility.)
> 
> > only about visibility of a class/method/field only, by example youcan 
change
> > the value of a public final field of a public class using 
setAccessible. In
> > my opinion, exposing the API to 'deep' reflection is a different 
concern as
> > exporting a package, thus using 'private' as a modifier on export is a 
bad
> > idea (it also doesn't work well with the notion of restricted export 
(export
> > to ...)).  Adding a new keyword 'expose' as Stephen propose to enable 
'deep'
> > reflection is a better way to answer to the goal 1 and 2.
> 
> Reflective access to the elements of packages, deep or otherwise, is
> intimately associated with language/bytecode-level access because the
> reflection API has always aimed to be an API-level simulation of what's
> possible at the language/bytecode level.  This is an extremely valuable
> property, since it makes it easy to reason about the correctness of
> reflective code.  (There are, obviously, some exceptions to this 
analogy,
> namely the long-standing `setAccessible` method and, more recently, the
> changes we made for #ReflectionWithoutReadability.)  Both the language
> and the reflection API are subject to the access-control rules of the
> module system, so using different keywords to express how they're made
> subject to those rules strikes me as confusing.
> 
> I've read Colebourne's proposal [1], and I can see its intuitive appeal
> to developers (like us!) who are used to thinking about all these 
things.
> To a new developer it might be easier to explain his `exposes *` than 
the
> above `exports private *`, but I think explaining weak modules is even
> easier.  A new developer is likely to hear the "reliable configuration +
> strong encapsulation" mantra fairly early on, and weak modules are 
simply
> the former, without the latter.

I disagree.  I like the Colebourne's approach because it keeps to two 
declarations ('exports' and 'exposes') as orthogonal concepts.  However,
I don't see the need to open the syntax up for wildcards '*' at this time.

> A couple of other problems I see with Colebourne's approach:
> 
>   - The only way to allow an exported API package to be used 
reflectively
>     is to expose it for deep reflection.  That requires typing another
>     directive and, worse, makes the internals of that package available
>     for deep reflection, which is probably not what was intended.
> 
>   - The interactions between `exports`, `exposes`, and the qualified
>     forms of these directives turns out to be pretty complicated once 
you
>     work out all the cases.  (We've done this internally, for a somewhat
>     similar proposal that allows both `private` and `dynamic` modifiers
>     on the `exports` directive.)  They wind up being nearly as bad as 
the
>     existing rules for protected members [2].  I'm not convinced that
>     this relatively small bit of expressive power is worth the 
additional
>     complexity.

It would be interesting to see that exploration exposed, because as you 
say
the Colebourne expression appears intuitive.

> > About the other proposed items:
> >   - using transitive instead of public for require, i fully agree.
> 
> Good.
> 
> >   - about removing the dynamic exports feature, a package declared as 
non
> >     exported but declared as exposed will have the same meaning, 
> no ? if yes,
> >     i agree that the notion dynamic export should be removed.
> 
> I'm not sure if you're thinking of my proposal here, or Colebourne's.
> 
> In my proposal you can't "expose" a package for reflection without also
> exporting it.  If you export a package then it's available for shallow
> reflection at run time; if you add the `private` modifier then it's also
> available for deep reflection via `setAccessible`.

As discussed, it would be useful to explore the options for changing the
proposal's constraint that you can't "expose" a package for reflection 
without 
also exporting it.

Regards,
Tim

> As I mentioned in one of the notes in the proposal, removing `dynamic`
> does remove some expressive power, since you can no longer export a
> package only at run time.  This means that an IDE or a compiler cannot
> (as easily) prevent you from writing code against a module that 
contains,
> e.g., JPA POJOs that are not also APIs (which most aren't).  I 
understand
> the use case, but the cost of supporting it by retaining the `dynamic`
> modifier (or its semantic equivalent) is rather high, as I noted above.
> If experience shows that it's really necessary, we could add it in a
> future release.
> 
> - Mark
> 
> 
> [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-
> September/009370.html
> [2] 
http://docs.oracle.com/javase/specs/jls/se8/html/jls-6.html#jls-6.6.2
> 

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU