Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`

[Remi Forax]

----- Mail original -----
> De: "Mark Reinhold" <mark.reinhold at oracle.com>
> À: jpms-spec-experts at openjdk.java.net
> Envoyé: Mardi 28 Juin 2016 23:18:15
> Objet: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
> 
> Issue summary
> -------------
> 
>   #ReflectiveAccessToNonExportedTypes --- Some kinds of framework
>   libraries require reflective access to members of the non-exported
>   types of other modules; examples include dependency injection (Guice),
>   persistence (JPA), debugging tools, code-automation tools, and
>   serialization (XStream).  In some cases the particular library to be
>   used is not known until run time (e.g., Hibernate and EclipseLink both
>   implement JPA).  This capability is also sometimes used to work around
>   bugs in unchangeable code.  Access to non-exported packages can, at
>   present, only be done via command-line flags, which is extremely
>   awkward.  Provide an easier way for reflective code to access such
>   non-exported types. [1]
> 
> Proposal
> --------
> 
> Extend the language of module declarations so that a package can be
> declared to be exported at run time but not at compile time.  This is,
> roughly, the dual of the `requires static` construct proposed for
> #CompileTimeDependences, hence we propose to introduce a new modifier,
> `dynamic`, for use on the `exports` directive.  It has the following
> meanings:
> 
>   - At compile time, `exports dynamic P` does not cause the package `P`
>     to be exported, though it does require `P` to be a package defined
>     in the module.
> 
>   - In phases after compile time, `exports dynamic P` behaves in exactly
>     the same way as `exports P`.  It therefore takes part fully in
>     resolution and configuration, and is subject to the same consistency
>     constraints as normally-exported packages (e.g., no split packages).
> 
> Thus a module declaration of the form
> 
>     module com.foo.app {
>         requires hibernate.core;
>         requires hibernate.entitymanager;
>         exports dynamic com.foo.app.model;
>     }
> 
> makes the types in the `com.foo.app.model` package accessible at run time
> but not at compile time.  In combination with the earlier change to
> enable #ReflectionWithoutReadability [2] this means that frameworks that
> today use core reflection to manipulate user classes at run time are more
> likely to work out-of-the-box, without change, as automatic modules.  If
> the `com.foo.app.model` package in this example includes entity classes
> to be managed by Hibernate then the framework will be able to access them
> without further ado, but under normal circumstances an attempt to compile
> code that refers directly to those classes will fail.
> 
> The `dynamic` modifier can be applied to both unqualified and qualified
> `exports` directives, though the caveats on using qualified exports [3]
> still apply.
> 
> Notes
> -----
> 
>   - To access a non-public member in a dynamically-exported package you
>     must still invoke the appropriate `setAccessible` method, just as
>     you do today to access non-public members in an exported package.
> 
>   - This proposal is similar to Rémi's suggestion to add a way to export
>     a package only for reflection, still requiring that `setAccessible`
>     be invoked [4].  This proposal differs in that it does not require
>     `setAccessible` to be invoked to access public members in a
>     dynamically-exported package.
> 
>   - This proposal primarily addresses "friendly" uses of reflection, such
>     as dependency injection and persistence, in which the author of a
>     module knows in advance that one or more packages must be exported at
>     run time for reflective access by frameworks.  This proposal is also
>     convenient for frameworks that generate code at run time, since the
>     constant pool of a generated class file can include static references
>     to types in dynamically exported packages.
> 
>   - This proposal opens the door to a category of second-class APIs.  If
>     a package is exported dynamically then you can still compile code
>     that refers to types in the package, by using `-XaddExports` or its
>     equivalent at compile time, and it will work as expected at run time.
>     It thus may be useful to use `exports dynamic` for packages that
>     contain legacy APIs whose use is strongly discouraged, e.g., those in
>     the `jdk.unsupported` module of the reference implementation, thereby
>     forcing anyone who wants to compile against them to go to the trouble
>     of using `-XaddExports`.
> 
>   - Intrusive access to arbitrary packages of arbitrary modules by, e.g.,
>     debugging tools, will still require the use of sharp knives such as
>     the `-XaddExports` command-line option or its equivalent, or JVM TI.
> 
>   - Using the `-XaddExports` option or its equivalent remains awkward,
>     and sometimes it's the only way out.  To ease migration I think it's
>     worth considering some way for an application packaged as a JAR file
>     to include such options in its `MANIFEST.MF` file, as suggested by
>     Simon Nash [5].  I'll create a separate issue for that.
> 
>   - New kinds of resolution failures are possible.  If module `A`
>     requires `B`, and `B` requires `C`, then if `B` and `C` both declare
>     some package `P` to be exported dynamically, then a split-package
>     error will be reported.  This may surprise, since the packages are
>     just internal implementation details that were exported dynamically
>     so that some framework could access their types.  This is not,
>     however, all that different from the kinds of resolution and
>     layer-creation failures that are already possible due to package
>     collisions.  It's unlikely to happen all that often in practice,
>     and the fix is straightforward: Just use reverse-DNS or otherwise
>     sufficiently-unique package names, as most people already do.  It
>     is worth exploring whether javac can detect at least some of these
>     kinds of collisions and issue appropriate warnings.

I would like to agree on that proposal only on the ground that it requires to change the classfile format thus allows in the same time to fix the issues of the current classfile spec (encoding of the name module as a Class instead of UTF8, wrong flag for requires public), but i'm not sure this proposal support it's own weight.

Without dynamic, a 'classical' export works the same way at runtime so adding 'dynamic' to an export only restrict the usage at compile time.
Given that we can already selectively export packages to some chosen modules which hide them for the rest of the other modules at compile time,
adding dynamic to the spec adds little value. 

And it has several drawbacks, the main one is that it weaken the compiletime/runtime fidelity story, with a dynamic export the runtime world is not the compiletime world anymore.
The main motivation to add dynamic seems to be for backward compatibility during the transition between the classpath world and the full module world,
for several other cases, we use the command line for that, i don't see why this case is so special that it should not work like the other compatibility issues.

Moreover, an agent, an annotation processor and a jlink plugin are all able to implement the same trick, add export after the compilation and before the runtime,
so i don't think it's a good idea to change the spec for that.

Rémi

> 
> 
> [1]
> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccessToNonExportedTypes
> [2]
> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithoutReadability
> [3] http://openjdk.java.net/projects/jigsaw/spec/sotms/#qualified-exports
> [4]
> http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-December/000205.html
> [5]
> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005745.html
>