Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`

Thu Jun 30 21:36:34 UTC 2016

 Hi!

> In my opinion, reflection should not apply any module-boundary
> checks.

Maybe this list is not a representative selection of Java developers,
maybe I'm just the only one thinking this but, to paraphrase:

Reflection over internals must die in a fire!

Yes, yes, tools to great things, we need to see everything, ... Still,
I think that modules need a safe space that they can use without
anybody intruding. Maybe I'm a hardliner but I already view the
command line parameters with suspicion.

In that light, I like the "dynamic" proposal.

Not the name, though. Having the symmetric pair "static vs dynamic"
looks nice but it takes a lot of words to explain why exactly these
terms apply here. We seem to be looking for qualifiers for "requires"
and "exports". Maybe just using other terms solves the problem?

	requires & consumes
	exports & contains

 so long ... Nicolai

On 30.06.2016 01:23, Rafael Winterhalter wrote:
> I also agree here that the use of reflection should not require any
> form of explicit permission by the user of a library. In a way,
> this ties an implementation detail (reflection in this case) to the
> usage of the library. The library cannot work without the explicit
> configuration by a user and this only adds boilerplate to using
> Hibernate.
> 
> Unfortunately, many Java developers still do not know much about
> reflection or about the inner workings of many libraries in the
> market. If a user needed to explicitly export packages, this might
> just lead to people "over exporting" their packages just to "make
> Hibernate work".
> 
> In my opinion, reflection should not apply any module-boundary
> checks.
> 
> Best regards, Rafael
> 
> 2016-06-29 20:13 GMT+02:00 Paul Benedict <pbenedict at apache.org>:
> 
>> Replying to "observers" list...
>> 
>> I think the main problem here is that core reflection has become
>> too restrictive. Reflection should be "magical" and have no
>> knowledge of any kind of module and/or visibility boundaries. If
>> you really want to control what can be reflected upon, that
>> should be done via the Policy file of the Security Manager.
>> 
>> This proposal is trying to hack out-of-the-box created by too
>> strict of a design. It is very awkward to denote which packages
>> must be reflectable ahead of time --- how can you figure that
>> out?? The consumers who are doing the reflection are innumerable.
>> I'd go as far to say it's it's impossible to devise a preemptive
>> comprehensive strategy that may satisfy the kinds of tooling in
>> the wild.
>> 
>> It's best just to open up reflection so it can continue doing
>> what it always has done well: let me see anything I want unless
>> the Security Manager says no.
>> 
>> Cheers, Paul
>> 
>> On Wed, Jun 29, 2016 at 3:51 AM, Remi Forax <forax at univ-mlv.fr>
>> wrote:
>> 
>>> 
>>> 
>>> ----- Mail original -----
>>>> De: "Mark Reinhold" <mark.reinhold at oracle.com> À:
>>>> jpms-spec-experts at openjdk.java.net Envoyé: Mardi 28 Juin 2016
>>>> 23:18:15 Objet: Proposal:
>>>> #ReflectiveAccessToNonExportedTypes: `exports dynamic`
>>>> 
>>>> Issue summary -------------
>>>> 
>>>> #ReflectiveAccessToNonExportedTypes --- Some kinds of
>>>> framework libraries require reflective access to members of
>>>> the non-exported types of other modules; examples include
>>>> dependency injection
>> (Guice),
>>>> persistence (JPA), debugging tools, code-automation tools,
>>>> and serialization (XStream).  In some cases the particular
>>>> library to be used is not known until run time (e.g.,
>>>> Hibernate and EclipseLink
>> both
>>>> implement JPA).  This capability is also sometimes used to
>>>> work
>> around
>>>> bugs in unchangeable code.  Access to non-exported packages
>>>> can, at present, only be done via command-line flags, which
>>>> is extremely awkward.  Provide an easier way for reflective
>>>> code to access such non-exported types. [1]
>>>> 
>>>> Proposal --------
>>>> 
>>>> Extend the language of module declarations so that a package
>>>> can be declared to be exported at run time but not at compile
>>>> time.  This is, roughly, the dual of the `requires static`
>>>> construct proposed for #CompileTimeDependences, hence we
>>>> propose to introduce a new modifier, `dynamic`, for use on
>>>> the `exports` directive.  It has the following meanings:
>>>> 
>>>> - At compile time, `exports dynamic P` does not cause the
>>>> package `P` to be exported, though it does require `P` to be
>>>> a package defined in the module.
>>>> 
>>>> - In phases after compile time, `exports dynamic P` behaves
>>>> in
>> exactly
>>>> the same way as `exports P`.  It therefore takes part fully
>>>> in resolution and configuration, and is subject to the same
>> consistency
>>>> constraints as normally-exported packages (e.g., no split
>> packages).
>>>> 
>>>> Thus a module declaration of the form
>>>> 
>>>> module com.foo.app { requires hibernate.core; requires
>>>> hibernate.entitymanager; exports dynamic com.foo.app.model; 
>>>> }
>>>> 
>>>> makes the types in the `com.foo.app.model` package accessible
>>>> at run
>> time
>>>> but not at compile time.  In combination with the earlier
>>>> change to enable #ReflectionWithoutReadability [2] this means
>>>> that frameworks
>> that
>>>> today use core reflection to manipulate user classes at run
>>>> time are
>> more
>>>> likely to work out-of-the-box, without change, as automatic
>>>> modules.
>> If
>>>> the `com.foo.app.model` package in this example includes
>>>> entity classes to be managed by Hibernate then the framework
>>>> will be able to access
>> them
>>>> without further ado, but under normal circumstances an
>>>> attempt to
>> compile
>>>> code that refers directly to those classes will fail.
>>>> 
>>>> The `dynamic` modifier can be applied to both unqualified and
>>>> qualified `exports` directives, though the caveats on using
>>>> qualified exports [3] still apply.
>>>> 
>>>> Notes -----
>>>> 
>>>> - To access a non-public member in a dynamically-exported
>>>> package you must still invoke the appropriate `setAccessible`
>>>> method, just as you do today to access non-public members in
>>>> an exported package.
>>>> 
>>>> - This proposal is similar to Rémi's suggestion to add a way
>>>> to
>> export
>>>> a package only for reflection, still requiring that
>>>> `setAccessible` be invoked [4].  This proposal differs in
>>>> that it does not require `setAccessible` to be invoked to
>>>> access public members in a dynamically-exported package.
>>>> 
>>>> - This proposal primarily addresses "friendly" uses of
>>>> reflection,
>> such
>>>> as dependency injection and persistence, in which the author
>>>> of a module knows in advance that one or more packages must
>>>> be exported
>> at
>>>> run time for reflective access by frameworks.  This proposal
>>>> is
>> also
>>>> convenient for frameworks that generate code at run time,
>>>> since the constant pool of a generated class file can include
>>>> static
>> references
>>>> to types in dynamically exported packages.
>>>> 
>>>> - This proposal opens the door to a category of second-class
>>>> APIs.
>> If
>>>> a package is exported dynamically then you can still compile
>>>> code that refers to types in the package, by using
>>>> `-XaddExports` or its equivalent at compile time, and it will
>>>> work as expected at run
>> time.
>>>> It thus may be useful to use `exports dynamic` for packages
>>>> that contain legacy APIs whose use is strongly discouraged,
>>>> e.g., those
>> in
>>>> the `jdk.unsupported` module of the reference
>>>> implementation,
>> thereby
>>>> forcing anyone who wants to compile against them to go to
>>>> the
>> trouble
>>>> of using `-XaddExports`.
>>>> 
>>>> - Intrusive access to arbitrary packages of arbitrary modules
>>>> by,
>> e.g.,
>>>> debugging tools, will still require the use of sharp knives
>>>> such as the `-XaddExports` command-line option or its
>>>> equivalent, or JVM
>> TI.
>>>> 
>>>> - Using the `-XaddExports` option or its equivalent remains
>>>> awkward, and sometimes it's the only way out.  To ease
>>>> migration I think
>> it's
>>>> worth considering some way for an application packaged as a
>>>> JAR
>> file
>>>> to include such options in its `MANIFEST.MF` file, as
>>>> suggested by Simon Nash [5].  I'll create a separate issue
>>>> for that.
>>>> 
>>>> - New kinds of resolution failures are possible.  If module
>>>> `A` requires `B`, and `B` requires `C`, then if `B` and `C`
>>>> both
>> declare
>>>> some package `P` to be exported dynamically, then a
>>>> split-package error will be reported.  This may surprise,
>>>> since the packages are just internal implementation details
>>>> that were exported dynamically so that some framework could
>>>> access their types.  This is not, however, all that different
>>>> from the kinds of resolution and layer-creation failures that
>>>> are already possible due to package collisions.  It's
>>>> unlikely to happen all that often in practice, and the fix is
>>>> straightforward: Just use reverse-DNS or otherwise 
>>>> sufficiently-unique package names, as most people already do.
>>>> It is worth exploring whether javac can detect at least some
>>>> of these kinds of collisions and issue appropriate warnings.
>>> 
>>> I would like to agree on that proposal only on the ground that
>>> it
>> requires
>>> to change the classfile format thus allows in the same time to
>>> fix the issues of the current classfile spec (encoding of the
>>> name module as a Class instead of UTF8, wrong flag for requires
>>> public), but i'm not sure this proposal support it's own
>>> weight.
>>> 
>>> Without dynamic, a 'classical' export works the same way at
>>> runtime so adding 'dynamic' to an export only restrict the
>>> usage at compile time. Given that we can already selectively
>>> export packages to some chosen modules which hide them for the
>>> rest of the other modules at compile
>> time,
>>> adding dynamic to the spec adds little value.
>>> 
>>> And it has several drawbacks, the main one is that it weaken
>>> the compiletime/runtime fidelity story, with a dynamic export
>>> the runtime
>> world
>>> is not the compiletime world anymore. The main motivation to
>>> add dynamic seems to be for backward compatibility during the
>>> transition between the classpath world and the full module
>> world,
>>> for several other cases, we use the command line for that, i
>>> don't see
>> why
>>> this case is so special that it should not work like the other 
>>> compatibility issues.
>>> 
>>> Moreover, an agent, an annotation processor and a jlink plugin
>>> are all able to implement the same trick, add export after the
>>> compilation and before the runtime, so i don't think it's a
>>> good idea to change the spec for that.
>>> 
>>> Rémi
>>> 
>>>> 
>>>> 
>>>> [1]
>>>> 
>>> 
>> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccess
ToNonExportedTypes
>>>>
>> 
[2]
>>>> 
>>> 
>> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithou
tReadability
>>>>
>> 
[3]
>>> http://openjdk.java.net/projects/jigsaw/spec/sotms/#qualified-export
s
>>>>
>>> 
[4]
>>>> 
>>> 
>> http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-Decembe
r/000205.html
>>>>
>> 
[5]
>>>> 
>>> 
>> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/00574
5.html
>>>>
>>>
>>
>
>> 
-- 

PGP Key:
    http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509

Web:
    http://codefx.org
        a blog about software development
    http://do-foss.de
        Free and Open Source Software for the City of Dortmund

Twitter:
    https://twitter.com/nipafx