Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`

Fri Jul 1 08:41:48 UTC 2016

Hi all,

regarding exporting DTO to hibernate..

I think the whole point of having JPA is to not have explicit dependency to
hibernate..

So if I want to make my DTO's visible the declaration should be towards
JPA..

-Sven
Am 01.07.2016 09:28 schrieb "Nicolai Parlog" <nipa at codefx.org>:

>  Hi!
>
> > I do not think that shielding internals is a problem with regards
> > to reflection.
>
> I think it is. I've not been emerged in the community for very long so
> I know only two examples (but still): Unsafe and JUnit 4. In both
> cases internals were accessed via reflection. And in both cases
> dependencies became so ubiquitous that changing any details became
> impossible for the maintainers.
>
> I think the outcry when Unsafe was planned to go away and the fact
> that JUnit 4 had to be rewritten because evolution was effectively
> prevented by unwanted dependencies make my point here.
>
> > I do neither think that reflection only interests internals.
>
> I agree. Not "only" but "all too often", though.
>
> > For example, maybe you use Hibernate for your persistance layer
> > without exposing those DTOs from your module.
>
> Why would I do that?! This is exactly where reflection got us used to
> sloppy architecture. If a tool uses my DTOs to implement my
> persistence layer, then it looks like those DTOs are part of my API.
> Maybe not public to everyone but qualified and dynamic exports give me
> tools to reduce visibility.
>
> > I do not care about how Hibernate works as long as it does work.
> > Almost any library uses reflection and I would not want to dig
> > into implementation details to find out how to set up my dynamic
> > exports.
>
> Maybe I'm missing something here but to me it looks like "export to a
> module whatever that module is supposed to interact with" is a very
> sensible default that should cover a vast majority of uses cases, no?
>
>         I have DTOs && Hibernate needs DTOs -> export DTOs to Hibernate.
>
> Why would I look at the implementation? And if there were non-obvious
> scenarios they should be properly documented so I can react accordingly.
>
> > As a library implementor, you aim for the biggest convenience.
>
> Maybe we should aim for clarity and well-defined interactions instead.
>
>  so long ... Nicolai
>
>
>
> On 01.07.2016 00:20, Rafael Winterhalter wrote:
> > I disagree on two ends here.
> >
> > I do not think that shielding internals is a problem with regards
> > to reflection. You want to shield internals in order to not break
> > dependants by prohibiting their existance. With reflection, you do
> > however typically react to the actual implementation by discovering
> > it at runtime. If your use pattern was fixed, you would always
> > require a static dependency over reflection in the first place. I
> > do neither think that reflection only interests internals. For
> > example, maybe you use Hibernate for your persistance layer without
> > exposing those DTOs from your module. The problem is that you
> > invert responsibilities. I do not care about how Hibernate works as
> > long as it does work. Almost any library uses reflection and I
> > would not want to dig into implementation details to find out how
> > to set up my dynamic exports. I want to require a module and
> > hopefully the compiler tells me about anything that's wrong.
> > Reflection access errors would only be an awkward side effect that
> > can be avoided by not applying modules alltogether. I think this
> > limitation would therefore hurt Java 9 adoption.
> >
> > The second problem I have is the issue of attempting to fix an old
> > regret. From the perspective of Hibernate, it would be easiest to
> > not attempt modules as their introduction would require its users
> > to apply additional configuration. As a library implementor, you
> > aim for the biggest convenience. Adding this restriction neither
> > adds additional security where a security manager should be used. I
> > again think this would hurt adoption.
> >
> > I agree with Stephen's argument even though I do not think that
> > forbidding dynamic access would be a feature anybody would use so
> > it should not be added. Rather, something should be integrated in
> > the security manager, if this is considered necessary.
> >
> > Cheers, Rafael
> >
> > Am 30.06.2016 11:37 nachm. schrieb "Nicolai Parlog"
> > <nipa at codefx.org>:
> >>
> >> Hi!
> >>
> >>> In my opinion, reflection should not apply any module-boundary
> >>> checks.
> >>
> >> Maybe this list is not a representative selection of Java
> >> developers, maybe I'm just the only one thinking this but, to
> >> paraphrase:
> >>
> >> Reflection over internals must die in a fire!
> >>
> >> Yes, yes, tools to great things, we need to see everything, ...
> >> Still, I think that modules need a safe space that they can use
> >> without anybody intruding. Maybe I'm a hardliner but I already
> >> view the command line parameters with suspicion.
> >>
> >> In that light, I like the "dynamic" proposal.
> >>
> >> Not the name, though. Having the symmetric pair "static vs
> >> dynamic" looks nice but it takes a lot of words to explain why
> >> exactly these terms apply here. We seem to be looking for
> >> qualifiers for "requires" and "exports". Maybe just using other
> >> terms solves the problem?
> >>
> >> requires & consumes exports & contains
> >>
> >> so long ... Nicolai
> >>
> >>
> >>
> >> On 30.06.2016 01:23, Rafael Winterhalter wrote:
> >>> I also agree here that the use of reflection should not require
> >>> any form of explicit permission by the user of a library. In a
> >>> way, this ties an implementation detail (reflection in this
> >>> case) to the usage of the library. The library cannot work
> >>> without the explicit configuration by a user and this only adds
> >>> boilerplate to using Hibernate.
> >>>
> >>> Unfortunately, many Java developers still do not know much
> >>> about reflection or about the inner workings of many libraries
> >>> in the market. If a user needed to explicitly export packages,
> >>> this might just lead to people "over exporting" their packages
> >>> just to "make Hibernate work".
> >>>
> >>> In my opinion, reflection should not apply any module-boundary
> >>> checks.
> >>>
> >>> Best regards, Rafael
> >>>
> >>> 2016-06-29 20:13 GMT+02:00 Paul Benedict
> >>> <pbenedict at apache.org>:
> >>>
> >>>> Replying to "observers" list...
> >>>>
> >>>> I think the main problem here is that core reflection has
> >>>> become too restrictive. Reflection should be "magical" and
> >>>> have no knowledge of any kind of module and/or visibility
> >>>> boundaries. If you really want to control what can be
> >>>> reflected upon, that should be done via the Policy file of
> >>>> the Security Manager.
> >>>>
> >>>> This proposal is trying to hack out-of-the-box created by
> >>>> too strict of a design. It is very awkward to denote which
> >>>> packages must be reflectable ahead of time --- how can you
> >>>> figure that out?? The consumers who are doing the reflection
> >>>> are innumerable. I'd go as far to say it's it's impossible to
> >>>> devise a preemptive comprehensive strategy that may satisfy
> >>>> the kinds of tooling in the wild.
> >>>>
> >>>> It's best just to open up reflection so it can continue
> >>>> doing what it always has done well: let me see anything I
> >>>> want unless the Security Manager says no.
> >>>>
> >>>> Cheers, Paul
> >>>>
> >>>> On Wed, Jun 29, 2016 at 3:51 AM, Remi Forax
> >>>> <forax at univ-mlv.fr> wrote:
> >>>>
> >>>>>
> >>>>>
> >>>>> ----- Mail original -----
> >>>>>> De: "Mark Reinhold" <mark.reinhold at oracle.com> À:
> >>>>>> jpms-spec-experts at openjdk.java.net Envoyé: Mardi 28 Juin
> >>>>>> 2016 23:18:15 Objet: Proposal:
> >>>>>> #ReflectiveAccessToNonExportedTypes: `exports dynamic`
> >>>>>>
> >>>>>> Issue summary -------------
> >>>>>>
> >>>>>> #ReflectiveAccessToNonExportedTypes --- Some kinds of
> >>>>>> framework libraries require reflective access to members
> >>>>>> of the non-exported types of other modules; examples
> >>>>>> include dependency injection
> >>>> (Guice),
> >>>>>> persistence (JPA), debugging tools, code-automation
> >>>>>> tools, and serialization (XStream).  In some cases the
> >>>>>> particular library to be used is not known until run time
> >>>>>> (e.g., Hibernate and EclipseLink
> >>>> both
> >>>>>> implement JPA).  This capability is also sometimes used
> >>>>>> to work
> >>>> around
> >>>>>> bugs in unchangeable code.  Access to non-exported
> >>>>>> packages can, at present, only be done via command-line
> >>>>>> flags, which is extremely awkward.  Provide an easier way
> >>>>>> for reflective code to access such non-exported types.
> >>>>>> [1]
> >>>>>>
> >>>>>> Proposal --------
> >>>>>>
> >>>>>> Extend the language of module declarations so that a
> >>>>>> package can be declared to be exported at run time but
> >>>>>> not at compile time.  This is, roughly, the dual of the
> >>>>>> `requires static` construct proposed for
> >>>>>> #CompileTimeDependences, hence we propose to introduce a
> >>>>>> new modifier, `dynamic`, for use on the `exports`
> >>>>>> directive.  It has the following meanings:
> >>>>>>
> >>>>>> - At compile time, `exports dynamic P` does not cause
> >>>>>> the package `P` to be exported, though it does require
> >>>>>> `P` to be a package defined in the module.
> >>>>>>
> >>>>>> - In phases after compile time, `exports dynamic P`
> >>>>>> behaves in
> >>>> exactly
> >>>>>> the same way as `exports P`.  It therefore takes part
> >>>>>> fully in resolution and configuration, and is subject to
> >>>>>> the same
> >>>> consistency
> >>>>>> constraints as normally-exported packages (e.g., no
> >>>>>> split
> >>>> packages).
> >>>>>>
> >>>>>> Thus a module declaration of the form
> >>>>>>
> >>>>>> module com.foo.app { requires hibernate.core; requires
> >>>>>> hibernate.entitymanager; exports dynamic
> >>>>>> com.foo.app.model; }
> >>>>>>
> >>>>>> makes the types in the `com.foo.app.model` package
> >>>>>> accessible at run
> >>>> time
> >>>>>> but not at compile time.  In combination with the
> >>>>>> earlier change to enable #ReflectionWithoutReadability
> >>>>>> [2] this means that frameworks
> >>>> that
> >>>>>> today use core reflection to manipulate user classes at
> >>>>>> run time are
> >>>> more
> >>>>>> likely to work out-of-the-box, without change, as
> >>>>>> automatic modules.
> >>>> If
> >>>>>> the `com.foo.app.model` package in this example includes
> >>>>>> entity classes to be managed by Hibernate then the
> >>>>>> framework will be able to access
> >>>> them
> >>>>>> without further ado, but under normal circumstances an
> >>>>>> attempt to
> >>>> compile
> >>>>>> code that refers directly to those classes will fail.
> >>>>>>
> >>>>>> The `dynamic` modifier can be applied to both unqualified
> >>>>>> and qualified `exports` directives, though the caveats on
> >>>>>> using qualified exports [3] still apply.
> >>>>>>
> >>>>>> Notes -----
> >>>>>>
> >>>>>> - To access a non-public member in a
> >>>>>> dynamically-exported package you must still invoke the
> >>>>>> appropriate `setAccessible` method, just as you do today
> >>>>>> to access non-public members in an exported package.
> >>>>>>
> >>>>>> - This proposal is similar to Rémi's suggestion to add a
> >>>>>> way to
> >>>> export
> >>>>>> a package only for reflection, still requiring that
> >>>>>> `setAccessible` be invoked [4].  This proposal differs
> >>>>>> in that it does not require `setAccessible` to be invoked
> >>>>>> to access public members in a dynamically-exported
> >>>>>> package.
> >>>>>>
> >>>>>> - This proposal primarily addresses "friendly" uses of
> >>>>>> reflection,
> >>>> such
> >>>>>> as dependency injection and persistence, in which the
> >>>>>> author of a module knows in advance that one or more
> >>>>>> packages must be exported
> >>>> at
> >>>>>> run time for reflective access by frameworks.  This
> >>>>>> proposal is
> >>>> also
> >>>>>> convenient for frameworks that generate code at run
> >>>>>> time, since the constant pool of a generated class file
> >>>>>> can include static
> >>>> references
> >>>>>> to types in dynamically exported packages.
> >>>>>>
> >>>>>> - This proposal opens the door to a category of
> >>>>>> second-class APIs.
> >>>> If
> >>>>>> a package is exported dynamically then you can still
> >>>>>> compile code that refers to types in the package, by
> >>>>>> using `-XaddExports` or its equivalent at compile time,
> >>>>>> and it will work as expected at run
> >>>> time.
> >>>>>> It thus may be useful to use `exports dynamic` for
> >>>>>> packages that contain legacy APIs whose use is strongly
> >>>>>> discouraged, e.g., those
> >>>> in
> >>>>>> the `jdk.unsupported` module of the reference
> >>>>>> implementation,
> >>>> thereby
> >>>>>> forcing anyone who wants to compile against them to go
> >>>>>> to the
> >>>> trouble
> >>>>>> of using `-XaddExports`.
> >>>>>>
> >>>>>> - Intrusive access to arbitrary packages of arbitrary
> >>>>>> modules by,
> >>>> e.g.,
> >>>>>> debugging tools, will still require the use of sharp
> >>>>>> knives such as the `-XaddExports` command-line option or
> >>>>>> its equivalent, or JVM
> >>>> TI.
> >>>>>>
> >>>>>> - Using the `-XaddExports` option or its equivalent
> >>>>>> remains awkward, and sometimes it's the only way out.  To
> >>>>>> ease migration I think
> >>>> it's
> >>>>>> worth considering some way for an application packaged as
> >>>>>> a JAR
> >>>> file
> >>>>>> to include such options in its `MANIFEST.MF` file, as
> >>>>>> suggested by Simon Nash [5].  I'll create a separate
> >>>>>> issue for that.
> >>>>>>
> >>>>>> - New kinds of resolution failures are possible.  If
> >>>>>> module `A` requires `B`, and `B` requires `C`, then if
> >>>>>> `B` and `C` both
> >>>> declare
> >>>>>> some package `P` to be exported dynamically, then a
> >>>>>> split-package error will be reported.  This may
> >>>>>> surprise, since the packages are just internal
> >>>>>> implementation details that were exported dynamically so
> >>>>>> that some framework could access their types.  This is
> >>>>>> not, however, all that different from the kinds of
> >>>>>> resolution and layer-creation failures that are already
> >>>>>> possible due to package collisions.  It's unlikely to
> >>>>>> happen all that often in practice, and the fix is
> >>>>>> straightforward: Just use reverse-DNS or otherwise
> >>>>>> sufficiently-unique package names, as most people already
> >>>>>> do. It is worth exploring whether javac can detect at
> >>>>>> least some of these kinds of collisions and issue
> >>>>>> appropriate warnings.
> >>>>>
> >>>>> I would like to agree on that proposal only on the ground
> >>>>> that it
> >>>> requires
> >>>>> to change the classfile format thus allows in the same time
> >>>>> to fix the issues of the current classfile spec (encoding
> >>>>> of the name module as a Class instead of UTF8, wrong flag
> >>>>> for requires public), but i'm not sure this proposal
> >>>>> support it's own weight.
> >>>>>
> >>>>> Without dynamic, a 'classical' export works the same way
> >>>>> at runtime so adding 'dynamic' to an export only restrict
> >>>>> the usage at compile time. Given that we can already
> >>>>> selectively export packages to some chosen modules which
> >>>>> hide them for the rest of the other modules at compile
> >>>> time,
> >>>>> adding dynamic to the spec adds little value.
> >>>>>
> >>>>> And it has several drawbacks, the main one is that it
> >>>>> weaken the compiletime/runtime fidelity story, with a
> >>>>> dynamic export the runtime
> >>>> world
> >>>>> is not the compiletime world anymore. The main motivation
> >>>>> to add dynamic seems to be for backward compatibility
> >>>>> during the transition between the classpath world and the
> >>>>> full module
> >>>> world,
> >>>>> for several other cases, we use the command line for that,
> >>>>> i don't see
> >>>> why
> >>>>> this case is so special that it should not work like the
> >>>>> other compatibility issues.
> >>>>>
> >>>>> Moreover, an agent, an annotation processor and a jlink
> >>>>> plugin are all able to implement the same trick, add export
> >>>>> after the compilation and before the runtime, so i don't
> >>>>> think it's a good idea to change the spec for that.
> >>>>>
> >>>>> Rémi
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>> [1]
> >>>>>>
> >>>>>
> >>>> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccess
> >>
> >>>>
> ToNonExportedTypes
> >>>>>>
> >>>>
> >> [2]
> >>>>>>
> >>>>>
> >>>> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithou
> >>
> >>>>
> tReadability
> >>>>>>
> >>>>
> >> [3]
> >>>>> http://openjdk.java.net/projects/jigsaw/spec/sotms/#qualified-export
> >>
> >>>>>
> s
> >>>>>>
> >>>>>
> >> [4]
> >>>>>>
> >>>>>
> >>>> http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-Decembe
> >>
> >>>>
> r/000205.html
> >>>>>>
> >>>>
> >> [5]
> >>>>>>
> >>>>>
> >>>> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/00574
> >>
> >>>>
> 5.html
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>>>
> >> --
> >>
> >> PGP Key:
> >> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
> >>
> >>
> >>
> Web:
> >> http://codefx.org a blog about software development
> >> http://do-foss.de Free and Open Source Software for the City of
> >> Dortmund
> >>
> >> Twitter: https://twitter.com/nipafx
> >>
> >
>
> --
>
> PGP Key:
>     http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
>
> Web:
>     http://codefx.org
>         a blog about software development
>     http://do-foss.de
>         Free and Open Source Software for the City of Dortmund
>
> Twitter:
>     https://twitter.com/nipafx
>
>