From david.lloyd at redhat.com Tue Jun 21 22:11:36 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Tue, 21 Jun 2016 17:11:36 -0500 Subject: JDK 9 is not (yet) Feature Complete -- how will we get there? In-Reply-To: <20160621145607.146493062eggemoggin.niobe.net> References: <20160610072437.975638333eggemoggin.niobe.net> <20160621145607.146493062eggemoggin.niobe.net> Message-ID: <5769BB98.1000902@redhat.com> On 06/21/2016 04:56 PM, mark.reinhold at oracle.com wrote: > 2016/6/10 7:24:37 -0700, mark.reinhold at oracle.com: >> ... >> >> To manage the remaining JEPs and small enhancements so that we can reach >> the Feature Complete state in a timely fashion I hereby propose the >> following process: >> >> ... >> >> JDK 9 Committers are invited to comment on this process proposal. If no >> serious objections are raised in one week's time, by 15:00 UTC on 17 June >> 2016, then this is the process that we'll use. > > Hearing no objections, this process is now adopted. > > I've summarized it here: > > http://openjdk.java.net/projects/jdk9/fc-extension-process > > Link to JBS query for pending requests: > > http://j.mp/jdk9-fc-pending I have one question about this - how are the remaining JPMS issues fitting in to this process, or does this fall into some different category? -- - DML From mark.reinhold at oracle.com Tue Jun 21 22:37:07 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Tue, 21 Jun 2016 15:37:07 -0700 Subject: JDK 9 is not (yet) Feature Complete -- how will we get there? In-Reply-To: <5769BB98.1000902@redhat.com> References: <20160610072437.975638333eggemoggin.niobe.net> <20160621145607.146493062eggemoggin.niobe.net> <5769BB98.1000902@redhat.com> Message-ID: <20160621153707.716452057eggemoggin.niobe.net> 2016/6/21 15:11:36 -0700, david.lloyd at redhat.com: > On 06/21/2016 04:56 PM, mark.reinhold at oracle.com wrote: >> 2016/6/10 7:24:37 -0700, mark.reinhold at oracle.com: >>> ... >>> >>> To manage the remaining JEPs and small enhancements so that we can reach >>> the Feature Complete state in a timely fashion I hereby propose the >>> following process: >>> >>> ... >>> >>> JDK 9 Committers are invited to comment on this process proposal. If no >>> serious objections are raised in one week's time, by 15:00 UTC on 17 June >>> 2016, then this is the process that we'll use. >> >> Hearing no objections, this process is now adopted. >> >> I've summarized it here: >> >> http://openjdk.java.net/projects/jdk9/fc-extension-process >> >> Link to JBS query for pending requests: >> >> http://j.mp/jdk9-fc-pending > > I have one question about this - how are the remaining JPMS issues > fitting in to this process, or does this fall into some different category? Insofar as the resolution of those issues requires changes in JDK 9 then those changes will be subject to the above process, but I don't expect that to present any difficulties. All of the relevant JEPs should shortly be granted FC extensions. As to the issues themselves, I'm drafting a few proposals which I hope to post here shortly. - Mark From david.lloyd at redhat.com Wed Jun 22 13:03:31 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Wed, 22 Jun 2016 08:03:31 -0500 Subject: Minor confusion on the issues doc Message-ID: <576A8CA3.6060701@redhat.com> The #ReflectionWithoutReadability issue resolution is not indented to the same level as the bullet point for that issue, leading multiple people to ask me how that resolution applies to the whole section. Is there any way to either fix the indent, or barring that, indicate that the resolution applies to just that issue? http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithoutReadability -- - DML From volker.berlin at goebel-clan.de Wed Jun 22 20:22:36 2016 From: volker.berlin at goebel-clan.de (Volker Berlin) Date: Wed, 22 Jun 2016 22:22:36 +0200 Subject: Minor confusion on the issues doc In-Reply-To: <576A8CA3.6060701@redhat.com> References: <576A8CA3.6060701@redhat.com> Message-ID: <86c492df-46ef-f74c-5059-7ec40648c02c@goebel-clan.de> A good question. Will the command line switch -XaddExports be the only option to enable the reflection? Am 22.06.2016 um 15:03 schrieb David M. Lloyd: > The #ReflectionWithoutReadability issue resolution is not indented to > the same level as the bullet point for that issue, leading multiple > people to ask me how that resolution applies to the whole section. Is > there any way to either fix the indent, or barring that, indicate that > the resolution applies to just that issue? > > http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithoutReadability > From pbenedict at apache.org Wed Jun 22 20:39:54 2016 From: pbenedict at apache.org (Paul Benedict) Date: Wed, 22 Jun 2016 15:39:54 -0500 Subject: Minor confusion on the issues doc In-Reply-To: <86c492df-46ef-f74c-5059-7ec40648c02c@goebel-clan.de> References: <576A8CA3.6060701@redhat.com> <86c492df-46ef-f74c-5059-7ec40648c02c@goebel-clan.de> Message-ID: Since reflection is all about "magic" that transgressing programming boundaries (such as reading private fields), why isn't letting core reflection be "jailbreaking" a compelling argument? Minus the enforcement of the security policy, I think core reflection should allow programs to roam free like they are today. Introducing module enforcement is way too strict for the magic that people want with reflection. Cheers, Paul On Wed, Jun 22, 2016 at 3:22 PM, Volker Berlin wrote: > A good question. Will the command line switch -XaddExports be the only > option to enable the reflection? > > > > Am 22.06.2016 um 15:03 schrieb David M. Lloyd: > >> The #ReflectionWithoutReadability issue resolution is not indented to the >> same level as the bullet point for that issue, leading multiple people to >> ask me how that resolution applies to the whole section. Is there any way >> to either fix the indent, or barring that, indicate that the resolution >> applies to just that issue? >> >> >> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithoutReadability >> > > From mark.reinhold at oracle.com Thu Jun 23 16:53:52 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 23 Jun 2016 09:53:52 -0700 Subject: Minor confusion on the issues doc In-Reply-To: <576A8CA3.6060701@redhat.com> References: <576A8CA3.6060701@redhat.com> Message-ID: <20160623095352.405957043eggemoggin.niobe.net> 2016/6/22 6:03:31 -0700, david.lloyd at redhat.com: > The #ReflectionWithoutReadability issue resolution is not indented to > the same level as the bullet point for that issue, leading multiple > people to ask me how that resolution applies to the whole section. Is > there any way to either fix the indent, or barring that, indicate that > the resolution applies to just that issue? The issue list is formatted in the same way as the requirements list. Each item is summarized in indented text, with applicable commentary following the item in non-indented text. I can see how that's confusing though, so I'll think about how to fix it in the next revision. - Mark From mark.reinhold at oracle.com Tue Jun 28 21:16:15 2016 From: mark.reinhold at oracle.com (Mark Reinhold) Date: Tue, 28 Jun 2016 14:16:15 -0700 (PDT) Subject: JSR 376 schedule Message-ID: <20160628211615.81862B59BA@eggemoggin.niobe.net> At this point we have quite a number of issues to work through [1]. Rather than spend time generating an Early Draft Review specification I suggest that we instead focus on the issues themselves, aiming to finish up and publish an EDR around the end of the summer. Thus our new schedule would be: Early Draft Review September 2016 Public Review November 2016 Proposed Final Draft January 2017 Final Release March 2017 I realize that this is a bit compressed, but given that the draft specification and prototype RI have been publicly available and regularly updated since last September, and that many developers have sent feedback on them via several different channels, it's not clear to me that doing an EDR any earlier would have had, or would now have, any practical benefit in terms of transparency. Please let me know by this time next week if you have any concerns with this new schedule. In the meantime I'll post specific proposals for some of the open issues in messages that follow, and then update the issue list with links thereto. If you comment on one of these proposals then please either reply on the existing thread or else include the hashtag of the relevant issue(s) in the subject line of a new thread, to simplify tracking. - Mark [1] http://openjdk.java.net/projects/jigsaw/spec/issues/ From mark.reinhold at oracle.com Tue Jun 28 21:17:15 2016 From: mark.reinhold at oracle.com (Mark Reinhold) Date: Tue, 28 Jun 2016 14:17:15 -0700 (PDT) Subject: Proposal: #CompileTimeDependences: `requires static` Message-ID: <20160628211715.84610B59BF@eggemoggin.niobe.net> Issue summary ------------- #CompileTimeDependences --- Provide a means to specify a module dependence that is mandatory at compile time but optional at run time, for use with libraries that are not strictly necessary but can be leveraged if present at run time. (Detail: If a dependence is "optional at run time" then does the module system try to satisfy it during resolution but fail silently if it cannot, or does it ignore it during resolution but add the corresponding read edge to that module if it was resolved for some other reason?) [1] Use cases for this feature include annotations that need not be present at run time [2] and "auto-configuring" libraries and frameworks such as Joda Beans [3] and Spring [4]. Proposal -------- Extend the language of module declarations to allow the `static` modifier to be used on a `requires` directive, with the following meanings: - At compile time, `requires static M` expresses a mandatory dependence. It is an error if a suitable module cannot be found amongst the observable modules and resolved. - In phases after compile time, `requires static M` expresses an optional dependence. The module system will not search the observable modules for a suitable module during resolution, but if the resulting module graph contains a suitable module then it will add the appropriate readability edge prior to doing the usual post-resolution sanity checks. (The parenthetical question in the issue summary is, in other words, answered in the negative.) Thus a hypothetical module declaration of the form module joda.beans { requires static joda.collect; ... } would ensure that the `joda.collect` module is available at compile time, so that code in the `joda.beans` module that refers to `joda.collect` can be compiled without any fuss. It would not, however, guarantee that `joda.collect` is available at link time or run time. A user who wants to use `joda.beans` together with `joda.collect` in those later phases must ensure that `joda.collect` is added to the module graph by, e.g., declaring a dependence upon it in some other module, or by using the `-addmods` option or its equivalent. The code in `joda.beans` that refers to types in `joda.collect` must, of course, be written defensively so that it fails gracefully at run time when the `joda.collect` module is not present. It is possible to combine the `public` and `static` modifiers on a `requires` directive: module joda.beans { requires static joda.collect; requires public static freemarker; ... } This is useful in this particular case since `joda.beans` defines two exported public classes that extend classes defined in the `freemarker` module, and so those classes are logically part of the `joda.beans` API. In general, however, this idiom leads to fragile APIs and is therefore not advised except when dealing with legacy code. Notes ----- - In phases after compile time, why doesn't the module system search the observable modules in an attempt to satisfy an optional dependence? This would not be difficult to arrange but it could lead to surprising behavior in which optional dependences silently cause many additional modules to be resolved. Given the use cases we have it does not seem unreasonable to expect an end user to indicate explicitly, either in source code or out-of-band (e.g., with a command-line option such as `-addmods`), that an otherwise-optional module is actually needed. [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#CompileTimeDependences [2] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-October/004915.html [3] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005462.html [4] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005814.html From mark.reinhold at oracle.com Tue Jun 28 21:18:15 2016 From: mark.reinhold at oracle.com (Mark Reinhold) Date: Tue, 28 Jun 2016 14:18:15 -0700 (PDT) Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` Message-ID: <20160628211815.87E88B59C4@eggemoggin.niobe.net> Issue summary ------------- #ReflectiveAccessToNonExportedTypes --- Some kinds of framework libraries require reflective access to members of the non-exported types of other modules; examples include dependency injection (Guice), persistence (JPA), debugging tools, code-automation tools, and serialization (XStream). In some cases the particular library to be used is not known until run time (e.g., Hibernate and EclipseLink both implement JPA). This capability is also sometimes used to work around bugs in unchangeable code. Access to non-exported packages can, at present, only be done via command-line flags, which is extremely awkward. Provide an easier way for reflective code to access such non-exported types. [1] Proposal -------- Extend the language of module declarations so that a package can be declared to be exported at run time but not at compile time. This is, roughly, the dual of the `requires static` construct proposed for #CompileTimeDependences, hence we propose to introduce a new modifier, `dynamic`, for use on the `exports` directive. It has the following meanings: - At compile time, `exports dynamic P` does not cause the package `P` to be exported, though it does require `P` to be a package defined in the module. - In phases after compile time, `exports dynamic P` behaves in exactly the same way as `exports P`. It therefore takes part fully in resolution and configuration, and is subject to the same consistency constraints as normally-exported packages (e.g., no split packages). Thus a module declaration of the form module com.foo.app { requires hibernate.core; requires hibernate.entitymanager; exports dynamic com.foo.app.model; } makes the types in the `com.foo.app.model` package accessible at run time but not at compile time. In combination with the earlier change to enable #ReflectionWithoutReadability [2] this means that frameworks that today use core reflection to manipulate user classes at run time are more likely to work out-of-the-box, without change, as automatic modules. If the `com.foo.app.model` package in this example includes entity classes to be managed by Hibernate then the framework will be able to access them without further ado, but under normal circumstances an attempt to compile code that refers directly to those classes will fail. The `dynamic` modifier can be applied to both unqualified and qualified `exports` directives, though the caveats on using qualified exports [3] still apply. Notes ----- - To access a non-public member in a dynamically-exported package you must still invoke the appropriate `setAccessible` method, just as you do today to access non-public members in an exported package. - This proposal is similar to R??mi's suggestion to add a way to export a package only for reflection, still requiring that `setAccessible` be invoked [4]. This proposal differs in that it does not require `setAccessible` to be invoked to access public members in a dynamically-exported package. - This proposal primarily addresses "friendly" uses of reflection, such as dependency injection and persistence, in which the author of a module knows in advance that one or more packages must be exported at run time for reflective access by frameworks. This proposal is also convenient for frameworks that generate code at run time, since the constant pool of a generated class file can include static references to types in dynamically exported packages. - This proposal opens the door to a category of second-class APIs. If a package is exported dynamically then you can still compile code that refers to types in the package, by using `-XaddExports` or its equivalent at compile time, and it will work as expected at run time. It thus may be useful to use `exports dynamic` for packages that contain legacy APIs whose use is strongly discouraged, e.g., those in the `jdk.unsupported` module of the reference implementation, thereby forcing anyone who wants to compile against them to go to the trouble of using `-XaddExports`. - Intrusive access to arbitrary packages of arbitrary modules by, e.g., debugging tools, will still require the use of sharp knives such as the `-XaddExports` command-line option or its equivalent, or JVM TI. - Using the `-XaddExports` option or its equivalent remains awkward, and sometimes it's the only way out. To ease migration I think it's worth considering some way for an application packaged as a JAR file to include such options in its `MANIFEST.MF` file, as suggested by Simon Nash [5]. I'll create a separate issue for that. - New kinds of resolution failures are possible. If module `A` requires `B`, and `B` requires `C`, then if `B` and `C` both declare some package `P` to be exported dynamically, then a split-package error will be reported. This may surprise, since the packages are just internal implementation details that were exported dynamically so that some framework could access their types. This is not, however, all that different from the kinds of resolution and layer-creation failures that are already possible due to package collisions. It's unlikely to happen all that often in practice, and the fix is straightforward: Just use reverse-DNS or otherwise sufficiently-unique package names, as most people already do. It is worth exploring whether javac can detect at least some of these kinds of collisions and issue appropriate warnings. [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccessToNonExportedTypes [2] http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithoutReadability [3] http://openjdk.java.net/projects/jigsaw/spec/sotms/#qualified-exports [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-December/000205.html [5] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005745.html From mark.reinhold at oracle.com Tue Jun 28 21:19:15 2016 From: mark.reinhold at oracle.com (Mark Reinhold) Date: Tue, 28 Jun 2016 14:19:15 -0700 (PDT) Subject: Proposal: #ModuleAnnotations and #ModuleDeprecation Message-ID: <20160628211915.9706BB59C8@eggemoggin.niobe.net> Issue summaries --------------- #ModuleAnnotations --- Should it be possible to annotate module declarations? As previously discussed this is technically feasible but it would add significant complexity. [1] #ModuleDeprecation -- If module declarations cannot be annotated, should there be some other way to deprecate modules? [2] Proposal -------- Allow arbitrary annotations to be placed on module declarations, and revise the `java.lang.Deprecated` annotation so that it can be applied to modules. - Add a new value, `MODULE`, to `java.lang.annotation.ElementType`, to denote the top level of a module declaration. - Extend `javax.lang.model` and `java.lang.reflect` to provide access to module annotations. Repeating annotations are supported, subject to the usual rules for repeatable annotation types. (This proposal will not add similar support to `java.lang.module.ModuleDescriptor`; that would add yet more complexity, and we haven't seen a use case for it.) - Annotations on modules will never affect the operation of the module system itself. - Revise `java.lang.Deprecated` and `java.lang.SuppressWarnings` to allow `ElementType.MODULE` as a target. For now, no other existing Java SE annotations will apply to modules. Also, allow `import` declarations [3] to be used in module compilation units. - Any `import` declarations must appear before the `module` declaration. - As a consequence of supporting `import` declarations, simple type names can now be used in the `uses` and `provides` directives of a module declaration. Type annotations on these type names are not supported. Rationale --------- When we first discussed this topic I wrote [4]: > We haven't seen a compelling use case (so far) for annotations on module > declarations, or on clauses in module declarations, and they aren't a > documented requirement, so for simplicity we left them out of the initial > design and the prototype. The compelling use case is, of course, deprecation, as R??mi pointed out [5]. In Java SE 9, in particular, we very much want to be able to write something like this: @Deprecated(since = "9", forRemoval = true) module java.corba { ... } The most natural way to do this is to allow module declarations to be annotated in the same way as other top-level declarations (i.e., packages and classes). If we don't allow the `@Deprecated` annotation to be used on modules then we'd have to invent some other way to deprecate modules (another new keyword?), and that would just be confusing. Like most language features, annotations have sometimes been over-used and even abused [6]. Allowing modules to be annotated opens up a whole new avenue for unfortunate design decisions that lead to unreadable code. We could try to limit the damage by allowing just a few annotations on module declarations (e.g., only those in the `java.lang` package), or by not supporting `import` declarations so that the use of non-platform annotations is discouragingly verbose. Such an approach would lead to unpleasant asymmetries in the language, however, so instead we propose to support arbitrary annotations on module declarations in the obvious way. I think the guidance I wrote earlier [4] still holds. Annotations are useful for extra-linguistic statements, such as deprecation, that make sense in source code and can usefully be carried through the tool chain into the resulting class file. I still don't think they're appropriate for information that is more appropriately the domain of build systems, e.g., version strings. Notes ----- - Should it be possible to annotate the individual directives in a module declaration? This would allow you to deprecate individual exports and requires-public directives, which could be useful in the process of evolving an API: If you plan to remove a package, or remove a public dependence upon some other module, then you could deprecate the corresponding `exports` or `requires public` directive (presumably with `forRemoval = true`) to warn your users that the API is going to change. There are no cases in the JDK itself where we need this ability, at least not desperately. Designing and implementing this would be a non-trivial exercise, and it is a separable feature that could be added later on if justified by experience. [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleAnnotations [2] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleDeprecation [3] http://docs.oracle.com/javase/specs/jls/se8/html/jls-7.html#jls-7.5 [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-October/000153.html [5] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-March/000263.html [6] http://annotatiomania.com/ From mark.reinhold at oracle.com Tue Jun 28 21:20:15 2016 From: mark.reinhold at oracle.com (Mark Reinhold) Date: Tue, 28 Jun 2016 14:20:15 -0700 (PDT) Subject: Proposal: #ResourceEncapsulation and #ClassFilesAsResources Message-ID: <20160628212015.99A06B59CC@eggemoggin.niobe.net> Issue summaries --------------- #ClassFilesAsResources --- If a type is visible and was loaded from a class file then it should be possible to read that file by invoking the `getResourceAsStream` method of the type's class loader, as it is in earlier releases. [1] #ResourceEncapsulation --- The `Module::getResourceAsStream` method can be used to read the resources of any named module, without restriction, which violates the resource-encapsulation requirement [2]. This method should be restricted somehow so that only "suitably-privileged" code (for some definition of that term) can access resources in a named module other than its own. An alternative is to drop this requirement. [3] Proposal -------- Drop the agreed resource-encapsulation requirement, which reads: Resource encapsulation --- The run-time system must ensure that the static resource files within a module are directly accessible only by code within that module. The existing resource-access APIs should continue to work as they do today when used to access a module's own resource files. [2] Make the following changes to the various `getResource*` methods: - The `ClassLoader::getResource*` methods will first delegate to the loader's parent, if any, and will then search for resources in the named modules defined to the loader, and will then search in the loader's unnamed module, typically defined by a class path. In the case of the `getResources` method the order of the resources in the returned enumeration will be unspecified, except that the values of `cl.getResources(name).nextElement()` and `cl.getResource(name)` will be equal. - The `Class::getResource*` methods will only search for resources in the module that defines the class. In the case of a class in an unnamed module this will typically result in searching the class path of the corresponding class loader. The `java.lang.reflect.Module::getResourceAsStream` method will remain, so that it's possible to look up a resource in a module without having a reference to a class defined in that module. It will always be the case that, for a given a class `c`, `c.getResourceAsStream(name)` will be equivalent to `c.getModule().getResourceAsStream(name)`. Rationale --------- The encapsulation of resources was intended to be part of the overall encapsulation story, on the view that a module's resources are just internal implementation details that need not, and should not, be available to code in other modules. Allowing external code to depend upon a module's internal resources is just as problematic as allowing such code to depend upon internal APIs. The reality of existing code today, however, is rather different, with two common use cases: - Resources are often used intentionally to convey information to external code, e.g., as a way to publish configuration files such as `persistence.xml`. This sort of thing should, ideally, be done via services [4][5], but that is not the current practice and it will take years to migrate to it. - Various popular tools and byte-code manipulation frameworks expect to be able to load class files as resources, i.e., to access the class file that defines a class `p.q.C` by invoking `getResource("p.q.C")` on that class's loader. With the API as it stands today these use cases can still be supported, but the code that looks up the resource must be converted to use the `Module::getResourceAsStream` method. This is a barrier to migration, since it means that fewer existing frameworks will work out-of-the-box as automatic modules. We should not drop agreed requirements lightly, but now that we have some practical experience with an implementation of this one I think it's clear that the pain outweighs the gain. [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ClassFilesAsResources [2] http://openjdk.java.net/projects/jigsaw/spec/reqs/#resource-encapsulation [3] http://openjdk.java.net/projects/jigsaw/spec/issues/#ResourceEncapsulation [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-October/000163.html [5] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-December/000207.html [6] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-September/004486.html [7] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-November/005240.html From mark.reinhold at oracle.com Tue Jun 28 21:21:15 2016 From: mark.reinhold at oracle.com (Mark Reinhold) Date: Tue, 28 Jun 2016 14:21:15 -0700 (PDT) Subject: Proposal: #ReflectiveAccessByInstrumentationAgents Message-ID: <20160628212115.9BDC4B59D0@eggemoggin.niobe.net> Issue summary ------------- #ReflectiveAccessByInstrumentationAgents --- As noted in the related issue #ReflectiveAccessToNonExportedTypes [1], the core reflection API cannot be used to gain access to members of packages that are not exported by their defining modules. Even if that issue is addressed for the use cases cited, a distinct use case is that of instrumentation agents that need to enable injected code to invoke non-public methods and access non-public fields. A possible solution for this use case is specifically to provide instrumentation agents with the ability to use core reflection, or some equivalent, even on elements of non-exported packages. [2] Proposal -------- Extend the `java.lang.instrument.Instrumentation` interface, and the corresponding JVM TI native API, to support a `redefineModule` method. This method is roughly analogous to the existing `redefineClasses` method. It will allow instrumentation agents to update an existing module, represented by a `java.lang.reflect.Module` object, to read additional modules, export additional packages, use additional services, and provide additional services. Remove the `addModuleReads` method from the `Instrumentation` interface, and remove the `AddModuleReads` and `CanReadModule` functions from JNI, since they are no longer needed. Notes ----- - This new API will not allow read, export, uses, and provides relationships to be dropped at run time. The use cases do not require this, and implementing it would be very difficult. - A sufficiently-skilled instrumentation-agent developer can already do some of this with the existing API, by injecting new classes into target modules with initializers that invoke the necessary `Module` methods. This is, however, awkward at best. - Objects that implement the `Instrumentation` interface are already very powerful, since they can be used to redefine arbitrary classes. Just as is the case today, in order to preserve platform integrity instrumentation agents must be coded very carefully so that they do not leak such objects to user code. [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccessToNonExportedTypes [2] http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccessByInstrumentationAgents From mark.reinhold at oracle.com Tue Jun 28 21:22:15 2016 From: mark.reinhold at oracle.com (Mark Reinhold) Date: Tue, 28 Jun 2016 14:22:15 -0700 (PDT) Subject: Proposal: #BootstrapClassLoaderSearchInJVMTI Message-ID: <20160628212215.9E7D9B59D4@eggemoggin.niobe.net> Issue summary ------------- #BootstrapClassLoaderSearchInJVMTI --- The JVM TI API [1] defines a function, `AddToBootstrapClassLoaderSearch` [2], by which an instrumentation agent can append a file-system path element to the bootstrap class loader's class path. The bootstrap class path, as such, no longer exists, but this function remains supported for now. Should this function be deprecated, with the intent to remove it in a future release, or should it remain supported? It appears to be critical to the operation of at least one popular instrumentation agent. [3] Proposal -------- Retain this method. Do not deprecate it. [1] http://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html [2] http://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html#AddToBootstrapClassLoaderSearch [3] http://openjdk.java.net/projects/jigsaw/spec/issues/#BootstrapClassLoaderSearchInJVMTI From mark.reinhold at oracle.com Tue Jun 28 21:23:15 2016 From: mark.reinhold at oracle.com (Mark Reinhold) Date: Tue, 28 Jun 2016 14:23:15 -0700 (PDT) Subject: Proposal: #CustomizableAutomaticModuleNameMapping Message-ID: <20160628212315.A247FB59D8@eggemoggin.niobe.net> Issue summary ------------- #CustomizableAutomaticModuleNameMapping --- Provide a means to customize the algorithm that computes the names and version strings of automatic modules. [1] Proposal -------- Extend the `java.lang.module.ModuleDescriptor.Builder` API so that it can be used to create automatic as well as explicit modules. This will allow a container application to use a custom `ModuleFinder` to manage the mapping from artifact names to module names, version strings, and so forth. [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#CustomizableAutomaticModuleNameMapping From forax at univ-mlv.fr Tue Jun 28 22:50:52 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 29 Jun 2016 00:50:52 +0200 (CEST) Subject: Proposal: #CompileTimeDependences: `requires static` In-Reply-To: <20160628211715.84610B59BF@eggemoggin.niobe.net> References: <20160628211715.84610B59BF@eggemoggin.niobe.net> Message-ID: <1808187852.884711.1467154252074.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Mark Reinhold" > ?: jpms-spec-experts at openjdk.java.net > Envoy?: Mardi 28 Juin 2016 23:17:15 > Objet: Proposal: #CompileTimeDependences: `requires static` > > Issue summary > ------------- > > #CompileTimeDependences --- Provide a means to specify a module > dependence that is mandatory at compile time but optional at run time, > for use with libraries that are not strictly necessary but can be > leveraged if present at run time. (Detail: If a dependence is > "optional at run time" then does the module system try to satisfy it > during resolution but fail silently if it cannot, or does it ignore it > during resolution but add the corresponding read edge to that module if > it was resolved for some other reason?) [1] > > Use cases for this feature include annotations that need not be present > at run time [2] and "auto-configuring" libraries and frameworks such as > Joda Beans [3] and Spring [4]. > > Proposal > -------- > > Extend the language of module declarations to allow the `static` modifier > to be used on a `requires` directive, with the following meanings: > > - At compile time, `requires static M` expresses a mandatory > dependence. It is an error if a suitable module cannot be found > amongst the observable modules and resolved. > > - In phases after compile time, `requires static M` expresses an > optional dependence. The module system will not search the > observable modules for a suitable module during resolution, but > if the resulting module graph contains a suitable module then it > will add the appropriate readability edge prior to doing the usual > post-resolution sanity checks. (The parenthetical question in the > issue summary is, in other words, answered in the negative.) > > Thus a hypothetical module declaration of the form > > module joda.beans { > requires static joda.collect; > ... > } > > would ensure that the `joda.collect` module is available at compile time, > so that code in the `joda.beans` module that refers to `joda.collect` can > be compiled without any fuss. It would not, however, guarantee that > `joda.collect` is available at link time or run time. A user who wants > to use `joda.beans` together with `joda.collect` in those later phases > must ensure that `joda.collect` is added to the module graph by, e.g., > declaring a dependence upon it in some other module, or by using the > `-addmods` option or its equivalent. > > The code in `joda.beans` that refers to types in `joda.collect` must, of > course, be written defensively so that it fails gracefully at run time > when the `joda.collect` module is not present. > > It is possible to combine the `public` and `static` modifiers on a > `requires` directive: > > module joda.beans { > requires static joda.collect; > requires public static freemarker; > ... > } > > This is useful in this particular case since `joda.beans` defines two > exported public classes that extend classes defined in the `freemarker` > module, and so those classes are logically part of the `joda.beans` API. > In general, however, this idiom leads to fragile APIs and is therefore > not advised except when dealing with legacy code. > > Notes > ----- > > - In phases after compile time, why doesn't the module system search > the observable modules in an attempt to satisfy an optional > dependence? > > This would not be difficult to arrange but it could lead to surprising > behavior in which optional dependences silently cause many additional > modules to be resolved. Given the use cases we have it does not seem > unreasonable to expect an end user to indicate explicitly, either in > source code or out-of-band (e.g., with a command-line option such as > `-addmods`), that an otherwise-optional module is actually needed. Apart the fact that 'static' should be spelled 'optional', there is no reason to reuse static as it doesn't convey the semantics we want, i.e. optional at runtime, i fully agree with this proposal. R?mi > > > [1] > http://openjdk.java.net/projects/jigsaw/spec/issues/#CompileTimeDependences > [2] > http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-October/004915.html > [3] > http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005462.html > [4] > http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005814.html > From mark.reinhold at oracle.com Tue Jun 28 23:31:50 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Tue, 28 Jun 2016 16:31:50 -0700 Subject: Proposal: #CompileTimeDependences: `requires static` In-Reply-To: <1808187852.884711.1467154252074.JavaMail.zimbra@u-pem.fr> References: <20160628211715.84610B59BF@eggemoggin.niobe.net> <1808187852.884711.1467154252074.JavaMail.zimbra@u-pem.fr> Message-ID: <20160628163150.929695476eggemoggin.niobe.net> 2016/6/28 15:50:52 -0700, Remi Forax : > Apart the fact that 'static' should be spelled 'optional', there is no > reason to reuse static as it doesn't convey the semantics we want, > i.e. optional at runtime, i fully agree with this proposal. In this context `static` is intended to mean "at compile time" or, equivalently, before the more "dynamic" phases of run time and link time (the latter of which is more like run time than compile time). I agree that `static` is potentially confusing since its meaning here is different from what `static` means when used on a member of a class. It does, however, fit nicely with its (nearly) dual directive, `exports dynamic`, proposed for #ReflectiveAccessToNonExportedTypes. I think `optional` is a non-starter, since `requires optional` reads as an oxymoron, and it's optional at run time but mandatory at compile time, so in what sense is it, really, "optional"? Suggestions of other alternatives are welcome ... - Mark From simon at cjnash.com Wed Jun 29 08:20:25 2016 From: simon at cjnash.com (Simon Nash) Date: Wed, 29 Jun 2016 09:20:25 +0100 Subject: Proposal: #CompileTimeDependences: `requires static` In-Reply-To: <20160628163150.929695476eggemoggin.niobe.net> References: <20160628211715.84610B59BF@eggemoggin.niobe.net> <1808187852.884711.1467154252074.JavaMail.zimbra@u-pem.fr> <20160628163150.929695476eggemoggin.niobe.net> Message-ID: <577384C9.4010209@cjnash.com> mark.reinhold at oracle.com wrote: > 2016/6/28 15:50:52 -0700, Remi Forax : >> Apart the fact that 'static' should be spelled 'optional', there is no >> reason to reuse static as it doesn't convey the semantics we want, >> i.e. optional at runtime, i fully agree with this proposal. > > In this context `static` is intended to mean "at compile time" or, > equivalently, before the more "dynamic" phases of run time and link > time (the latter of which is more like run time than compile time). > > I agree that `static` is potentially confusing since its meaning here > is different from what `static` means when used on a member of a class. > > It does, however, fit nicely with its (nearly) dual directive, `exports > dynamic`, proposed for #ReflectiveAccessToNonExportedTypes. > > I think `optional` is a non-starter, since `requires optional` reads as > an oxymoron, and it's optional at run time but mandatory at compile > time, so in what sense is it, really, "optional"? > > Suggestions of other alternatives are welcome ... > > - Mark > I agree with Remi that 'static' isn't a good choice. One the nice improvements of Java over C was Java's cleaner and more consistent usage of the 'static' keyword. How about 'requires import' or 'requires imported'? This is consistent with the current usage of 'import' to declare compile-time dependencies on other packages. Simon From forax at univ-mlv.fr Wed Jun 29 08:51:16 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 29 Jun 2016 10:51:16 +0200 (CEST) Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` In-Reply-To: <20160628211815.87E88B59C4@eggemoggin.niobe.net> References: <20160628211815.87E88B59C4@eggemoggin.niobe.net> Message-ID: <1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Mark Reinhold" > ?: jpms-spec-experts at openjdk.java.net > Envoy?: Mardi 28 Juin 2016 23:18:15 > Objet: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` > > Issue summary > ------------- > > #ReflectiveAccessToNonExportedTypes --- Some kinds of framework > libraries require reflective access to members of the non-exported > types of other modules; examples include dependency injection (Guice), > persistence (JPA), debugging tools, code-automation tools, and > serialization (XStream). In some cases the particular library to be > used is not known until run time (e.g., Hibernate and EclipseLink both > implement JPA). This capability is also sometimes used to work around > bugs in unchangeable code. Access to non-exported packages can, at > present, only be done via command-line flags, which is extremely > awkward. Provide an easier way for reflective code to access such > non-exported types. [1] > > Proposal > -------- > > Extend the language of module declarations so that a package can be > declared to be exported at run time but not at compile time. This is, > roughly, the dual of the `requires static` construct proposed for > #CompileTimeDependences, hence we propose to introduce a new modifier, > `dynamic`, for use on the `exports` directive. It has the following > meanings: > > - At compile time, `exports dynamic P` does not cause the package `P` > to be exported, though it does require `P` to be a package defined > in the module. > > - In phases after compile time, `exports dynamic P` behaves in exactly > the same way as `exports P`. It therefore takes part fully in > resolution and configuration, and is subject to the same consistency > constraints as normally-exported packages (e.g., no split packages). > > Thus a module declaration of the form > > module com.foo.app { > requires hibernate.core; > requires hibernate.entitymanager; > exports dynamic com.foo.app.model; > } > > makes the types in the `com.foo.app.model` package accessible at run time > but not at compile time. In combination with the earlier change to > enable #ReflectionWithoutReadability [2] this means that frameworks that > today use core reflection to manipulate user classes at run time are more > likely to work out-of-the-box, without change, as automatic modules. If > the `com.foo.app.model` package in this example includes entity classes > to be managed by Hibernate then the framework will be able to access them > without further ado, but under normal circumstances an attempt to compile > code that refers directly to those classes will fail. > > The `dynamic` modifier can be applied to both unqualified and qualified > `exports` directives, though the caveats on using qualified exports [3] > still apply. > > Notes > ----- > > - To access a non-public member in a dynamically-exported package you > must still invoke the appropriate `setAccessible` method, just as > you do today to access non-public members in an exported package. > > - This proposal is similar to R?mi's suggestion to add a way to export > a package only for reflection, still requiring that `setAccessible` > be invoked [4]. This proposal differs in that it does not require > `setAccessible` to be invoked to access public members in a > dynamically-exported package. > > - This proposal primarily addresses "friendly" uses of reflection, such > as dependency injection and persistence, in which the author of a > module knows in advance that one or more packages must be exported at > run time for reflective access by frameworks. This proposal is also > convenient for frameworks that generate code at run time, since the > constant pool of a generated class file can include static references > to types in dynamically exported packages. > > - This proposal opens the door to a category of second-class APIs. If > a package is exported dynamically then you can still compile code > that refers to types in the package, by using `-XaddExports` or its > equivalent at compile time, and it will work as expected at run time. > It thus may be useful to use `exports dynamic` for packages that > contain legacy APIs whose use is strongly discouraged, e.g., those in > the `jdk.unsupported` module of the reference implementation, thereby > forcing anyone who wants to compile against them to go to the trouble > of using `-XaddExports`. > > - Intrusive access to arbitrary packages of arbitrary modules by, e.g., > debugging tools, will still require the use of sharp knives such as > the `-XaddExports` command-line option or its equivalent, or JVM TI. > > - Using the `-XaddExports` option or its equivalent remains awkward, > and sometimes it's the only way out. To ease migration I think it's > worth considering some way for an application packaged as a JAR file > to include such options in its `MANIFEST.MF` file, as suggested by > Simon Nash [5]. I'll create a separate issue for that. > > - New kinds of resolution failures are possible. If module `A` > requires `B`, and `B` requires `C`, then if `B` and `C` both declare > some package `P` to be exported dynamically, then a split-package > error will be reported. This may surprise, since the packages are > just internal implementation details that were exported dynamically > so that some framework could access their types. This is not, > however, all that different from the kinds of resolution and > layer-creation failures that are already possible due to package > collisions. It's unlikely to happen all that often in practice, > and the fix is straightforward: Just use reverse-DNS or otherwise > sufficiently-unique package names, as most people already do. It > is worth exploring whether javac can detect at least some of these > kinds of collisions and issue appropriate warnings. I would like to agree on that proposal only on the ground that it requires to change the classfile format thus allows in the same time to fix the issues of the current classfile spec (encoding of the name module as a Class instead of UTF8, wrong flag for requires public), but i'm not sure this proposal support it's own weight. Without dynamic, a 'classical' export works the same way at runtime so adding 'dynamic' to an export only restrict the usage at compile time. Given that we can already selectively export packages to some chosen modules which hide them for the rest of the other modules at compile time, adding dynamic to the spec adds little value. And it has several drawbacks, the main one is that it weaken the compiletime/runtime fidelity story, with a dynamic export the runtime world is not the compiletime world anymore. The main motivation to add dynamic seems to be for backward compatibility during the transition between the classpath world and the full module world, for several other cases, we use the command line for that, i don't see why this case is so special that it should not work like the other compatibility issues. Moreover, an agent, an annotation processor and a jlink plugin are all able to implement the same trick, add export after the compilation and before the runtime, so i don't think it's a good idea to change the spec for that. R?mi > > > [1] > http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccessToNonExportedTypes > [2] > http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithoutReadability > [3] http://openjdk.java.net/projects/jigsaw/spec/sotms/#qualified-exports > [4] > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-December/000205.html > [5] > http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005745.html > From stef at epardaud.fr Wed Jun 29 09:01:56 2016 From: stef at epardaud.fr (Stephane Epardaud) Date: Wed, 29 Jun 2016 11:01:56 +0200 Subject: Proposal: #ModuleAnnotations and #ModuleDeprecation In-Reply-To: <20160628211915.9706BB59C8@eggemoggin.niobe.net> References: <20160628211915.9706BB59C8@eggemoggin.niobe.net> Message-ID: <57738E84.2050105@epardaud.fr> Hi, This all sounds great, except: On 28/06/16 23:19, Mark Reinhold wrote: > - Should it be possible to annotate the individual directives in a > module declaration? This would allow you to deprecate individual > exports and requires-public directives, which could be useful in the > process of evolving an API: If you plan to remove a package, or > remove a public dependence upon some other module, then you could > deprecate the corresponding `exports` or `requires public` directive > (presumably with `forRemoval = true`) to warn your users that the > API is going to change. > > There are no cases in the JDK itself where we need this ability, at least > not desperately. Designing and implementing this would be a non-trivial > exercise, and it is a separable feature that could be added later on if > justified by experience. In Ceylon we allow annotations on module imports, and we were planning on recording imported module versions in annotations for Jigsaw. I fear that disallowing annotations on module imports will lead to people sticking these annotations on the module, like this for example: @Versions({ @Version(name = "bar", version = "2.0"), @Version(name = "gee", version = "1.2") }) module foo { import bar; import gee; } So, there's always a way to get around this limitation, but it's quite horrible and ad-hoc. From forax at univ-mlv.fr Wed Jun 29 09:08:38 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 29 Jun 2016 11:08:38 +0200 (CEST) Subject: Proposal: #ModuleAnnotations and #ModuleDeprecation In-Reply-To: <20160628211915.9706BB59C8@eggemoggin.niobe.net> References: <20160628211915.9706BB59C8@eggemoggin.niobe.net> Message-ID: <1683195038.1023897.1467191318948.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Mark Reinhold" > ?: jpms-spec-experts at openjdk.java.net > Envoy?: Mardi 28 Juin 2016 23:19:15 > Objet: Proposal: #ModuleAnnotations and #ModuleDeprecation > > Issue summaries > --------------- > > #ModuleAnnotations --- Should it be possible to annotate module > declarations? As previously discussed this is technically feasible > but it would add significant complexity. [1] > > #ModuleDeprecation -- If module declarations cannot be annotated, > should there be some other way to deprecate modules? [2] > > Proposal > -------- > > Allow arbitrary annotations to be placed on module declarations, and > revise the `java.lang.Deprecated` annotation so that it can be applied > to modules. > > - Add a new value, `MODULE`, to `java.lang.annotation.ElementType`, > to denote the top level of a module declaration. > > - Extend `javax.lang.model` and `java.lang.reflect` to provide access > to module annotations. Repeating annotations are supported, subject > to the usual rules for repeatable annotation types. (This proposal > will not add similar support to `java.lang.module.ModuleDescriptor`; > that would add yet more complexity, and we haven't seen a use case > for it.) > > - Annotations on modules will never affect the operation of the module > system itself. > > - Revise `java.lang.Deprecated` and `java.lang.SuppressWarnings` to > allow `ElementType.MODULE` as a target. For now, no other existing > Java SE annotations will apply to modules. > > Also, allow `import` declarations [3] to be used in module compilation > units. > > - Any `import` declarations must appear before the `module` declaration. > > - As a consequence of supporting `import` declarations, simple type > names can now be used in the `uses` and `provides` directives of a > module declaration. Type annotations on these type names are not > supported. > > Rationale > --------- > > When we first discussed this topic I wrote [4]: > > > We haven't seen a compelling use case (so far) for annotations on module > > declarations, or on clauses in module declarations, and they aren't a > > documented requirement, so for simplicity we left them out of the initial > > design and the prototype. > > The compelling use case is, of course, deprecation, as R?mi pointed out > [5]. In Java SE 9, in particular, we very much want to be able to write > something like this: > > @Deprecated(since = "9", forRemoval = true) > module java.corba { > ... > } > > The most natural way to do this is to allow module declarations to be > annotated in the same way as other top-level declarations (i.e., packages > and classes). If we don't allow the `@Deprecated` annotation to be used > on modules then we'd have to invent some other way to deprecate modules > (another new keyword?), and that would just be confusing. > > Like most language features, annotations have sometimes been over-used > and even abused [6]. Allowing modules to be annotated opens up a whole > new avenue for unfortunate design decisions that lead to unreadable code. > We could try to limit the damage by allowing just a few annotations on > module declarations (e.g., only those in the `java.lang` package), or by > not supporting `import` declarations so that the use of non-platform > annotations is discouragingly verbose. Such an approach would lead to > unpleasant asymmetries in the language, however, so instead we propose to > support arbitrary annotations on module declarations in the obvious way. > > I think the guidance I wrote earlier [4] still holds. Annotations are > useful for extra-linguistic statements, such as deprecation, that make > sense in source code and can usefully be carried through the tool chain > into the resulting class file. I still don't think they're appropriate > for information that is more appropriately the domain of build systems, > e.g., version strings. > > Notes > ----- > > - Should it be possible to annotate the individual directives in a > module declaration? This would allow you to deprecate individual > exports and requires-public directives, which could be useful in the > process of evolving an API: If you plan to remove a package, or > remove a public dependence upon some other module, then you could > deprecate the corresponding `exports` or `requires public` directive > (presumably with `forRemoval = true`) to warn your users that the > API is going to change. > > There are no cases in the JDK itself where we need this ability, at least > not desperately. Designing and implementing this would be a non-trivial > exercise, and it is a separable feature that could be added later on if > justified by experience. I like both proposals, adding annotations and imports on module. I disagree that those annotations should not be visible at runtime, i don't see why we need a special case here, it's not complex to implement because the VM already provide the hook to access to those annotations at runtime, @Deprecated is an annotation visible at runtime for a good reason, it allows by example to implement policy like disallowing or emit a warning when calling a deprecated code at runtime. R?mi > > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleAnnotations > [2] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleDeprecation > [3] http://docs.oracle.com/javase/specs/jls/se8/html/jls-7.html#jls-7.5 > [4] > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-October/000153.html > [5] > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-March/000263.html > [6] http://annotatiomania.com/ > From stef at epardaud.fr Wed Jun 29 09:11:12 2016 From: stef at epardaud.fr (Stephane Epardaud) Date: Wed, 29 Jun 2016 11:11:12 +0200 Subject: Proposal: #CompileTimeDependences: `requires static` In-Reply-To: <20160628163150.929695476eggemoggin.niobe.net> References: <20160628211715.84610B59BF@eggemoggin.niobe.net> <1808187852.884711.1467154252074.JavaMail.zimbra@u-pem.fr> <20160628163150.929695476eggemoggin.niobe.net> Message-ID: <577390B0.9060106@epardaud.fr> Hi, Great resolution, except for `static` which sounds like reusing a keyword for no good reason, since module descriptors can have new keywords such as `uses`. If `requires optional` is unacceptable (it does not really sound so outrageous to my non-native-english eyes), perhaps `requires softly` would work? Frankly, at this point I think the problem is that `requires` should be `import` which would also solve this issue with `import optional`. Regards. From forax at univ-mlv.fr Wed Jun 29 09:11:12 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 29 Jun 2016 11:11:12 +0200 (CEST) Subject: Proposal: #ReflectiveAccessByInstrumentationAgents In-Reply-To: <20160628212115.9BDC4B59D0@eggemoggin.niobe.net> References: <20160628212115.9BDC4B59D0@eggemoggin.niobe.net> Message-ID: <1551556214.1025252.1467191472326.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Mark Reinhold" > ?: jpms-spec-experts at openjdk.java.net > Envoy?: Mardi 28 Juin 2016 23:21:15 > Objet: Proposal: #ReflectiveAccessByInstrumentationAgents > > Issue summary > ------------- > > #ReflectiveAccessByInstrumentationAgents --- As noted in the related > issue #ReflectiveAccessToNonExportedTypes [1], the core reflection API > cannot be used to gain access to members of packages that are not > exported by their defining modules. Even if that issue is addressed > for the use cases cited, a distinct use case is that of instrumentation > agents that need to enable injected code to invoke non-public methods > and access non-public fields. A possible solution for this use case is > specifically to provide instrumentation agents with the ability to use > core reflection, or some equivalent, even on elements of non-exported > packages. [2] > > Proposal > -------- > > Extend the `java.lang.instrument.Instrumentation` interface, and the > corresponding JVM TI native API, to support a `redefineModule` method. > This method is roughly analogous to the existing `redefineClasses` > method. It will allow instrumentation agents to update an existing > module, represented by a `java.lang.reflect.Module` object, to read > additional modules, export additional packages, use additional services, > and provide additional services. > > Remove the `addModuleReads` method from the `Instrumentation` interface, > and remove the `AddModuleReads` and `CanReadModule` functions from JNI, > since they are no longer needed. > I like this proposal ! R?mi > > Notes > ----- > > - This new API will not allow read, export, uses, and provides > relationships to be dropped at run time. The use cases do not > require this, and implementing it would be very difficult. > > - A sufficiently-skilled instrumentation-agent developer can already do > some of this with the existing API, by injecting new classes into > target modules with initializers that invoke the necessary `Module` > methods. This is, however, awkward at best. > > - Objects that implement the `Instrumentation` interface are already > very powerful, since they can be used to redefine arbitrary classes. > Just as is the case today, in order to preserve platform integrity > instrumentation agents must be coded very carefully so that they do > not leak such objects to user code. > > > [1] > http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccessToNonExportedTypes > [2] > http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccessByInstrumentationAgents > From forax at univ-mlv.fr Wed Jun 29 09:14:21 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 29 Jun 2016 11:14:21 +0200 (CEST) Subject: Proposal: #BootstrapClassLoaderSearchInJVMTI In-Reply-To: <20160628212215.9E7D9B59D4@eggemoggin.niobe.net> References: <20160628212215.9E7D9B59D4@eggemoggin.niobe.net> Message-ID: <466027723.1027081.1467191661253.JavaMail.zimbra@u-pem.fr> I like this proposal. R?mi ----- Mail original ----- > De: "Mark Reinhold" > ?: jpms-spec-experts at openjdk.java.net > Envoy?: Mardi 28 Juin 2016 23:22:15 > Objet: Proposal: #BootstrapClassLoaderSearchInJVMTI > > Issue summary > ------------- > > #BootstrapClassLoaderSearchInJVMTI --- The JVM TI API [1] defines a > function, `AddToBootstrapClassLoaderSearch` [2], by which an > instrumentation agent can append a file-system path element to the > bootstrap class loader's class path. The bootstrap class path, as > such, no longer exists, but this function remains supported for now. > Should this function be deprecated, with the intent to remove it in a > future release, or should it remain supported? It appears to be > critical to the operation of at least one popular instrumentation > agent. [3] > > Proposal > -------- > > Retain this method. Do not deprecate it. > > > [1] http://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html > [2] > http://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html#AddToBootstrapClassLoaderSearch > [3] > http://openjdk.java.net/projects/jigsaw/spec/issues/#BootstrapClassLoaderSearchInJVMTI > From forax at univ-mlv.fr Wed Jun 29 09:14:44 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 29 Jun 2016 11:14:44 +0200 (CEST) Subject: Proposal: #CustomizableAutomaticModuleNameMapping In-Reply-To: <20160628212315.A247FB59D8@eggemoggin.niobe.net> References: <20160628212315.A247FB59D8@eggemoggin.niobe.net> Message-ID: <1490559688.1027428.1467191684507.JavaMail.zimbra@u-pem.fr> A fine idea. R?mi ----- Mail original ----- > De: "Mark Reinhold" > ?: jpms-spec-experts at openjdk.java.net > Envoy?: Mardi 28 Juin 2016 23:23:15 > Objet: Proposal: #CustomizableAutomaticModuleNameMapping > > Issue summary > ------------- > > #CustomizableAutomaticModuleNameMapping --- Provide a means to > customize the algorithm that computes the names and version strings > of automatic modules. [1] > > Proposal > -------- > > Extend the `java.lang.module.ModuleDescriptor.Builder` API so that it > can be used to create automatic as well as explicit modules. This will > allow a container application to use a custom `ModuleFinder` to manage > the mapping from artifact names to module names, version strings, and > so forth. > > > [1] > http://openjdk.java.net/projects/jigsaw/spec/issues/#CustomizableAutomaticModuleNameMapping > From Tim_Ellison at uk.ibm.com Wed Jun 29 09:30:28 2016 From: Tim_Ellison at uk.ibm.com (Tim Ellison) Date: Wed, 29 Jun 2016 10:30:28 +0100 Subject: Proposal: #CompileTimeDependences: `requires static` In-Reply-To: <20160628211715.84610B59BF@eggemoggin.niobe.net> References: <20160628211715.84610B59BF@eggemoggin.niobe.net> Message-ID: mark.reinhold at oracle.com (Mark Reinhold) wrote on 28/06/2016 22:17:15: > Issue summary > ------------- > > #CompileTimeDependences --- Provide a means to specify a module > dependence that is mandatory at compile time but optional at run time, > for use with libraries that are not strictly necessary but can be > leveraged if present at run time. (Detail: If a dependence is > "optional at run time" then does the module system try to satisfy it > during resolution but fail silently if it cannot, or does it ignore it > during resolution but add the corresponding read edge to that module if > it was resolved for some other reason?) [1] > > Use cases for this feature include annotations that need not be present > at run time [2] and "auto-configuring" libraries and frameworks such as > Joda Beans [3] and Spring [4]. For some reason I can't quite explain, this feels like we are giving up on one of the fundamental completeness assurances of a coherent modular application. Is this a new failure mode for link/run time modular systems that has not previously existed? It may be that this proposal satisfies an important compatibility story for existing code bases, and that is the justification, because on first reading my reaction is that the code has to be written to deal with the absence of the optional module anyway, more details below. > Proposal > -------- > > Extend the language of module declarations to allow the `static` modifier > to be used on a `requires` directive, with the following meanings: > > - At compile time, `requires static M` expresses a mandatory > dependence. It is an error if a suitable module cannot be found > amongst the observable modules and resolved. > > - In phases after compile time, `requires static M` expresses an > optional dependence. The module system will not search the > observable modules for a suitable module during resolution, but > if the resulting module graph contains a suitable module then it > will add the appropriate readability edge prior to doing the usual > post-resolution sanity checks. (The parenthetical question in the > issue summary is, in other words, answered in the negative.) I read the parenthetical question as an either-or ... > Thus a hypothetical module declaration of the form > > module joda.beans { > requires static joda.collect; > ... > } > > would ensure that the `joda.collect` module is available at compile time, > so that code in the `joda.beans` module that refers to `joda.collect` can > be compiled without any fuss. It would not, however, guarantee that > `joda.collect` is available at link time or run time. A user who wants > to use `joda.beans` together with `joda.collect` in those later phases > must ensure that `joda.collect` is added to the module graph by, e.g., > declaring a dependence upon it in some other module, or by using the > `-addmods` option or its equivalent. > > The code in `joda.beans` that refers to types in `joda.collect` must, of > course, be written defensively so that it fails gracefully at run time > when the `joda.collect` module is not present. This is a key point. Since the application code has to be robust in the face of a missing "requires static" module, wouldn't it be better served by coding the behvior of the required module as a service that has a "fail gracefully" implementation and can be replaced by a productive implementation if required at link/run time? I guess a weak analogy would be using compile against JARs that have no runtime equivalent, and are used to produce code that can fail at runtime, vs. having a type of service or strategy pattern that encompasses the fail gracefully / be productive implementations. > It is possible to combine the `public` and `static` modifiers on a > `requires` directive: > > module joda.beans { > requires static joda.collect; > requires public static freemarker; > ... > } > > This is useful in this particular case since `joda.beans` defines two > exported public classes that extend classes defined in the `freemarker` > module, and so those classes are logically part of the `joda.beans` API. > In general, however, this idiom leads to fragile APIs and is therefore > not advised except when dealing with legacy code. > > Notes > ----- > > - In phases after compile time, why doesn't the module system search > the observable modules in an attempt to satisfy an optional > dependence? > > This would not be difficult to arrange but it could lead to surprising > behavior in which optional dependences silently cause many additional > modules to be resolved. Given the use cases we have it does not seem > unreasonable to expect an end user to indicate explicitly, either in > source code or out-of-band (e.g., with a command-line option such as > `-addmods`), that an otherwise-optional module is actually needed. Another option would be to provide a command-line option that explicitly allows the linker/launcher to ignore missing modules that are required where there is an assurance that it will not actually be required in the particular context; that is, if I design a single module with "requires" or "requires static" dependencies if it is used in a stand-alone or container context, for example. I'd prefer to fail because of an explicit -ignoremods option on the command line rather than a missing -addmods. Regards, Tim > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/ > #CompileTimeDependences > [2] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-October/004915.html > [3] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/ > 005462.html > [4] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/ > 005814.html > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Tim_Ellison at uk.ibm.com Wed Jun 29 09:45:31 2016 From: Tim_Ellison at uk.ibm.com (Tim Ellison) Date: Wed, 29 Jun 2016 10:45:31 +0100 Subject: Proposal: #ModuleAnnotations and #ModuleDeprecation In-Reply-To: <20160628211915.9706BB59C8@eggemoggin.niobe.net> References: <20160628211915.9706BB59C8@eggemoggin.niobe.net> Message-ID: mark.reinhold at oracle.com (Mark Reinhold) wrote on 28/06/2016 22:19:15: > Issue summaries > --------------- > > #ModuleAnnotations --- Should it be possible to annotate module > declarations? As previously discussed this is technically feasible > but it would add significant complexity. [1] > > #ModuleDeprecation -- If module declarations cannot be annotated, > should there be some other way to deprecate modules? [2] > > Proposal > -------- > > Allow arbitrary annotations to be placed on module declarations, and > revise the `java.lang.Deprecated` annotation so that it can be applied > to modules. > > - Add a new value, `MODULE`, to `java.lang.annotation.ElementType`, > to denote the top level of a module declaration. > > - Extend `javax.lang.model` and `java.lang.reflect` to provide access > to module annotations. Repeating annotations are supported, subject > to the usual rules for repeatable annotation types. (This proposal > will not add similar support to `java.lang.module.ModuleDescriptor`; > that would add yet more complexity, and we haven't seen a use case > for it.) > > - Annotations on modules will never affect the operation of the module > system itself. > > - Revise `java.lang.Deprecated` and `java.lang.SuppressWarnings` to > allow `ElementType.MODULE` as a target. For now, no other existing > Java SE annotations will apply to modules. > > Also, allow `import` declarations [3] to be used in module compilation > units. > > - Any `import` declarations must appear before the `module` declaration. > > - As a consequence of supporting `import` declarations, simple type > names can now be used in the `uses` and `provides` directives of a > module declaration. Type annotations on these type names are not > supported. > > Rationale > --------- > > When we first discussed this topic I wrote [4]: > > > We haven't seen a compelling use case (so far) for annotations on module > > declarations, or on clauses in module declarations, and they aren't a > > documented requirement, so for simplicity we left them out of the initial > > design and the prototype. > > The compelling use case is, of course, deprecation, as R??mi pointed out > [5]. In Java SE 9, in particular, we very much want to be able to write > something like this: > > @Deprecated(since = "9", forRemoval = true) > module java.corba { > ... > } > > The most natural way to do this is to allow module declarations to be > annotated in the same way as other top-level declarations (i.e., packages > and classes). If we don't allow the `@Deprecated` annotation to be used > on modules then we'd have to invent some other way to deprecate modules > (another new keyword?), and that would just be confusing. > > Like most language features, annotations have sometimes been over-used > and even abused [6]. Allowing modules to be annotated opens up a whole > new avenue for unfortunate design decisions that lead to unreadable code. > We could try to limit the damage by allowing just a few annotations on > module declarations (e.g., only those in the `java.lang` package), or by > not supporting `import` declarations so that the use of non-platform > annotations is discouragingly verbose. Such an approach would lead to > unpleasant asymmetries in the language, however, so instead we propose to > support arbitrary annotations on module declarations in the obvious way. > > I think the guidance I wrote earlier [4] still holds. Annotations are > useful for extra-linguistic statements, such as deprecation, that make > sense in source code and can usefully be carried through the tool chain > into the resulting class file. I still don't think they're appropriate > for information that is more appropriately the domain of build systems, > e.g., version strings. > > Notes > ----- > > - Should it be possible to annotate the individual directives in a > module declaration? This would allow you to deprecate individual > exports and requires-public directives, which could be useful in the > process of evolving an API: If you plan to remove a package, or > remove a public dependence upon some other module, then you could > deprecate the corresponding `exports` or `requires public` directive > (presumably with `forRemoval = true`) to warn your users that the > API is going to change. > > There are no cases in the JDK itself where we need this ability, at least > not desperately. Designing and implementing this would be a non-trivial > exercise, and it is a separable feature that could be added later on if > justified by experience. > > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleAnnotations > [2] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleDeprecation > [3] http://docs.oracle.com/javase/specs/jls/se8/html/jls-7.html#jls-7.5 > [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015- > October/000153.html > [5] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016- > March/000263.html > [6] http://annotatiomania.com/ > I'm +1 to this proposal, and agree with all that is written above about rationale and scope (though, of course, you know the module annotations *will* be abused immediately!). Regards, Tim Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From forax at univ-mlv.fr Wed Jun 29 10:01:03 2016 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Wed, 29 Jun 2016 12:01:03 +0200 (CEST) Subject: Proposal: #CompileTimeDependences: `requires static` In-Reply-To: <577384C9.4010209@cjnash.com> References: <20160628211715.84610B59BF@eggemoggin.niobe.net> <1808187852.884711.1467154252074.JavaMail.zimbra@u-pem.fr> <20160628163150.929695476eggemoggin.niobe.net> <577384C9.4010209@cjnash.com> Message-ID: <1031264805.1055240.1467194463228.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Simon Nash" > ?: "mark reinhold" > Cc: "Remi Forax" , jpms-spec-observers at openjdk.java.net > Envoy?: Mercredi 29 Juin 2016 10:20:25 > Objet: Re: Proposal: #CompileTimeDependences: `requires static` > > mark.reinhold at oracle.com wrote: > > 2016/6/28 15:50:52 -0700, Remi Forax : > >> Apart the fact that 'static' should be spelled 'optional', there is no > >> reason to reuse static as it doesn't convey the semantics we want, > >> i.e. optional at runtime, i fully agree with this proposal. > > > > In this context `static` is intended to mean "at compile time" or, > > equivalently, before the more "dynamic" phases of run time and link > > time (the latter of which is more like run time than compile time). > > > > I agree that `static` is potentially confusing since its meaning here > > is different from what `static` means when used on a member of a class. > > > > It does, however, fit nicely with its (nearly) dual directive, `exports > > dynamic`, proposed for #ReflectiveAccessToNonExportedTypes. > > > > I think `optional` is a non-starter, since `requires optional` reads as > > an oxymoron, and it's optional at run time but mandatory at compile > > time, so in what sense is it, really, "optional"? > > > > Suggestions of other alternatives are welcome ... > > > > - Mark > > > I agree with Remi that 'static' isn't a good choice. One the nice > improvements of Java over C was Java's cleaner and more consistent usage > of the 'static' keyword. > > How about 'requires import' or 'requires imported'? This is consistent > with the current usage of 'import' to declare compile-time dependencies > on other packages. > > Simon > @Mark, yes, optional comes from OSGI where you can optionally import something, optional make more sense for OSGI because OSGI only works at runtime so optional is implicitly optional at runtime. @Simon The issue with 'import' meaning compile time only is that more Java developers than we want doesn't make the difference between import in Java and say import in Python. so what about 'compileonly' ? R?mi From Tim_Ellison at uk.ibm.com Wed Jun 29 10:15:38 2016 From: Tim_Ellison at uk.ibm.com (Tim Ellison) Date: Wed, 29 Jun 2016 11:15:38 +0100 Subject: Proposal: #ResourceEncapsulation and #ClassFilesAsResources In-Reply-To: <20160628212015.99A06B59CC@eggemoggin.niobe.net> References: <20160628212015.99A06B59CC@eggemoggin.niobe.net> Message-ID: mark.reinhold at oracle.com (Mark Reinhold) wrote on 28/06/2016 22:20:15: > Issue summaries > --------------- > > #ClassFilesAsResources --- If a type is visible and was loaded from a > class file then it should be possible to read that file by invoking the > `getResourceAsStream` method of the type's class loader, as it is in > earlier releases. [1] > > #ResourceEncapsulation --- The `Module::getResourceAsStream` method can > be used to read the resources of any named module, without restriction, > which violates the resource-encapsulation requirement [2]. This method > should be restricted somehow so that only "suitably-privileged" code > (for some definition of that term) can access resources in a named > module other than its own. An alternative is to drop this > requirement. [3] > > Proposal > -------- > > Drop the agreed resource-encapsulation requirement, which reads: > > Resource encapsulation --- The run-time system must ensure that the > static resource files within a module are directly accessible only by > code within that module. The existing resource-access APIs should > continue to work as they do today when used to access a module's own > resource files. [2] > > Make the following changes to the various `getResource*` methods: > > - The `ClassLoader::getResource*` methods will first delegate to the > loader's parent, if any, and will then search for resources in the > named modules defined to the loader, and will then search in the > loader's unnamed module, typically defined by a class path. In the > case of the `getResources` method the order of the resources in the > returned enumeration will be unspecified, except that the values of > `cl.getResources(name).nextElement()` and `cl.getResource(name)` > will be equal. > > - The `Class::getResource*` methods will only search for resources in > the module that defines the class. In the case of a class in an > unnamed module this will typically result in searching the class > path of the corresponding class loader. > > The `java.lang.reflect.Module::getResourceAsStream` method will remain, > so that it's possible to look up a resource in a module without having a > reference to a class defined in that module. It will always be the case > that, for a given a class `c`, `c.getResourceAsStream(name)` will be > equivalent to `c.getModule().getResourceAsStream(name)`. > > > Rationale > --------- > > The encapsulation of resources was intended to be part of the overall > encapsulation story, on the view that a module's resources are just > internal implementation details that need not, and should not, be > available to code in other modules. Allowing external code to depend > upon a module's internal resources is just as problematic as allowing > such code to depend upon internal APIs. > > The reality of existing code today, however, is rather different, with > two common use cases: > > - Resources are often used intentionally to convey information to > external code, e.g., as a way to publish configuration files such as > `persistence.xml`. This sort of thing should, ideally, be done via > services [4][5], but that is not the current practice and it will > take years to migrate to it. > > - Various popular tools and byte-code manipulation frameworks expect to > be able to load class files as resources, i.e., to access the class > file that defines a class `p.q.C` by invoking `getResource("p.q.C")` > on that class's loader. > > With the API as it stands today these use cases can still be supported, > but the code that looks up the resource must be converted to use the > `Module::getResourceAsStream` method. This is a barrier to migration, > since it means that fewer existing frameworks will work out-of-the-box > as automatic modules. > > We should not drop agreed requirements lightly, but now that we have > some practical experience with an implementation of this one I think > it's clear that the pain outweighs the gain. > > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ClassFilesAsResources > [2] http://openjdk.java.net/projects/jigsaw/spec/reqs/#resource-encapsulation > [3] http://openjdk.java.net/projects/jigsaw/spec/issues/#ResourceEncapsulation > [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015- > October/000163.html > [5] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015- > December/000207.html > [6] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015- > September/004486.html > [7] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-November/ > 005240.html The proposal to drop this requirement leaves no way for a module to encapsulate its resources. Have you considered an alternative that preserves the compatibility of existing APIs which I agree is desirable to support existing applications, but introduces a module-level resource API that allows those modules that want to hide their implementation details? Regards, Tim Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Tim_Ellison at uk.ibm.com Wed Jun 29 10:30:45 2016 From: Tim_Ellison at uk.ibm.com (Tim Ellison) Date: Wed, 29 Jun 2016 11:30:45 +0100 Subject: Proposal: #ReflectiveAccessByInstrumentationAgents In-Reply-To: <20160628212115.9BDC4B59D0@eggemoggin.niobe.net> References: <20160628212115.9BDC4B59D0@eggemoggin.niobe.net> Message-ID: mark.reinhold at oracle.com (Mark Reinhold) wrote on 28/06/2016 22:21:15: > Issue summary > ------------- > > #ReflectiveAccessByInstrumentationAgents --- As noted in the related > issue #ReflectiveAccessToNonExportedTypes [1], the core reflection API > cannot be used to gain access to members of packages that are not > exported by their defining modules. Even if that issue is addressed > for the use cases cited, a distinct use case is that of instrumentation > agents that need to enable injected code to invoke non-public methods > and access non-public fields. A possible solution for this use case is > specifically to provide instrumentation agents with the ability to use > core reflection, or some equivalent, even on elements of non-exported > packages. [2] > > Proposal > -------- > > Extend the `java.lang.instrument.Instrumentation` interface, and the > corresponding JVM TI native API, to support a `redefineModule` method. > This method is roughly analogous to the existing `redefineClasses` > method. It will allow instrumentation agents to update an existing > module, represented by a `java.lang.reflect.Module` object, to read > additional modules, export additional packages, use additional services, > and provide additional services. > > Remove the `addModuleReads` method from the `Instrumentation` interface, > and remove the `AddModuleReads` and `CanReadModule` functions from JNI, > since they are no longer needed. > > > Notes > ----- > > - This new API will not allow read, export, uses, and provides > relationships to be dropped at run time. The use cases do not > require this, and implementing it would be very difficult. > > - A sufficiently-skilled instrumentation-agent developer can already do > some of this with the existing API, by injecting new classes into > target modules with initializers that invoke the necessary `Module` > methods. This is, however, awkward at best. > > - Objects that implement the `Instrumentation` interface are already > very powerful, since they can be used to redefine arbitrary classes. > Just as is the case today, in order to preserve platform integrity > instrumentation agents must be coded very carefully so that they do > not leak such objects to user code. > > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/ > #ReflectiveAccessToNonExportedTypes > [2] http://openjdk.java.net/projects/jigsaw/spec/issues/ > #ReflectiveAccessByInstrumentationAgents I agree with this proposal. As already pointed out, instrumentation agents are a powerful, somewhat specialist case that suits the full set of manipulation capabilities. Regards, Tim Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Tim_Ellison at uk.ibm.com Wed Jun 29 10:45:53 2016 From: Tim_Ellison at uk.ibm.com (Tim Ellison) Date: Wed, 29 Jun 2016 11:45:53 +0100 Subject: Proposal: #BootstrapClassLoaderSearchInJVMTI In-Reply-To: <20160628212215.9E7D9B59D4@eggemoggin.niobe.net> References: <20160628212215.9E7D9B59D4@eggemoggin.niobe.net> Message-ID: mark.reinhold at oracle.com (Mark Reinhold) wrote on 28/06/2016 22:22:15: > Issue summary > ------------- > > #BootstrapClassLoaderSearchInJVMTI --- The JVM TI API [1] defines a > function, `AddToBootstrapClassLoaderSearch` [2], by which an > instrumentation agent can append a file-system path element to the > bootstrap class loader's class path. The bootstrap class path, as > such, no longer exists, but this function remains supported for now. > Should this function be deprecated, with the intent to remove it in a > future release, or should it remain supported? It appears to be > critical to the operation of at least one popular instrumentation > agent. [3] > > Proposal > -------- > > Retain this method. Do not deprecate it. What would be the effect calling this function on a modular runtime when passed a directory search path segment? a modular JAR? a regular JAR? Regards, Tim > [1] http://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html > [2] http://docs.oracle.com/javase/8/docs/platform/jvmti/ > jvmti.html#AddToBootstrapClassLoaderSearch > [3] http://openjdk.java.net/projects/jigsaw/spec/issues/ > #BootstrapClassLoaderSearchInJVMTI > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From david.lloyd at redhat.com Wed Jun 29 12:25:34 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Wed, 29 Jun 2016 07:25:34 -0500 Subject: Proposal: #ModuleAnnotations and #ModuleDeprecation In-Reply-To: <20160628211915.9706BB59C8@eggemoggin.niobe.net> References: <20160628211915.9706BB59C8@eggemoggin.niobe.net> Message-ID: <5773BE3E.7060104@redhat.com> I like the proposal but I agree with R?mi - it would be useful to allow run-time access. On 06/28/2016 04:19 PM, Mark Reinhold wrote: > Issue summaries > --------------- > > #ModuleAnnotations --- Should it be possible to annotate module > declarations? As previously discussed this is technically feasible > but it would add significant complexity. [1] > > #ModuleDeprecation -- If module declarations cannot be annotated, > should there be some other way to deprecate modules? [2] > > Proposal > -------- > > Allow arbitrary annotations to be placed on module declarations, and > revise the `java.lang.Deprecated` annotation so that it can be applied > to modules. > > - Add a new value, `MODULE`, to `java.lang.annotation.ElementType`, > to denote the top level of a module declaration. > > - Extend `javax.lang.model` and `java.lang.reflect` to provide access > to module annotations. Repeating annotations are supported, subject > to the usual rules for repeatable annotation types. (This proposal > will not add similar support to `java.lang.module.ModuleDescriptor`; > that would add yet more complexity, and we haven't seen a use case > for it.) > > - Annotations on modules will never affect the operation of the module > system itself. > > - Revise `java.lang.Deprecated` and `java.lang.SuppressWarnings` to > allow `ElementType.MODULE` as a target. For now, no other existing > Java SE annotations will apply to modules. > > Also, allow `import` declarations [3] to be used in module compilation > units. > > - Any `import` declarations must appear before the `module` declaration. > > - As a consequence of supporting `import` declarations, simple type > names can now be used in the `uses` and `provides` directives of a > module declaration. Type annotations on these type names are not > supported. > > Rationale > --------- > > When we first discussed this topic I wrote [4]: > >> We haven't seen a compelling use case (so far) for annotations on module >> declarations, or on clauses in module declarations, and they aren't a >> documented requirement, so for simplicity we left them out of the initial >> design and the prototype. > > The compelling use case is, of course, deprecation, as R?mi pointed out > [5]. In Java SE 9, in particular, we very much want to be able to write > something like this: > > @Deprecated(since = "9", forRemoval = true) > module java.corba { > ... > } > > The most natural way to do this is to allow module declarations to be > annotated in the same way as other top-level declarations (i.e., packages > and classes). If we don't allow the `@Deprecated` annotation to be used > on modules then we'd have to invent some other way to deprecate modules > (another new keyword?), and that would just be confusing. > > Like most language features, annotations have sometimes been over-used > and even abused [6]. Allowing modules to be annotated opens up a whole > new avenue for unfortunate design decisions that lead to unreadable code. > We could try to limit the damage by allowing just a few annotations on > module declarations (e.g., only those in the `java.lang` package), or by > not supporting `import` declarations so that the use of non-platform > annotations is discouragingly verbose. Such an approach would lead to > unpleasant asymmetries in the language, however, so instead we propose to > support arbitrary annotations on module declarations in the obvious way. > > I think the guidance I wrote earlier [4] still holds. Annotations are > useful for extra-linguistic statements, such as deprecation, that make > sense in source code and can usefully be carried through the tool chain > into the resulting class file. I still don't think they're appropriate > for information that is more appropriately the domain of build systems, > e.g., version strings. > > Notes > ----- > > - Should it be possible to annotate the individual directives in a > module declaration? This would allow you to deprecate individual > exports and requires-public directives, which could be useful in the > process of evolving an API: If you plan to remove a package, or > remove a public dependence upon some other module, then you could > deprecate the corresponding `exports` or `requires public` directive > (presumably with `forRemoval = true`) to warn your users that the > API is going to change. > > There are no cases in the JDK itself where we need this ability, at least > not desperately. Designing and implementing this would be a non-trivial > exercise, and it is a separable feature that could be added later on if > justified by experience. > > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleAnnotations > [2] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleDeprecation > [3] http://docs.oracle.com/javase/specs/jls/se8/html/jls-7.html#jls-7.5 > [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-October/000153.html > [5] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-March/000263.html > [6] http://annotatiomania.com/ > -- - DML From scolebourne at joda.org Wed Jun 29 12:30:00 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 29 Jun 2016 13:30:00 +0100 Subject: Proposal: #ModuleAnnotations and #ModuleDeprecation In-Reply-To: <20160628211915.9706BB59C8@eggemoggin.niobe.net> References: <20160628211915.9706BB59C8@eggemoggin.niobe.net> Message-ID: Like Remi and David, I support the proposal, but feel that annotations should be available at runtime. Not doing so would be very surprising to end users, and would be an "unpleasant asymmetry in the language". While I can see the case for annotations within the module-info file (Stephane makes a good case), I'm OK with omitting it. Stephen On 28 June 2016 at 22:19, Mark Reinhold wrote: > Issue summaries > --------------- > > #ModuleAnnotations --- Should it be possible to annotate module > declarations? As previously discussed this is technically feasible > but it would add significant complexity. [1] > > #ModuleDeprecation -- If module declarations cannot be annotated, > should there be some other way to deprecate modules? [2] > > Proposal > -------- > > Allow arbitrary annotations to be placed on module declarations, and > revise the `java.lang.Deprecated` annotation so that it can be applied > to modules. > > - Add a new value, `MODULE`, to `java.lang.annotation.ElementType`, > to denote the top level of a module declaration. > > - Extend `javax.lang.model` and `java.lang.reflect` to provide access > to module annotations. Repeating annotations are supported, subject > to the usual rules for repeatable annotation types. (This proposal > will not add similar support to `java.lang.module.ModuleDescriptor`; > that would add yet more complexity, and we haven't seen a use case > for it.) > > - Annotations on modules will never affect the operation of the module > system itself. > > - Revise `java.lang.Deprecated` and `java.lang.SuppressWarnings` to > allow `ElementType.MODULE` as a target. For now, no other existing > Java SE annotations will apply to modules. > > Also, allow `import` declarations [3] to be used in module compilation > units. > > - Any `import` declarations must appear before the `module` declaration. > > - As a consequence of supporting `import` declarations, simple type > names can now be used in the `uses` and `provides` directives of a > module declaration. Type annotations on these type names are not > supported. > > Rationale > --------- > > When we first discussed this topic I wrote [4]: > >> We haven't seen a compelling use case (so far) for annotations on module >> declarations, or on clauses in module declarations, and they aren't a >> documented requirement, so for simplicity we left them out of the initial >> design and the prototype. > > The compelling use case is, of course, deprecation, as R?mi pointed out > [5]. In Java SE 9, in particular, we very much want to be able to write > something like this: > > @Deprecated(since = "9", forRemoval = true) > module java.corba { > ... > } > > The most natural way to do this is to allow module declarations to be > annotated in the same way as other top-level declarations (i.e., packages > and classes). If we don't allow the `@Deprecated` annotation to be used > on modules then we'd have to invent some other way to deprecate modules > (another new keyword?), and that would just be confusing. > > Like most language features, annotations have sometimes been over-used > and even abused [6]. Allowing modules to be annotated opens up a whole > new avenue for unfortunate design decisions that lead to unreadable code. > We could try to limit the damage by allowing just a few annotations on > module declarations (e.g., only those in the `java.lang` package), or by > not supporting `import` declarations so that the use of non-platform > annotations is discouragingly verbose. Such an approach would lead to > unpleasant asymmetries in the language, however, so instead we propose to > support arbitrary annotations on module declarations in the obvious way. > > I think the guidance I wrote earlier [4] still holds. Annotations are > useful for extra-linguistic statements, such as deprecation, that make > sense in source code and can usefully be carried through the tool chain > into the resulting class file. I still don't think they're appropriate > for information that is more appropriately the domain of build systems, > e.g., version strings. > > Notes > ----- > > - Should it be possible to annotate the individual directives in a > module declaration? This would allow you to deprecate individual > exports and requires-public directives, which could be useful in the > process of evolving an API: If you plan to remove a package, or > remove a public dependence upon some other module, then you could > deprecate the corresponding `exports` or `requires public` directive > (presumably with `forRemoval = true`) to warn your users that the > API is going to change. > > There are no cases in the JDK itself where we need this ability, at least > not desperately. Designing and implementing this would be a non-trivial > exercise, and it is a separable feature that could be added later on if > justified by experience. > > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleAnnotations > [2] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleDeprecation > [3] http://docs.oracle.com/javase/specs/jls/se8/html/jls-7.html#jls-7.5 > [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-October/000153.html > [5] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-March/000263.html > [6] http://annotatiomania.com/ From david.lloyd at redhat.com Wed Jun 29 12:37:34 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Wed, 29 Jun 2016 07:37:34 -0500 Subject: Proposal: #CompileTimeDependences: `requires static` In-Reply-To: <20160628163150.929695476eggemoggin.niobe.net> References: <20160628211715.84610B59BF@eggemoggin.niobe.net> <1808187852.884711.1467154252074.JavaMail.zimbra@u-pem.fr> <20160628163150.929695476eggemoggin.niobe.net> Message-ID: <5773C10E.7000108@redhat.com> On 06/28/2016 06:31 PM, mark.reinhold at oracle.com wrote: > 2016/6/28 15:50:52 -0700, Remi Forax : >> Apart the fact that 'static' should be spelled 'optional', there is no >> reason to reuse static as it doesn't convey the semantics we want, >> i.e. optional at runtime, i fully agree with this proposal. I agree that 'static' really doesn't have any existing meaning in this context, so we'd be inventing another connotation for that term. > In this context `static` is intended to mean "at compile time" or, > equivalently, before the more "dynamic" phases of run time and link > time (the latter of which is more like run time than compile time). > > I agree that `static` is potentially confusing since its meaning here > is different from what `static` means when used on a member of a class. > > It does, however, fit nicely with its (nearly) dual directive, `exports > dynamic`, proposed for #ReflectiveAccessToNonExportedTypes. > > I think `optional` is a non-starter, since `requires optional` reads as > an oxymoron, and it's optional at run time but mandatory at compile > time, so in what sense is it, really, "optional"? > > Suggestions of other alternatives are welcome ... Some random ideas: "optional_at_runtime" - hey, it's descriptive. Use "compile" or "compile_time" instead, i.e. indicate where the dependency is required, not where it isn't required. Use "optional" instead of "requires"; this has a grammatical problem though because "requires" is a verb but "optional" is an adjective. You could use "required" instead, but that's weird I guess. Drop "requires" completely and describe dependencies in a different way, like "dependenc[ey] joda.collect run(optional) compile(required)". Annotations? Just let build tools rewrite the descriptor after the fact, as seems inevitable anyway. -- - DML From Tim_Ellison at uk.ibm.com Wed Jun 29 13:01:23 2016 From: Tim_Ellison at uk.ibm.com (Tim Ellison) Date: Wed, 29 Jun 2016 14:01:23 +0100 Subject: Proposal: #CustomizableAutomaticModuleNameMapping In-Reply-To: <20160628212315.A247FB59D8@eggemoggin.niobe.net> References: <20160628212315.A247FB59D8@eggemoggin.niobe.net> Message-ID: mark.reinhold at oracle.com (Mark Reinhold) wrote on 28/06/2016 22:23:15: > Issue summary > ------------- > > #CustomizableAutomaticModuleNameMapping --- Provide a means to > customize the algorithm that computes the names and version strings > of automatic modules. [1] > > Proposal > -------- > > Extend the `java.lang.module.ModuleDescriptor.Builder` API so that it > can be used to create automatic as well as explicit modules. This will > allow a container application to use a custom `ModuleFinder` to manage > the mapping from artifact names to module names, version strings, and > so forth. > > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/ > #CustomizableAutomaticModuleNameMapping Enthusiastic +1 for this proposal. I envisage a number of places from which such auto information may be derived. Regards, Tim Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From scolebourne at joda.org Wed Jun 29 13:12:34 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 29 Jun 2016 14:12:34 +0100 Subject: Proposal: #CompileTimeDependences: `requires static` In-Reply-To: References: <20160628211715.84610B59BF@eggemoggin.niobe.net> Message-ID: There are three use cases that I have used personally for this: 1) The dependency contains an annotation that is not required at runtime (classes load fine if annotations are not available). For example, Joda-Time [1] dependency on Joda-Convert. 2) The main module has reflective code that checks whether a dependency exists and auto-configures if it is available. For example, Joda-Beans [2] dependency on Joda-Collect and Guava. (If Joda-Collect is available then any data structure type the library can be serialized using Joda-Beans). 3) The main module has isolated classes that can only be used if the option dependency exists. For example, ElSql [3] dependency on Spring. (If you use ElSql class as the main entry point you do not need Spring, but if you use the ElSqlBundle class you do) The issue and resolution are designed to support the flexible approach to modules that applications have made use of for 20 years. While it might be possible to migrate (2) to a service, that would block adoption. Items (1) and (3) are not feasible via a service. Note that I do not believe command line options satisfy this issue - because they are unwieldy, unfriendly and a barrier to adoption. In addition, they fail DRY. In this case, the library author knows that the dependency is optional, and the library user is warned of that. Adding a command line flag is just unnecessary noise that the system should deal with. [1] http://www.joda.org/joda-time/ [2] http://www.joda.org/joda-beans/ [3] https://github.com/OpenGamma/ElSql On 29 June 2016 at 10:30, Tim Ellison wrote: > For some reason I can't quite explain, this feels like we are giving up > on one of the fundamental completeness assurances of a coherent modular > application. Is this a new failure mode for link/run time modular systems > that has not previously existed? > Another option would be to provide a command-line option that explicitly > allows > the linker/launcher to ignore missing modules that are required where > there is > an assurance that it will not actually be required in the particular > context; > that is, if I design a single module with "requires" or "requires static" > dependencies if it is used in a stand-alone or container context, for > example. > > I'd prefer to fail because of an explicit -ignoremods option on the > command > line rather than a missing -addmods. From david.lloyd at redhat.com Wed Jun 29 13:21:39 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Wed, 29 Jun 2016 08:21:39 -0500 Subject: Proposal: #ResourceEncapsulation and #ClassFilesAsResources In-Reply-To: <20160628212015.99A06B59CC@eggemoggin.niobe.net> References: <20160628212015.99A06B59CC@eggemoggin.niobe.net> Message-ID: <5773CB63.6030308@redhat.com> Responses inline & at bottom. On 06/28/2016 04:20 PM, Mark Reinhold wrote: > Issue summaries > --------------- > > #ClassFilesAsResources --- If a type is visible and was loaded from a > class file then it should be possible to read that file by invoking the > `getResourceAsStream` method of the type's class loader, as it is in > earlier releases. [1] > > #ResourceEncapsulation --- The `Module::getResourceAsStream` method can > be used to read the resources of any named module, without restriction, > which violates the resource-encapsulation requirement [2]. This method > should be restricted somehow so that only "suitably-privileged" code > (for some definition of that term) can access resources in a named > module other than its own. An alternative is to drop this > requirement. [3] > > Proposal > -------- > > Drop the agreed resource-encapsulation requirement, which reads: > > Resource encapsulation --- The run-time system must ensure that the > static resource files within a module are directly accessible only by > code within that module. The existing resource-access APIs should > continue to work as they do today when used to access a module's own > resource files. [2] OK so far. > Make the following changes to the various `getResource*` methods: > > - The `ClassLoader::getResource*` methods will first delegate to the > loader's parent, if any, and will then search for resources in the > named modules defined to the loader, and will then search in the > loader's unnamed module, typically defined by a class path. In the > case of the `getResources` method the order of the resources in the > returned enumeration will be unspecified, except that the values of > `cl.getResources(name).nextElement()` and `cl.getResource(name)` > will be equal. I am really not a fan of unspecified resource order, especially when you can easily derive a stable order from the graph topology. Resources however should always load from the child-most loader first. This isn't a problem in Jigsaw for classes because packages cannot be duplicated between modules (in a visible manner), but it becomes a problem with resources, especially if/when run-time cycles are permitted: you can run into an awkward situation where two mutually dependent modules each may see the other's resource "first" before their own, which is almost always undesirable. > - The `Class::getResource*` methods will only search for resources in > the module that defines the class. In the case of a class in an > unnamed module this will typically result in searching the class > path of the corresponding class loader. Does this also apply to automatic modules? > The `java.lang.reflect.Module::getResourceAsStream` method will remain, > so that it's possible to look up a resource in a module without having a > reference to a class defined in that module. It will always be the case > that, for a given a class `c`, `c.getResourceAsStream(name)` will be > equivalent to `c.getModule().getResourceAsStream(name)`. I believe that in the former case, the "name" was historically relative to the location of the class. Is that behavior changing? > Rationale > --------- > > The encapsulation of resources was intended to be part of the overall > encapsulation story, on the view that a module's resources are just > internal implementation details that need not, and should not, be > available to code in other modules. Allowing external code to depend > upon a module's internal resources is just as problematic as allowing > such code to depend upon internal APIs. > > The reality of existing code today, however, is rather different, with > two common use cases: > > - Resources are often used intentionally to convey information to > external code, e.g., as a way to publish configuration files such as > `persistence.xml`. This sort of thing should, ideally, be done via > services [4][5], but that is not the current practice and it will > take years to migrate to it. In many cases services are too limiting. It is useful for users to have a more powerful, general facility available that can use the same basic mechanism as services, but for arbitrary resources. For example, one case that came up quite recently (on Jigsaw-dev I think) was that it is useful to acquire not the service instance, but the class name, so that it could be constructed in various ways. Services as they are now require another front-end class to encapsulate this kind of variation. Also it is not uncommon for a situation to arise where many classes must be produced for variant services which all have the same basic behavior but differ only in details, which is awkward as well as it can result in large quantities of similar classes. It would be nice if service loading and resource loading continued to follow a similar set of rules. When Jigsaw switched ServiceLoader from a convenience API over resources to a separate concept, it lost the concept of using resources *as* an API component or participant, which is a powerful feature. This change will restore that to some degree, but it is still strange that loading a META-INF/services/com.foo.Service file will have different search semantics than using ServiceLoader. Basically we are taking this capability from users unless they do it the way that Jigsaw wants you to: via ServiceLoader, and no other way; it almost seems like Jigsaw is attempting to take on the characteristics of a container (not a very powerful one though because it only supports linking via services (a pretty thin concept when you think about it), not by (for example) annotations or anything else). Anyway I hope you can extract some kind of cogency from all that. > - Various popular tools and byte-code manipulation frameworks expect to > be able to load class files as resources, i.e., to access the class > file that defines a class `p.q.C` by invoking `getResource("p.q.C")` > on that class's loader. > > With the API as it stands today these use cases can still be supported, > but the code that looks up the resource must be converted to use the > `Module::getResourceAsStream` method. This is a barrier to migration, > since it means that fewer existing frameworks will work out-of-the-box > as automatic modules. > > We should not drop agreed requirements lightly, but now that we have > some practical experience with an implementation of this one I think > it's clear that the pain outweighs the gain. Overall I think this is a step in the right direction, but I think the discussion needs to carry on a bit longer. -- - DML From scolebourne at joda.org Wed Jun 29 13:34:36 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 29 Jun 2016 14:34:36 +0100 Subject: Proposal: #CompileTimeDependences: `requires static` In-Reply-To: <57730904.83ac1c0a.ae446.214aSMTPIN_ADDED_BROKEN@mx.google.com> References: <20160628211715.84610B59BF@eggemoggin.niobe.net> <1808187852.884711.1467154252074.JavaMail.zimbra@u-pem.fr> <57730904.83ac1c0a.ae446.214aSMTPIN_ADDED_BROKEN@mx.google.com> Message-ID: Thank you for this proposal, which I believe satisfies the necessary semantics. As others have noted, the words used in the module-info file are not especially clear and serve to obscure what is going on. I think perhaps there should be a wider re-evaluation of the keywords. module A { requires B; requires static C; requires static public D; exports dynamic E; } The basic concept that I see is "A depends on B" so "depends" or "dependson" would be better than "requires". module A { depends B; depends optionally C; depends optionally export D; exports runtime E; } or module A { depends B; depends compiletime C; depends compiletime export D; exports runtime E; } or more radically: module A { depends B; compiletime { depends C; depends publish D; } runtime { exports E; } } FWIW the proposed use of "optional" by a number of people is consistent with Maven, where "optional" means mandatory at compiletime and optional therafter. Stephen On 29 June 2016 at 00:31, wrote: > 2016/6/28 15:50:52 -0700, Remi Forax : >> Apart the fact that 'static' should be spelled 'optional', there is no >> reason to reuse static as it doesn't convey the semantics we want, >> i.e. optional at runtime, i fully agree with this proposal. > > In this context `static` is intended to mean "at compile time" or, > equivalently, before the more "dynamic" phases of run time and link > time (the latter of which is more like run time than compile time). > > I agree that `static` is potentially confusing since its meaning here > is different from what `static` means when used on a member of a class. > > It does, however, fit nicely with its (nearly) dual directive, `exports > dynamic`, proposed for #ReflectiveAccessToNonExportedTypes. > > I think `optional` is a non-starter, since `requires optional` reads as > an oxymoron, and it's optional at run time but mandatory at compile > time, so in what sense is it, really, "optional"? > > Suggestions of other alternatives are welcome ... > > - Mark From pbenedict at apache.org Wed Jun 29 14:42:48 2016 From: pbenedict at apache.org (Paul Benedict) Date: Wed, 29 Jun 2016 09:42:48 -0500 Subject: Proposal: #CompileTimeDependences: `requires static` In-Reply-To: References: <20160628211715.84610B59BF@eggemoggin.niobe.net> <1808187852.884711.1467154252074.JavaMail.zimbra@u-pem.fr> <57730904.83ac1c0a.ae446.214aSMTPIN_ADDED_BROKEN@mx.google.com> Message-ID: I also believe the keywords "static" and "dynamic" are not a good fit here. I commend and sympathize with your attempt, Mark, but it is too much of a mental stretch to figure out what they are supposed to mean. At first it looks like "requires static" is analogous to "import static" -- (that's where the mind goes) but I knew that couldn't be the case.... What I propose is to treat "requires" as you would treat a class or method -- with access modifiers, so to speak. What are the modifiers? They should align to how the SOTMS speaks of the environment: compile and runtime. Thus, I think the keywords should be "compile" (javac) and "runtime" (java). That's just sensible, I believe. The implied/default is both and don't need to be specified. But the developer could explicitly list them out if he/she wanted to: module joda.beans { requires compile joda.collect; ... } I do not like "optional" -- it's not descriptive enough for what meaning it should convey. Optional when? The question gets begged; thus a person is forced to answer, again, with compile or runtime. Hence "compile" and "runtime" seem most appropriate. And if for whatever reason Java needs a third or fourth or etc. environment qualifier, the road is open to add more. Cheers, Paul On Wed, Jun 29, 2016 at 8:34 AM, Stephen Colebourne wrote: > Thank you for this proposal, which I believe satisfies the necessary > semantics. > > As others have noted, the words used in the module-info file are not > especially clear and serve to obscure what is going on. I think > perhaps there should be a wider re-evaluation of the keywords. > > module A { > requires B; > requires static C; > requires static public D; > exports dynamic E; > } > > The basic concept that I see is "A depends on B" so "depends" or > "dependson" would be better than "requires". > > module A { > depends B; > depends optionally C; > depends optionally export D; > exports runtime E; > } > > or > > module A { > depends B; > depends compiletime C; > depends compiletime export D; > exports runtime E; > } > > or more radically: > > module A { > depends B; > > compiletime { > depends C; > depends publish D; > } > runtime { > exports E; > } > } > > FWIW the proposed use of "optional" by a number of people is > consistent with Maven, where "optional" means mandatory at compiletime > and optional therafter. > > Stephen > > > On 29 June 2016 at 00:31, wrote: > > 2016/6/28 15:50:52 -0700, Remi Forax : > >> Apart the fact that 'static' should be spelled 'optional', there is no > >> reason to reuse static as it doesn't convey the semantics we want, > >> i.e. optional at runtime, i fully agree with this proposal. > > > > In this context `static` is intended to mean "at compile time" or, > > equivalently, before the more "dynamic" phases of run time and link > > time (the latter of which is more like run time than compile time). > > > > I agree that `static` is potentially confusing since its meaning here > > is different from what `static` means when used on a member of a class. > > > > It does, however, fit nicely with its (nearly) dual directive, `exports > > dynamic`, proposed for #ReflectiveAccessToNonExportedTypes. > > > > I think `optional` is a non-starter, since `requires optional` reads as > > an oxymoron, and it's optional at run time but mandatory at compile > > time, so in what sense is it, really, "optional"? > > > > Suggestions of other alternatives are welcome ... > > > > - Mark > From pbenedict at apache.org Wed Jun 29 14:54:38 2016 From: pbenedict at apache.org (Paul Benedict) Date: Wed, 29 Jun 2016 09:54:38 -0500 Subject: Proposal: #ResourceEncapsulation and #ClassFilesAsResources In-Reply-To: <5773CB63.6030308@redhat.com> References: <20160628212015.99A06B59CC@eggemoggin.niobe.net> <5773CB63.6030308@redhat.com> Message-ID: A reply to the "observers" list... Would a decent compromise be that resources in META-INF should be unconditionally public? I am just saying that perhaps the problem needs to be framed in a way where library producers *know* where to put their public resources. I am in no way able to say if dropping all encapsulation for resources is the right thing -- but if you wanted to save encapsulation in any form, I'd be interested in hearing opinions on what I just proposed. Hopefully allowing META-INF to be publicly readable is in line with existing SE and EE standards/specs anyway, and this would enforce compliance. Cheers, Paul On Wed, Jun 29, 2016 at 8:21 AM, David M. Lloyd wrote: > Responses inline & at bottom. > > On 06/28/2016 04:20 PM, Mark Reinhold wrote: > >> Issue summaries >> --------------- >> >> #ClassFilesAsResources --- If a type is visible and was loaded from a >> class file then it should be possible to read that file by invoking the >> `getResourceAsStream` method of the type's class loader, as it is in >> earlier releases. [1] >> >> #ResourceEncapsulation --- The `Module::getResourceAsStream` method can >> be used to read the resources of any named module, without restriction, >> which violates the resource-encapsulation requirement [2]. This method >> should be restricted somehow so that only "suitably-privileged" code >> (for some definition of that term) can access resources in a named >> module other than its own. An alternative is to drop this >> requirement. [3] >> >> Proposal >> -------- >> >> Drop the agreed resource-encapsulation requirement, which reads: >> >> Resource encapsulation --- The run-time system must ensure that the >> static resource files within a module are directly accessible only by >> code within that module. The existing resource-access APIs should >> continue to work as they do today when used to access a module's own >> resource files. [2] >> > > OK so far. > > Make the following changes to the various `getResource*` methods: >> >> - The `ClassLoader::getResource*` methods will first delegate to the >> loader's parent, if any, and will then search for resources in the >> named modules defined to the loader, and will then search in the >> loader's unnamed module, typically defined by a class path. In the >> case of the `getResources` method the order of the resources in the >> returned enumeration will be unspecified, except that the values of >> `cl.getResources(name).nextElement()` and `cl.getResource(name)` >> will be equal. >> > > I am really not a fan of unspecified resource order, especially when you > can easily derive a stable order from the graph topology. > > Resources however should always load from the child-most loader first. > This isn't a problem in Jigsaw for classes because packages cannot be > duplicated between modules (in a visible manner), but it becomes a problem > with resources, especially if/when run-time cycles are permitted: you can > run into an awkward situation where two mutually dependent modules each may > see the other's resource "first" before their own, which is almost always > undesirable. > > - The `Class::getResource*` methods will only search for resources in >> the module that defines the class. In the case of a class in an >> unnamed module this will typically result in searching the class >> path of the corresponding class loader. >> > > Does this also apply to automatic modules? > > The `java.lang.reflect.Module::getResourceAsStream` method will remain, >> so that it's possible to look up a resource in a module without having a >> reference to a class defined in that module. It will always be the case >> that, for a given a class `c`, `c.getResourceAsStream(name)` will be >> equivalent to `c.getModule().getResourceAsStream(name)`. >> > > I believe that in the former case, the "name" was historically relative to > the location of the class. Is that behavior changing? > > Rationale >> --------- >> >> The encapsulation of resources was intended to be part of the overall >> encapsulation story, on the view that a module's resources are just >> internal implementation details that need not, and should not, be >> available to code in other modules. Allowing external code to depend >> upon a module's internal resources is just as problematic as allowing >> such code to depend upon internal APIs. >> >> The reality of existing code today, however, is rather different, with >> two common use cases: >> >> - Resources are often used intentionally to convey information to >> external code, e.g., as a way to publish configuration files such as >> `persistence.xml`. This sort of thing should, ideally, be done via >> services [4][5], but that is not the current practice and it will >> take years to migrate to it. >> > > In many cases services are too limiting. It is useful for users to have a > more powerful, general facility available that can use the same basic > mechanism as services, but for arbitrary resources. For example, one case > that came up quite recently (on Jigsaw-dev I think) was that it is useful > to acquire not the service instance, but the class name, so that it could > be constructed in various ways. Services as they are now require another > front-end class to encapsulate this kind of variation. Also it is not > uncommon for a situation to arise where many classes must be produced for > variant services which all have the same basic behavior but differ only in > details, which is awkward as well as it can result in large quantities of > similar classes. > > It would be nice if service loading and resource loading continued to > follow a similar set of rules. When Jigsaw switched ServiceLoader from a > convenience API over resources to a separate concept, it lost the concept > of using resources *as* an API component or participant, which is a > powerful feature. This change will restore that to some degree, but it is > still strange that loading a META-INF/services/com.foo.Service file will > have different search semantics than using ServiceLoader. Basically we are > taking this capability from users unless they do it the way that Jigsaw > wants you to: via ServiceLoader, and no other way; it almost seems like > Jigsaw is attempting to take on the characteristics of a container (not a > very powerful one though because it only supports linking via services (a > pretty thin concept when you think about it), not by (for example) > annotations or anything else). > > Anyway I hope you can extract some kind of cogency from all that. > > - Various popular tools and byte-code manipulation frameworks expect to >> be able to load class files as resources, i.e., to access the class >> file that defines a class `p.q.C` by invoking `getResource("p.q.C")` >> on that class's loader. >> >> With the API as it stands today these use cases can still be supported, >> but the code that looks up the resource must be converted to use the >> `Module::getResourceAsStream` method. This is a barrier to migration, >> since it means that fewer existing frameworks will work out-of-the-box >> as automatic modules. >> >> We should not drop agreed requirements lightly, but now that we have >> some practical experience with an implementation of this one I think >> it's clear that the pain outweighs the gain. >> > > Overall I think this is a step in the right direction, but I think the > discussion needs to carry on a bit longer. > > -- > - DML > From pbenedict at apache.org Wed Jun 29 14:57:41 2016 From: pbenedict at apache.org (Paul Benedict) Date: Wed, 29 Jun 2016 09:57:41 -0500 Subject: Proposal: #CustomizableAutomaticModuleNameMapping In-Reply-To: References: <20160628212315.A247FB59D8@eggemoggin.niobe.net> Message-ID: Replying to the "observers" list... Mark, can you please opine how this proposed resolution relates to JDK-8160293 [1]? [1] https://bugs.openjdk.java.net/browse/JDK-8160293 Cheers, Paul On Wed, Jun 29, 2016 at 8:01 AM, Tim Ellison wrote: > mark.reinhold at oracle.com (Mark Reinhold) wrote on 28/06/2016 22:23:15: > > Issue summary > > ------------- > > > > #CustomizableAutomaticModuleNameMapping --- Provide a means to > > customize the algorithm that computes the names and version strings > > of automatic modules. [1] > > > > Proposal > > -------- > > > > Extend the `java.lang.module.ModuleDescriptor.Builder` API so that it > > can be used to create automatic as well as explicit modules. This will > > allow a container application to use a custom `ModuleFinder` to manage > > the mapping from artifact names to module names, version strings, and > > so forth. > > > > > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/ > > #CustomizableAutomaticModuleNameMapping > > Enthusiastic +1 for this proposal. I envisage a number of places from > which > such auto information may be derived. > > Regards, > Tim > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > From david.lloyd at redhat.com Wed Jun 29 15:08:40 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Wed, 29 Jun 2016 10:08:40 -0500 Subject: Proposal: #CustomizableAutomaticModuleNameMapping In-Reply-To: <20160628212315.A247FB59D8@eggemoggin.niobe.net> References: <20160628212315.A247FB59D8@eggemoggin.niobe.net> Message-ID: <5773E478.9020903@redhat.com> On 06/28/2016 04:23 PM, Mark Reinhold wrote: > Issue summary > ------------- > > #CustomizableAutomaticModuleNameMapping --- Provide a means to > customize the algorithm that computes the names and version strings > of automatic modules. [1] > > Proposal > -------- > > Extend the `java.lang.module.ModuleDescriptor.Builder` API so that it > can be used to create automatic as well as explicit modules. This will > allow a container application to use a custom `ModuleFinder` to manage > the mapping from artifact names to module names, version strings, and > so forth. Could you please clarify how, after this change, an automatic module would be different from a regular module, from the perspective of any code which is using the descriptor Builder API? -- - DML From pbenedict at apache.org Wed Jun 29 15:18:17 2016 From: pbenedict at apache.org (Paul Benedict) Date: Wed, 29 Jun 2016 10:18:17 -0500 Subject: Proposal: #ModuleAnnotations and #ModuleDeprecation In-Reply-To: References: <20160628211915.9706BB59C8@eggemoggin.niobe.net> Message-ID: Replying to the "observers" list... >> Annotations on modules will never affect the operation of the module system itself. Is that really true? If by "module system" you are excluding layer interaction, okay, but I do expect annotations to impact how modules are loaded. At the minimum, annotations may instruct the module system or layer to do extra logging. Envisioning greater impact, I also could see the use case where a layer uses annotations, such as identifying the Maven GAV, to check against the Victims CVE database and fail loading compromised modules. [1] If I remember correctly, layers do have the chance to customize the module descriptor -- which includes failing to load it with an exception, no? If not, please consider a layer's veto power as an RFE. However, regarding module annotations being invisible, making them invisible was supposed to prevent providing actionable information. So if module annotations are runtime visible, expect layers to use them to customize how modules are loaded. >> Should it be possible to annotate the individual directives in a module declaration? If you do allow this, you should have corresponding mirror elements created in javax.lang.model so that developers can inspect the element constructs. [1] https://www.owasp.org/index.php/OWASP_Dependency_Check Cheers, Paul On Wed, Jun 29, 2016 at 7:30 AM, Stephen Colebourne wrote: > Like Remi and David, I support the proposal, but feel that annotations > should be available at runtime. Not doing so would be very surprising > to end users, and would be an "unpleasant asymmetry in the language". > > While I can see the case for annotations within the module-info file > (Stephane makes a good case), I'm OK with omitting it. > > Stephen > > > On 28 June 2016 at 22:19, Mark Reinhold wrote: > > Issue summaries > > --------------- > > > > #ModuleAnnotations --- Should it be possible to annotate module > > declarations? As previously discussed this is technically feasible > > but it would add significant complexity. [1] > > > > #ModuleDeprecation -- If module declarations cannot be annotated, > > should there be some other way to deprecate modules? [2] > > > > Proposal > > -------- > > > > Allow arbitrary annotations to be placed on module declarations, and > > revise the `java.lang.Deprecated` annotation so that it can be applied > > to modules. > > > > - Add a new value, `MODULE`, to `java.lang.annotation.ElementType`, > > to denote the top level of a module declaration. > > > > - Extend `javax.lang.model` and `java.lang.reflect` to provide access > > to module annotations. Repeating annotations are supported, subject > > to the usual rules for repeatable annotation types. (This proposal > > will not add similar support to `java.lang.module.ModuleDescriptor`; > > that would add yet more complexity, and we haven't seen a use case > > for it.) > > > > - Annotations on modules will never affect the operation of the module > > system itself. > > > > - Revise `java.lang.Deprecated` and `java.lang.SuppressWarnings` to > > allow `ElementType.MODULE` as a target. For now, no other existing > > Java SE annotations will apply to modules. > > > > Also, allow `import` declarations [3] to be used in module compilation > > units. > > > > - Any `import` declarations must appear before the `module` > declaration. > > > > - As a consequence of supporting `import` declarations, simple type > > names can now be used in the `uses` and `provides` directives of a > > module declaration. Type annotations on these type names are not > > supported. > > > > Rationale > > --------- > > > > When we first discussed this topic I wrote [4]: > > > >> We haven't seen a compelling use case (so far) for annotations on module > >> declarations, or on clauses in module declarations, and they aren't a > >> documented requirement, so for simplicity we left them out of the > initial > >> design and the prototype. > > > > The compelling use case is, of course, deprecation, as R?mi pointed out > > [5]. In Java SE 9, in particular, we very much want to be able to write > > something like this: > > > > @Deprecated(since = "9", forRemoval = true) > > module java.corba { > > ... > > } > > > > The most natural way to do this is to allow module declarations to be > > annotated in the same way as other top-level declarations (i.e., packages > > and classes). If we don't allow the `@Deprecated` annotation to be used > > on modules then we'd have to invent some other way to deprecate modules > > (another new keyword?), and that would just be confusing. > > > > Like most language features, annotations have sometimes been over-used > > and even abused [6]. Allowing modules to be annotated opens up a whole > > new avenue for unfortunate design decisions that lead to unreadable code. > > We could try to limit the damage by allowing just a few annotations on > > module declarations (e.g., only those in the `java.lang` package), or by > > not supporting `import` declarations so that the use of non-platform > > annotations is discouragingly verbose. Such an approach would lead to > > unpleasant asymmetries in the language, however, so instead we propose to > > support arbitrary annotations on module declarations in the obvious way. > > > > I think the guidance I wrote earlier [4] still holds. Annotations are > > useful for extra-linguistic statements, such as deprecation, that make > > sense in source code and can usefully be carried through the tool chain > > into the resulting class file. I still don't think they're appropriate > > for information that is more appropriately the domain of build systems, > > e.g., version strings. > > > > Notes > > ----- > > > > - Should it be possible to annotate the individual directives in a > > module declaration? This would allow you to deprecate individual > > exports and requires-public directives, which could be useful in the > > process of evolving an API: If you plan to remove a package, or > > remove a public dependence upon some other module, then you could > > deprecate the corresponding `exports` or `requires public` directive > > (presumably with `forRemoval = true`) to warn your users that the > > API is going to change. > > > > There are no cases in the JDK itself where we need this ability, at least > > not desperately. Designing and implementing this would be a non-trivial > > exercise, and it is a separable feature that could be added later on if > > justified by experience. > > > > > > [1] > http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleAnnotations > > [2] > http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleDeprecation > > [3] http://docs.oracle.com/javase/specs/jls/se8/html/jls-7.html#jls-7.5 > > [4] > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-October/000153.html > > [5] > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-March/000263.html > > [6] http://annotatiomania.com/ > From pbenedict at apache.org Wed Jun 29 18:13:42 2016 From: pbenedict at apache.org (Paul Benedict) Date: Wed, 29 Jun 2016 13:13:42 -0500 Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` In-Reply-To: <1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr> References: <20160628211815.87E88B59C4@eggemoggin.niobe.net> <1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr> Message-ID: Replying to "observers" list... I think the main problem here is that core reflection has become too restrictive. Reflection should be "magical" and have no knowledge of any kind of module and/or visibility boundaries. If you really want to control what can be reflected upon, that should be done via the Policy file of the Security Manager. This proposal is trying to hack out-of-the-box created by too strict of a design. It is very awkward to denote which packages must be reflectable ahead of time --- how can you figure that out?? The consumers who are doing the reflection are innumerable. I'd go as far to say it's it's impossible to devise a preemptive comprehensive strategy that may satisfy the kinds of tooling in the wild. It's best just to open up reflection so it can continue doing what it always has done well: let me see anything I want unless the Security Manager says no. Cheers, Paul On Wed, Jun 29, 2016 at 3:51 AM, Remi Forax wrote: > > > ----- Mail original ----- > > De: "Mark Reinhold" > > ?: jpms-spec-experts at openjdk.java.net > > Envoy?: Mardi 28 Juin 2016 23:18:15 > > Objet: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` > > > > Issue summary > > ------------- > > > > #ReflectiveAccessToNonExportedTypes --- Some kinds of framework > > libraries require reflective access to members of the non-exported > > types of other modules; examples include dependency injection (Guice), > > persistence (JPA), debugging tools, code-automation tools, and > > serialization (XStream). In some cases the particular library to be > > used is not known until run time (e.g., Hibernate and EclipseLink both > > implement JPA). This capability is also sometimes used to work around > > bugs in unchangeable code. Access to non-exported packages can, at > > present, only be done via command-line flags, which is extremely > > awkward. Provide an easier way for reflective code to access such > > non-exported types. [1] > > > > Proposal > > -------- > > > > Extend the language of module declarations so that a package can be > > declared to be exported at run time but not at compile time. This is, > > roughly, the dual of the `requires static` construct proposed for > > #CompileTimeDependences, hence we propose to introduce a new modifier, > > `dynamic`, for use on the `exports` directive. It has the following > > meanings: > > > > - At compile time, `exports dynamic P` does not cause the package `P` > > to be exported, though it does require `P` to be a package defined > > in the module. > > > > - In phases after compile time, `exports dynamic P` behaves in exactly > > the same way as `exports P`. It therefore takes part fully in > > resolution and configuration, and is subject to the same consistency > > constraints as normally-exported packages (e.g., no split packages). > > > > Thus a module declaration of the form > > > > module com.foo.app { > > requires hibernate.core; > > requires hibernate.entitymanager; > > exports dynamic com.foo.app.model; > > } > > > > makes the types in the `com.foo.app.model` package accessible at run time > > but not at compile time. In combination with the earlier change to > > enable #ReflectionWithoutReadability [2] this means that frameworks that > > today use core reflection to manipulate user classes at run time are more > > likely to work out-of-the-box, without change, as automatic modules. If > > the `com.foo.app.model` package in this example includes entity classes > > to be managed by Hibernate then the framework will be able to access them > > without further ado, but under normal circumstances an attempt to compile > > code that refers directly to those classes will fail. > > > > The `dynamic` modifier can be applied to both unqualified and qualified > > `exports` directives, though the caveats on using qualified exports [3] > > still apply. > > > > Notes > > ----- > > > > - To access a non-public member in a dynamically-exported package you > > must still invoke the appropriate `setAccessible` method, just as > > you do today to access non-public members in an exported package. > > > > - This proposal is similar to R?mi's suggestion to add a way to export > > a package only for reflection, still requiring that `setAccessible` > > be invoked [4]. This proposal differs in that it does not require > > `setAccessible` to be invoked to access public members in a > > dynamically-exported package. > > > > - This proposal primarily addresses "friendly" uses of reflection, such > > as dependency injection and persistence, in which the author of a > > module knows in advance that one or more packages must be exported at > > run time for reflective access by frameworks. This proposal is also > > convenient for frameworks that generate code at run time, since the > > constant pool of a generated class file can include static references > > to types in dynamically exported packages. > > > > - This proposal opens the door to a category of second-class APIs. If > > a package is exported dynamically then you can still compile code > > that refers to types in the package, by using `-XaddExports` or its > > equivalent at compile time, and it will work as expected at run time. > > It thus may be useful to use `exports dynamic` for packages that > > contain legacy APIs whose use is strongly discouraged, e.g., those in > > the `jdk.unsupported` module of the reference implementation, thereby > > forcing anyone who wants to compile against them to go to the trouble > > of using `-XaddExports`. > > > > - Intrusive access to arbitrary packages of arbitrary modules by, e.g., > > debugging tools, will still require the use of sharp knives such as > > the `-XaddExports` command-line option or its equivalent, or JVM TI. > > > > - Using the `-XaddExports` option or its equivalent remains awkward, > > and sometimes it's the only way out. To ease migration I think it's > > worth considering some way for an application packaged as a JAR file > > to include such options in its `MANIFEST.MF` file, as suggested by > > Simon Nash [5]. I'll create a separate issue for that. > > > > - New kinds of resolution failures are possible. If module `A` > > requires `B`, and `B` requires `C`, then if `B` and `C` both declare > > some package `P` to be exported dynamically, then a split-package > > error will be reported. This may surprise, since the packages are > > just internal implementation details that were exported dynamically > > so that some framework could access their types. This is not, > > however, all that different from the kinds of resolution and > > layer-creation failures that are already possible due to package > > collisions. It's unlikely to happen all that often in practice, > > and the fix is straightforward: Just use reverse-DNS or otherwise > > sufficiently-unique package names, as most people already do. It > > is worth exploring whether javac can detect at least some of these > > kinds of collisions and issue appropriate warnings. > > I would like to agree on that proposal only on the ground that it requires > to change the classfile format thus allows in the same time to fix the > issues of the current classfile spec (encoding of the name module as a > Class instead of UTF8, wrong flag for requires public), but i'm not sure > this proposal support it's own weight. > > Without dynamic, a 'classical' export works the same way at runtime so > adding 'dynamic' to an export only restrict the usage at compile time. > Given that we can already selectively export packages to some chosen > modules which hide them for the rest of the other modules at compile time, > adding dynamic to the spec adds little value. > > And it has several drawbacks, the main one is that it weaken the > compiletime/runtime fidelity story, with a dynamic export the runtime world > is not the compiletime world anymore. > The main motivation to add dynamic seems to be for backward compatibility > during the transition between the classpath world and the full module world, > for several other cases, we use the command line for that, i don't see why > this case is so special that it should not work like the other > compatibility issues. > > Moreover, an agent, an annotation processor and a jlink plugin are all > able to implement the same trick, add export after the compilation and > before the runtime, > so i don't think it's a good idea to change the spec for that. > > R?mi > > > > > > > [1] > > > http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccessToNonExportedTypes > > [2] > > > http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithoutReadability > > [3] > http://openjdk.java.net/projects/jigsaw/spec/sotms/#qualified-exports > > [4] > > > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-December/000205.html > > [5] > > > http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005745.html > > > From rafael.wth at gmail.com Wed Jun 29 20:28:07 2016 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Wed, 29 Jun 2016 22:28:07 +0200 Subject: Proposal: #ResourceEncapsulation and #ClassFilesAsResources In-Reply-To: References: <20160628212015.99A06B59CC@eggemoggin.niobe.net> <5773CB63.6030308@redhat.com> Message-ID: Paul, this would still break millions of applications as viewing resources is a crucial assumption of many libraries. For example, Apache Wicket requires to read HTML pages from the folder (package) of the class that defines the dispatcher of this web page. And shadoing the view would neither allow to obtain a class's byte code what is common practice today. I currently contract for an APM vendor and my job is basically to write plugins for all sorts of Java frameworks and libraries which is why I think I have a pretty good overview of how this would affect a lot of code. One reason I was so explicit about this issue being problematic is that it would really break a wide range of applications. In this context, I think Mark's suggestion is the best possible idea; I do neither see a problem with the randomness of the resources as even today, it depends on the order that jar files are added to the class path which is by no means stable. As for named modules, package names should be exclusive anyways and for the unnamed module, the behavior does not change. I would still like to see an introduction of some form of ResourceLocatable interface that defines two methods: interface ResourceLocatable { URL getResource(String); InputStream getResourceAsStream(String); } This would add some structure to the API. As for the different lookup contexts I think the made suggestion is sound, too: 1. ClassLoader: Exposes any resource reachable by this class loader, therefore an Enumeration getResources(String) method makes sense, too. 2. Module: Only exposes resources within this module from an absolute location. 3. Class: Only exposes resources within the class's module from a relative location. Great work on this, I am really glad this proposal was made in this way. Cheers, Rafael 2016-06-29 16:54 GMT+02:00 Paul Benedict : > A reply to the "observers" list... > > Would a decent compromise be that resources in META-INF should be > unconditionally public? I am just saying that perhaps the problem needs to > be framed in a way where library producers *know* where to put their public > resources. I am in no way able to say if dropping all encapsulation for > resources is the right thing -- but if you wanted to save encapsulation in > any form, I'd be interested in hearing opinions on what I just proposed. > Hopefully allowing META-INF to be publicly readable is in line with > existing SE and EE standards/specs anyway, and this would enforce > compliance. > > Cheers, > Paul > > On Wed, Jun 29, 2016 at 8:21 AM, David M. Lloyd > wrote: > > > Responses inline & at bottom. > > > > On 06/28/2016 04:20 PM, Mark Reinhold wrote: > > > >> Issue summaries > >> --------------- > >> > >> #ClassFilesAsResources --- If a type is visible and was loaded from a > >> class file then it should be possible to read that file by invoking > the > >> `getResourceAsStream` method of the type's class loader, as it is in > >> earlier releases. [1] > >> > >> #ResourceEncapsulation --- The `Module::getResourceAsStream` method > can > >> be used to read the resources of any named module, without > restriction, > >> which violates the resource-encapsulation requirement [2]. This > method > >> should be restricted somehow so that only "suitably-privileged" code > >> (for some definition of that term) can access resources in a named > >> module other than its own. An alternative is to drop this > >> requirement. [3] > >> > >> Proposal > >> -------- > >> > >> Drop the agreed resource-encapsulation requirement, which reads: > >> > >> Resource encapsulation --- The run-time system must ensure that the > >> static resource files within a module are directly accessible only by > >> code within that module. The existing resource-access APIs should > >> continue to work as they do today when used to access a module's own > >> resource files. [2] > >> > > > > OK so far. > > > > Make the following changes to the various `getResource*` methods: > >> > >> - The `ClassLoader::getResource*` methods will first delegate to the > >> loader's parent, if any, and will then search for resources in the > >> named modules defined to the loader, and will then search in the > >> loader's unnamed module, typically defined by a class path. In the > >> case of the `getResources` method the order of the resources in the > >> returned enumeration will be unspecified, except that the values of > >> `cl.getResources(name).nextElement()` and `cl.getResource(name)` > >> will be equal. > >> > > > > I am really not a fan of unspecified resource order, especially when you > > can easily derive a stable order from the graph topology. > > > > Resources however should always load from the child-most loader first. > > This isn't a problem in Jigsaw for classes because packages cannot be > > duplicated between modules (in a visible manner), but it becomes a > problem > > with resources, especially if/when run-time cycles are permitted: you can > > run into an awkward situation where two mutually dependent modules each > may > > see the other's resource "first" before their own, which is almost always > > undesirable. > > > > - The `Class::getResource*` methods will only search for resources in > >> the module that defines the class. In the case of a class in an > >> unnamed module this will typically result in searching the class > >> path of the corresponding class loader. > >> > > > > Does this also apply to automatic modules? > > > > The `java.lang.reflect.Module::getResourceAsStream` method will remain, > >> so that it's possible to look up a resource in a module without having a > >> reference to a class defined in that module. It will always be the case > >> that, for a given a class `c`, `c.getResourceAsStream(name)` will be > >> equivalent to `c.getModule().getResourceAsStream(name)`. > >> > > > > I believe that in the former case, the "name" was historically relative > to > > the location of the class. Is that behavior changing? > > > > Rationale > >> --------- > >> > >> The encapsulation of resources was intended to be part of the overall > >> encapsulation story, on the view that a module's resources are just > >> internal implementation details that need not, and should not, be > >> available to code in other modules. Allowing external code to depend > >> upon a module's internal resources is just as problematic as allowing > >> such code to depend upon internal APIs. > >> > >> The reality of existing code today, however, is rather different, with > >> two common use cases: > >> > >> - Resources are often used intentionally to convey information to > >> external code, e.g., as a way to publish configuration files such > as > >> `persistence.xml`. This sort of thing should, ideally, be done via > >> services [4][5], but that is not the current practice and it will > >> take years to migrate to it. > >> > > > > In many cases services are too limiting. It is useful for users to have > a > > more powerful, general facility available that can use the same basic > > mechanism as services, but for arbitrary resources. For example, one > case > > that came up quite recently (on Jigsaw-dev I think) was that it is useful > > to acquire not the service instance, but the class name, so that it could > > be constructed in various ways. Services as they are now require another > > front-end class to encapsulate this kind of variation. Also it is not > > uncommon for a situation to arise where many classes must be produced for > > variant services which all have the same basic behavior but differ only > in > > details, which is awkward as well as it can result in large quantities of > > similar classes. > > > > It would be nice if service loading and resource loading continued to > > follow a similar set of rules. When Jigsaw switched ServiceLoader from a > > convenience API over resources to a separate concept, it lost the concept > > of using resources *as* an API component or participant, which is a > > powerful feature. This change will restore that to some degree, but it > is > > still strange that loading a META-INF/services/com.foo.Service file will > > have different search semantics than using ServiceLoader. Basically we > are > > taking this capability from users unless they do it the way that Jigsaw > > wants you to: via ServiceLoader, and no other way; it almost seems like > > Jigsaw is attempting to take on the characteristics of a container (not a > > very powerful one though because it only supports linking via services (a > > pretty thin concept when you think about it), not by (for example) > > annotations or anything else). > > > > Anyway I hope you can extract some kind of cogency from all that. > > > > - Various popular tools and byte-code manipulation frameworks expect > to > >> be able to load class files as resources, i.e., to access the class > >> file that defines a class `p.q.C` by invoking > `getResource("p.q.C")` > >> on that class's loader. > >> > >> With the API as it stands today these use cases can still be supported, > >> but the code that looks up the resource must be converted to use the > >> `Module::getResourceAsStream` method. This is a barrier to migration, > >> since it means that fewer existing frameworks will work out-of-the-box > >> as automatic modules. > >> > >> We should not drop agreed requirements lightly, but now that we have > >> some practical experience with an implementation of this one I think > >> it's clear that the pain outweighs the gain. > >> > > > > Overall I think this is a step in the right direction, but I think the > > discussion needs to carry on a bit longer. > > > > -- > > - DML > > > From pbenedict at apache.org Wed Jun 29 20:53:31 2016 From: pbenedict at apache.org (Paul Benedict) Date: Wed, 29 Jun 2016 15:53:31 -0500 Subject: Proposal: #ResourceEncapsulation and #ClassFilesAsResources In-Reply-To: References: <20160628212015.99A06B59CC@eggemoggin.niobe.net> <5773CB63.6030308@redhat.com> Message-ID: Rafael, you are right my suggestion would break such applications. You couldn't automatically use them as a Java 9 module. You would have to move those resources into META-INF. With that said, what about private resources? This is not a matter of reflection, which is where I advocate "jailbreaking" to see all types no matter what module. I consider getResource() to be a normal API that should respect module boundaries. If I want to provide a file with a secret key in my module, must it be exposed to everyone? I say no, it shouldn't. That's why I think encapsulation should not be dropped in this case. Cheers, Paul On Wed, Jun 29, 2016 at 3:28 PM, Rafael Winterhalter wrote: > Paul, this would still break millions of applications as viewing resources > is a crucial assumption of many libraries. For example, Apache Wicket > requires to read HTML pages from the folder (package) of the class that > defines the dispatcher of this web page. And shadoing the view would > neither allow to obtain a class's byte code what is common practice today. > I currently contract for an APM vendor and my job is basically to write > plugins for all sorts of Java frameworks and libraries which is why I think > I have a pretty good overview of how this would affect a lot of code. One > reason I was so explicit about this issue being problematic is that it > would really break a wide range of applications. > > In this context, I think Mark's suggestion is the best possible idea; I do > neither see a problem with the randomness of the resources as even today, > it depends on the order that jar files are added to the class path which is > by no means stable. As for named modules, package names should be exclusive > anyways and for the unnamed module, the behavior does not change. > > I would still like to see an introduction of some form of ResourceLocatable > interface that defines two methods: > > interface ResourceLocatable { > URL getResource(String); > InputStream getResourceAsStream(String); > } > > This would add some structure to the API. As for the different lookup > contexts I think the made suggestion is sound, too: > 1. ClassLoader: Exposes any resource reachable by this class loader, > therefore an Enumeration getResources(String) method makes sense, too. > 2. Module: Only exposes resources within this module from an absolute > location. > 3. Class: Only exposes resources within the class's module from a relative > location. > > Great work on this, I am really glad this proposal was made in this way. > Cheers, Rafael > > 2016-06-29 16:54 GMT+02:00 Paul Benedict : > > > A reply to the "observers" list... > > > > Would a decent compromise be that resources in META-INF should be > > unconditionally public? I am just saying that perhaps the problem needs > to > > be framed in a way where library producers *know* where to put their > public > > resources. I am in no way able to say if dropping all encapsulation for > > resources is the right thing -- but if you wanted to save encapsulation > in > > any form, I'd be interested in hearing opinions on what I just proposed. > > Hopefully allowing META-INF to be publicly readable is in line with > > existing SE and EE standards/specs anyway, and this would enforce > > compliance. > > > > Cheers, > > Paul > > > > On Wed, Jun 29, 2016 at 8:21 AM, David M. Lloyd > > wrote: > > > > > Responses inline & at bottom. > > > > > > On 06/28/2016 04:20 PM, Mark Reinhold wrote: > > > > > >> Issue summaries > > >> --------------- > > >> > > >> #ClassFilesAsResources --- If a type is visible and was loaded > from a > > >> class file then it should be possible to read that file by invoking > > the > > >> `getResourceAsStream` method of the type's class loader, as it is > in > > >> earlier releases. [1] > > >> > > >> #ResourceEncapsulation --- The `Module::getResourceAsStream` method > > can > > >> be used to read the resources of any named module, without > > restriction, > > >> which violates the resource-encapsulation requirement [2]. This > > method > > >> should be restricted somehow so that only "suitably-privileged" > code > > >> (for some definition of that term) can access resources in a named > > >> module other than its own. An alternative is to drop this > > >> requirement. [3] > > >> > > >> Proposal > > >> -------- > > >> > > >> Drop the agreed resource-encapsulation requirement, which reads: > > >> > > >> Resource encapsulation --- The run-time system must ensure that the > > >> static resource files within a module are directly accessible only > by > > >> code within that module. The existing resource-access APIs should > > >> continue to work as they do today when used to access a module's > own > > >> resource files. [2] > > >> > > > > > > OK so far. > > > > > > Make the following changes to the various `getResource*` methods: > > >> > > >> - The `ClassLoader::getResource*` methods will first delegate to > the > > >> loader's parent, if any, and will then search for resources in > the > > >> named modules defined to the loader, and will then search in the > > >> loader's unnamed module, typically defined by a class path. In > the > > >> case of the `getResources` method the order of the resources in > the > > >> returned enumeration will be unspecified, except that the values > of > > >> `cl.getResources(name).nextElement()` and `cl.getResource(name)` > > >> will be equal. > > >> > > > > > > I am really not a fan of unspecified resource order, especially when > you > > > can easily derive a stable order from the graph topology. > > > > > > Resources however should always load from the child-most loader first. > > > This isn't a problem in Jigsaw for classes because packages cannot be > > > duplicated between modules (in a visible manner), but it becomes a > > problem > > > with resources, especially if/when run-time cycles are permitted: you > can > > > run into an awkward situation where two mutually dependent modules each > > may > > > see the other's resource "first" before their own, which is almost > always > > > undesirable. > > > > > > - The `Class::getResource*` methods will only search for resources > in > > >> the module that defines the class. In the case of a class in an > > >> unnamed module this will typically result in searching the class > > >> path of the corresponding class loader. > > >> > > > > > > Does this also apply to automatic modules? > > > > > > The `java.lang.reflect.Module::getResourceAsStream` method will remain, > > >> so that it's possible to look up a resource in a module without > having a > > >> reference to a class defined in that module. It will always be the > case > > >> that, for a given a class `c`, `c.getResourceAsStream(name)` will be > > >> equivalent to `c.getModule().getResourceAsStream(name)`. > > >> > > > > > > I believe that in the former case, the "name" was historically relative > > to > > > the location of the class. Is that behavior changing? > > > > > > Rationale > > >> --------- > > >> > > >> The encapsulation of resources was intended to be part of the overall > > >> encapsulation story, on the view that a module's resources are just > > >> internal implementation details that need not, and should not, be > > >> available to code in other modules. Allowing external code to depend > > >> upon a module's internal resources is just as problematic as allowing > > >> such code to depend upon internal APIs. > > >> > > >> The reality of existing code today, however, is rather different, with > > >> two common use cases: > > >> > > >> - Resources are often used intentionally to convey information to > > >> external code, e.g., as a way to publish configuration files such > > as > > >> `persistence.xml`. This sort of thing should, ideally, be done > via > > >> services [4][5], but that is not the current practice and it will > > >> take years to migrate to it. > > >> > > > > > > In many cases services are too limiting. It is useful for users to > have > > a > > > more powerful, general facility available that can use the same basic > > > mechanism as services, but for arbitrary resources. For example, one > > case > > > that came up quite recently (on Jigsaw-dev I think) was that it is > useful > > > to acquire not the service instance, but the class name, so that it > could > > > be constructed in various ways. Services as they are now require > another > > > front-end class to encapsulate this kind of variation. Also it is not > > > uncommon for a situation to arise where many classes must be produced > for > > > variant services which all have the same basic behavior but differ only > > in > > > details, which is awkward as well as it can result in large quantities > of > > > similar classes. > > > > > > It would be nice if service loading and resource loading continued to > > > follow a similar set of rules. When Jigsaw switched ServiceLoader > from a > > > convenience API over resources to a separate concept, it lost the > concept > > > of using resources *as* an API component or participant, which is a > > > powerful feature. This change will restore that to some degree, but it > > is > > > still strange that loading a META-INF/services/com.foo.Service file > will > > > have different search semantics than using ServiceLoader. Basically we > > are > > > taking this capability from users unless they do it the way that Jigsaw > > > wants you to: via ServiceLoader, and no other way; it almost seems like > > > Jigsaw is attempting to take on the characteristics of a container > (not a > > > very powerful one though because it only supports linking via services > (a > > > pretty thin concept when you think about it), not by (for example) > > > annotations or anything else). > > > > > > Anyway I hope you can extract some kind of cogency from all that. > > > > > > - Various popular tools and byte-code manipulation frameworks expect > > to > > >> be able to load class files as resources, i.e., to access the > class > > >> file that defines a class `p.q.C` by invoking > > `getResource("p.q.C")` > > >> on that class's loader. > > >> > > >> With the API as it stands today these use cases can still be > supported, > > >> but the code that looks up the resource must be converted to use the > > >> `Module::getResourceAsStream` method. This is a barrier to migration, > > >> since it means that fewer existing frameworks will work out-of-the-box > > >> as automatic modules. > > >> > > >> We should not drop agreed requirements lightly, but now that we have > > >> some practical experience with an implementation of this one I think > > >> it's clear that the pain outweighs the gain. > > >> > > > > > > Overall I think this is a step in the right direction, but I think the > > > discussion needs to carry on a bit longer. > > > > > > -- > > > - DML > > > > > > From rafael.wth at gmail.com Wed Jun 29 21:06:42 2016 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Wed, 29 Jun 2016 23:06:42 +0200 Subject: Proposal: #ResourceEncapsulation and #ClassFilesAsResources In-Reply-To: References: <20160628212015.99A06B59CC@eggemoggin.niobe.net> <5773CB63.6030308@redhat.com> Message-ID: Paul, I agree with you as for that this is desirable and this is why the access should be modularized when locating a resource via the Class or Module instances what already provides a form of encapsulation. Obtaining a ClassLoader is already guarded by a security manager check so plugins cannot break through here once a library maintainer chooses to encapsulate a resource. Cheers, Rafael 2016-06-29 22:53 GMT+02:00 Paul Benedict : > Rafael, you are right my suggestion would break such applications. You > couldn't automatically use them as a Java 9 module. You would have to move > those resources into META-INF. > > With that said, what about private resources? This is not a matter of > reflection, which is where I advocate "jailbreaking" to see all types no > matter what module. I consider getResource() to be a normal API that should > respect module boundaries. If I want to provide a file with a secret key in > my module, must it be exposed to everyone? I say no, it shouldn't. That's > why I think encapsulation should not be dropped in this case. > > Cheers, > Paul > > On Wed, Jun 29, 2016 at 3:28 PM, Rafael Winterhalter > wrote: > >> Paul, this would still break millions of applications as viewing resources >> is a crucial assumption of many libraries. For example, Apache Wicket >> requires to read HTML pages from the folder (package) of the class that >> defines the dispatcher of this web page. And shadoing the view would >> neither allow to obtain a class's byte code what is common practice today. >> I currently contract for an APM vendor and my job is basically to write >> plugins for all sorts of Java frameworks and libraries which is why I >> think >> I have a pretty good overview of how this would affect a lot of code. One >> reason I was so explicit about this issue being problematic is that it >> would really break a wide range of applications. >> >> In this context, I think Mark's suggestion is the best possible idea; I do >> neither see a problem with the randomness of the resources as even today, >> it depends on the order that jar files are added to the class path which >> is >> by no means stable. As for named modules, package names should be >> exclusive >> anyways and for the unnamed module, the behavior does not change. >> >> I would still like to see an introduction of some form of >> ResourceLocatable >> interface that defines two methods: >> >> interface ResourceLocatable { >> URL getResource(String); >> InputStream getResourceAsStream(String); >> } >> >> This would add some structure to the API. As for the different lookup >> contexts I think the made suggestion is sound, too: >> 1. ClassLoader: Exposes any resource reachable by this class loader, >> therefore an Enumeration getResources(String) method makes sense, >> too. >> 2. Module: Only exposes resources within this module from an absolute >> location. >> 3. Class: Only exposes resources within the class's module from a relative >> location. >> >> Great work on this, I am really glad this proposal was made in this way. >> Cheers, Rafael >> >> 2016-06-29 16:54 GMT+02:00 Paul Benedict : >> >> > A reply to the "observers" list... >> > >> > Would a decent compromise be that resources in META-INF should be >> > unconditionally public? I am just saying that perhaps the problem needs >> to >> > be framed in a way where library producers *know* where to put their >> public >> > resources. I am in no way able to say if dropping all encapsulation for >> > resources is the right thing -- but if you wanted to save encapsulation >> in >> > any form, I'd be interested in hearing opinions on what I just proposed. >> > Hopefully allowing META-INF to be publicly readable is in line with >> > existing SE and EE standards/specs anyway, and this would enforce >> > compliance. >> > >> > Cheers, >> > Paul >> > >> > On Wed, Jun 29, 2016 at 8:21 AM, David M. Lloyd > > >> > wrote: >> > >> > > Responses inline & at bottom. >> > > >> > > On 06/28/2016 04:20 PM, Mark Reinhold wrote: >> > > >> > >> Issue summaries >> > >> --------------- >> > >> >> > >> #ClassFilesAsResources --- If a type is visible and was loaded >> from a >> > >> class file then it should be possible to read that file by >> invoking >> > the >> > >> `getResourceAsStream` method of the type's class loader, as it is >> in >> > >> earlier releases. [1] >> > >> >> > >> #ResourceEncapsulation --- The `Module::getResourceAsStream` >> method >> > can >> > >> be used to read the resources of any named module, without >> > restriction, >> > >> which violates the resource-encapsulation requirement [2]. This >> > method >> > >> should be restricted somehow so that only "suitably-privileged" >> code >> > >> (for some definition of that term) can access resources in a named >> > >> module other than its own. An alternative is to drop this >> > >> requirement. [3] >> > >> >> > >> Proposal >> > >> -------- >> > >> >> > >> Drop the agreed resource-encapsulation requirement, which reads: >> > >> >> > >> Resource encapsulation --- The run-time system must ensure that >> the >> > >> static resource files within a module are directly accessible >> only by >> > >> code within that module. The existing resource-access APIs should >> > >> continue to work as they do today when used to access a module's >> own >> > >> resource files. [2] >> > >> >> > > >> > > OK so far. >> > > >> > > Make the following changes to the various `getResource*` methods: >> > >> >> > >> - The `ClassLoader::getResource*` methods will first delegate to >> the >> > >> loader's parent, if any, and will then search for resources in >> the >> > >> named modules defined to the loader, and will then search in the >> > >> loader's unnamed module, typically defined by a class path. In >> the >> > >> case of the `getResources` method the order of the resources in >> the >> > >> returned enumeration will be unspecified, except that the >> values of >> > >> `cl.getResources(name).nextElement()` and `cl.getResource(name)` >> > >> will be equal. >> > >> >> > > >> > > I am really not a fan of unspecified resource order, especially when >> you >> > > can easily derive a stable order from the graph topology. >> > > >> > > Resources however should always load from the child-most loader first. >> > > This isn't a problem in Jigsaw for classes because packages cannot be >> > > duplicated between modules (in a visible manner), but it becomes a >> > problem >> > > with resources, especially if/when run-time cycles are permitted: you >> can >> > > run into an awkward situation where two mutually dependent modules >> each >> > may >> > > see the other's resource "first" before their own, which is almost >> always >> > > undesirable. >> > > >> > > - The `Class::getResource*` methods will only search for resources >> in >> > >> the module that defines the class. In the case of a class in an >> > >> unnamed module this will typically result in searching the class >> > >> path of the corresponding class loader. >> > >> >> > > >> > > Does this also apply to automatic modules? >> > > >> > > The `java.lang.reflect.Module::getResourceAsStream` method will >> remain, >> > >> so that it's possible to look up a resource in a module without >> having a >> > >> reference to a class defined in that module. It will always be the >> case >> > >> that, for a given a class `c`, `c.getResourceAsStream(name)` will be >> > >> equivalent to `c.getModule().getResourceAsStream(name)`. >> > >> >> > > >> > > I believe that in the former case, the "name" was historically >> relative >> > to >> > > the location of the class. Is that behavior changing? >> > > >> > > Rationale >> > >> --------- >> > >> >> > >> The encapsulation of resources was intended to be part of the overall >> > >> encapsulation story, on the view that a module's resources are just >> > >> internal implementation details that need not, and should not, be >> > >> available to code in other modules. Allowing external code to depend >> > >> upon a module's internal resources is just as problematic as allowing >> > >> such code to depend upon internal APIs. >> > >> >> > >> The reality of existing code today, however, is rather different, >> with >> > >> two common use cases: >> > >> >> > >> - Resources are often used intentionally to convey information to >> > >> external code, e.g., as a way to publish configuration files >> such >> > as >> > >> `persistence.xml`. This sort of thing should, ideally, be done >> via >> > >> services [4][5], but that is not the current practice and it >> will >> > >> take years to migrate to it. >> > >> >> > > >> > > In many cases services are too limiting. It is useful for users to >> have >> > a >> > > more powerful, general facility available that can use the same basic >> > > mechanism as services, but for arbitrary resources. For example, one >> > case >> > > that came up quite recently (on Jigsaw-dev I think) was that it is >> useful >> > > to acquire not the service instance, but the class name, so that it >> could >> > > be constructed in various ways. Services as they are now require >> another >> > > front-end class to encapsulate this kind of variation. Also it is not >> > > uncommon for a situation to arise where many classes must be produced >> for >> > > variant services which all have the same basic behavior but differ >> only >> > in >> > > details, which is awkward as well as it can result in large >> quantities of >> > > similar classes. >> > > >> > > It would be nice if service loading and resource loading continued to >> > > follow a similar set of rules. When Jigsaw switched ServiceLoader >> from a >> > > convenience API over resources to a separate concept, it lost the >> concept >> > > of using resources *as* an API component or participant, which is a >> > > powerful feature. This change will restore that to some degree, but >> it >> > is >> > > still strange that loading a META-INF/services/com.foo.Service file >> will >> > > have different search semantics than using ServiceLoader. Basically we >> > are >> > > taking this capability from users unless they do it the way that >> Jigsaw >> > > wants you to: via ServiceLoader, and no other way; it almost seems >> like >> > > Jigsaw is attempting to take on the characteristics of a container >> (not a >> > > very powerful one though because it only supports linking via >> services (a >> > > pretty thin concept when you think about it), not by (for example) >> > > annotations or anything else). >> > > >> > > Anyway I hope you can extract some kind of cogency from all that. >> > > >> > > - Various popular tools and byte-code manipulation frameworks >> expect >> > to >> > >> be able to load class files as resources, i.e., to access the >> class >> > >> file that defines a class `p.q.C` by invoking >> > `getResource("p.q.C")` >> > >> on that class's loader. >> > >> >> > >> With the API as it stands today these use cases can still be >> supported, >> > >> but the code that looks up the resource must be converted to use the >> > >> `Module::getResourceAsStream` method. This is a barrier to >> migration, >> > >> since it means that fewer existing frameworks will work >> out-of-the-box >> > >> as automatic modules. >> > >> >> > >> We should not drop agreed requirements lightly, but now that we have >> > >> some practical experience with an implementation of this one I think >> > >> it's clear that the pain outweighs the gain. >> > >> >> > > >> > > Overall I think this is a step in the right direction, but I think the >> > > discussion needs to carry on a bit longer. >> > > >> > > -- >> > > - DML >> > > >> > >> > > From rafael.wth at gmail.com Wed Jun 29 23:23:55 2016 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Thu, 30 Jun 2016 01:23:55 +0200 Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` In-Reply-To: References: <20160628211815.87E88B59C4@eggemoggin.niobe.net> <1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr> Message-ID: I also agree here that the use of reflection should not require any form of explicit permission by the user of a library. In a way, this ties an implementation detail (reflection in this case) to the usage of the library. The library cannot work without the explicit configuration by a user and this only adds boilerplate to using Hibernate. Unfortunately, many Java developers still do not know much about reflection or about the inner workings of many libraries in the market. If a user needed to explicitly export packages, this might just lead to people "over exporting" their packages just to "make Hibernate work". In my opinion, reflection should not apply any module-boundary checks. Best regards, Rafael 2016-06-29 20:13 GMT+02:00 Paul Benedict : > Replying to "observers" list... > > I think the main problem here is that core reflection has become too > restrictive. Reflection should be "magical" and have no knowledge of any > kind of module and/or visibility boundaries. If you really want to control > what can be reflected upon, that should be done via the Policy file of the > Security Manager. > > This proposal is trying to hack out-of-the-box created by too strict of a > design. It is very awkward to denote which packages must be reflectable > ahead of time --- how can you figure that out?? The consumers who are doing > the reflection are innumerable. I'd go as far to say it's it's impossible > to devise a preemptive comprehensive strategy that may satisfy the kinds of > tooling in the wild. > > It's best just to open up reflection so it can continue doing what it > always has done well: let me see anything I want unless the Security > Manager says no. > > Cheers, > Paul > > On Wed, Jun 29, 2016 at 3:51 AM, Remi Forax wrote: > > > > > > > ----- Mail original ----- > > > De: "Mark Reinhold" > > > ?: jpms-spec-experts at openjdk.java.net > > > Envoy?: Mardi 28 Juin 2016 23:18:15 > > > Objet: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` > > > > > > Issue summary > > > ------------- > > > > > > #ReflectiveAccessToNonExportedTypes --- Some kinds of framework > > > libraries require reflective access to members of the non-exported > > > types of other modules; examples include dependency injection > (Guice), > > > persistence (JPA), debugging tools, code-automation tools, and > > > serialization (XStream). In some cases the particular library to be > > > used is not known until run time (e.g., Hibernate and EclipseLink > both > > > implement JPA). This capability is also sometimes used to work > around > > > bugs in unchangeable code. Access to non-exported packages can, at > > > present, only be done via command-line flags, which is extremely > > > awkward. Provide an easier way for reflective code to access such > > > non-exported types. [1] > > > > > > Proposal > > > -------- > > > > > > Extend the language of module declarations so that a package can be > > > declared to be exported at run time but not at compile time. This is, > > > roughly, the dual of the `requires static` construct proposed for > > > #CompileTimeDependences, hence we propose to introduce a new modifier, > > > `dynamic`, for use on the `exports` directive. It has the following > > > meanings: > > > > > > - At compile time, `exports dynamic P` does not cause the package `P` > > > to be exported, though it does require `P` to be a package defined > > > in the module. > > > > > > - In phases after compile time, `exports dynamic P` behaves in > exactly > > > the same way as `exports P`. It therefore takes part fully in > > > resolution and configuration, and is subject to the same > consistency > > > constraints as normally-exported packages (e.g., no split > packages). > > > > > > Thus a module declaration of the form > > > > > > module com.foo.app { > > > requires hibernate.core; > > > requires hibernate.entitymanager; > > > exports dynamic com.foo.app.model; > > > } > > > > > > makes the types in the `com.foo.app.model` package accessible at run > time > > > but not at compile time. In combination with the earlier change to > > > enable #ReflectionWithoutReadability [2] this means that frameworks > that > > > today use core reflection to manipulate user classes at run time are > more > > > likely to work out-of-the-box, without change, as automatic modules. > If > > > the `com.foo.app.model` package in this example includes entity classes > > > to be managed by Hibernate then the framework will be able to access > them > > > without further ado, but under normal circumstances an attempt to > compile > > > code that refers directly to those classes will fail. > > > > > > The `dynamic` modifier can be applied to both unqualified and qualified > > > `exports` directives, though the caveats on using qualified exports [3] > > > still apply. > > > > > > Notes > > > ----- > > > > > > - To access a non-public member in a dynamically-exported package you > > > must still invoke the appropriate `setAccessible` method, just as > > > you do today to access non-public members in an exported package. > > > > > > - This proposal is similar to R?mi's suggestion to add a way to > export > > > a package only for reflection, still requiring that `setAccessible` > > > be invoked [4]. This proposal differs in that it does not require > > > `setAccessible` to be invoked to access public members in a > > > dynamically-exported package. > > > > > > - This proposal primarily addresses "friendly" uses of reflection, > such > > > as dependency injection and persistence, in which the author of a > > > module knows in advance that one or more packages must be exported > at > > > run time for reflective access by frameworks. This proposal is > also > > > convenient for frameworks that generate code at run time, since the > > > constant pool of a generated class file can include static > references > > > to types in dynamically exported packages. > > > > > > - This proposal opens the door to a category of second-class APIs. > If > > > a package is exported dynamically then you can still compile code > > > that refers to types in the package, by using `-XaddExports` or its > > > equivalent at compile time, and it will work as expected at run > time. > > > It thus may be useful to use `exports dynamic` for packages that > > > contain legacy APIs whose use is strongly discouraged, e.g., those > in > > > the `jdk.unsupported` module of the reference implementation, > thereby > > > forcing anyone who wants to compile against them to go to the > trouble > > > of using `-XaddExports`. > > > > > > - Intrusive access to arbitrary packages of arbitrary modules by, > e.g., > > > debugging tools, will still require the use of sharp knives such as > > > the `-XaddExports` command-line option or its equivalent, or JVM > TI. > > > > > > - Using the `-XaddExports` option or its equivalent remains awkward, > > > and sometimes it's the only way out. To ease migration I think > it's > > > worth considering some way for an application packaged as a JAR > file > > > to include such options in its `MANIFEST.MF` file, as suggested by > > > Simon Nash [5]. I'll create a separate issue for that. > > > > > > - New kinds of resolution failures are possible. If module `A` > > > requires `B`, and `B` requires `C`, then if `B` and `C` both > declare > > > some package `P` to be exported dynamically, then a split-package > > > error will be reported. This may surprise, since the packages are > > > just internal implementation details that were exported dynamically > > > so that some framework could access their types. This is not, > > > however, all that different from the kinds of resolution and > > > layer-creation failures that are already possible due to package > > > collisions. It's unlikely to happen all that often in practice, > > > and the fix is straightforward: Just use reverse-DNS or otherwise > > > sufficiently-unique package names, as most people already do. It > > > is worth exploring whether javac can detect at least some of these > > > kinds of collisions and issue appropriate warnings. > > > > I would like to agree on that proposal only on the ground that it > requires > > to change the classfile format thus allows in the same time to fix the > > issues of the current classfile spec (encoding of the name module as a > > Class instead of UTF8, wrong flag for requires public), but i'm not sure > > this proposal support it's own weight. > > > > Without dynamic, a 'classical' export works the same way at runtime so > > adding 'dynamic' to an export only restrict the usage at compile time. > > Given that we can already selectively export packages to some chosen > > modules which hide them for the rest of the other modules at compile > time, > > adding dynamic to the spec adds little value. > > > > And it has several drawbacks, the main one is that it weaken the > > compiletime/runtime fidelity story, with a dynamic export the runtime > world > > is not the compiletime world anymore. > > The main motivation to add dynamic seems to be for backward compatibility > > during the transition between the classpath world and the full module > world, > > for several other cases, we use the command line for that, i don't see > why > > this case is so special that it should not work like the other > > compatibility issues. > > > > Moreover, an agent, an annotation processor and a jlink plugin are all > > able to implement the same trick, add export after the compilation and > > before the runtime, > > so i don't think it's a good idea to change the spec for that. > > > > R?mi > > > > > > > > > > > [1] > > > > > > http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccessToNonExportedTypes > > > [2] > > > > > > http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithoutReadability > > > [3] > > http://openjdk.java.net/projects/jigsaw/spec/sotms/#qualified-exports > > > [4] > > > > > > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-December/000205.html > > > [5] > > > > > > http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005745.html > > > > > > From mark.reinhold at oracle.com Wed Jun 29 23:31:00 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 29 Jun 2016 16:31:00 -0700 Subject: Proposal: #BootstrapClassLoaderSearchInJVMTI In-Reply-To: References: <20160628212215.9E7D9B59D4@eggemoggin.niobe.net> Message-ID: <20160629163100.841006773eggemoggin.niobe.net> 2016/6/29 3:45:53 -0700, tim_ellison at uk.ibm.com: > mark.reinhold at oracle.com (Mark Reinhold) wrote on 28/06/2016 22:22:15: >> Issue summary >> ------------- >> >> #BootstrapClassLoaderSearchInJVMTI --- The JVM TI API [1] defines a >> function, `AddToBootstrapClassLoaderSearch` [2], by which an >> instrumentation agent can append a file-system path element to the >> bootstrap class loader's class path. ... >> >> Proposal >> -------- >> >> Retain this method. Do not deprecate it. > > What would be the effect calling this function on a modular runtime when > passed a directory search path segment? a modular JAR? a regular JAR? It will work just as it does today. If the bootstrap class loader cannot locate a suitable class file within the run-time system then it will consult the path defined by calls to this function. Each element of the path can be either a JAR file or an "exploded" directory tree corresponding to a package hierarchy. Whether a JAR file is a modular JAR is of no import -- any module descriptor within it will be ignored. - Mark From mark.reinhold at oracle.com Wed Jun 29 23:37:18 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 29 Jun 2016 16:37:18 -0700 Subject: Proposal: #ModuleAnnotations and #ModuleDeprecation In-Reply-To: <1683195038.1023897.1467191318948.JavaMail.zimbra@u-pem.fr> References: <20160628211915.9706BB59C8@eggemoggin.niobe.net> <1683195038.1023897.1467191318948.JavaMail.zimbra@u-pem.fr> Message-ID: <20160629163718.727221014eggemoggin.niobe.net> 2016/6/29 2:08:38 -0700, Remi Forax : > ... > > I like both proposals, adding annotations and imports on module. > > I disagree that those annotations should not be visible at runtime, i > don't see why we need a special case here, it's not complex to > implement because the VM already provide the hook to access to those > annotations at runtime, @Deprecated is an annotation visible at > runtime for a good reason, it allows by example to implement policy > like disallowing or emit a warning when calling a deprecated code at > runtime. I completely agree, and the intent of the proposal is to make annotations on module declarations available at run time -- that's why it specifically mentions extending the `java.lang.reflect` API. If there's text in the proposal that suggests otherwise then please point it out and I'll fix it. Thanks. - Mark From mark.reinhold at oracle.com Thu Jun 30 00:03:08 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 29 Jun 2016 17:03:08 -0700 Subject: Proposal: #CustomizableAutomaticModuleNameMapping In-Reply-To: <5773E478.9020903@redhat.com> References: <20160628212315.A247FB59D8@eggemoggin.niobe.net> <5773E478.9020903@redhat.com> Message-ID: <20160629170308.510416918eggemoggin.niobe.net> 2016/6/29 8:08:40 -0700, david.lloyd at redhat.com: > On 06/28/2016 04:23 PM, Mark Reinhold wrote: >> ... >> >> Proposal >> -------- >> >> Extend the `java.lang.module.ModuleDescriptor.Builder` API so that it >> can be used to create automatic as well as explicit modules. This will >> allow a container application to use a custom `ModuleFinder` to manage >> the mapping from artifact names to module names, version strings, and >> so forth. > > Could you please clarify how, after this change, an automatic module > would be different from a regular module, from the perspective of any > code which is using the descriptor Builder API? The `isAutomatic()` method of an explicitly-built automatic `ModuleDescriptor` instance will return `true`. - Mark From mark.reinhold at oracle.com Thu Jun 30 00:35:59 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 29 Jun 2016 17:35:59 -0700 Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` In-Reply-To: <1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr> References: <20160628211815.87E88B59C4@eggemoggin.niobe.net> <1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr> Message-ID: <20160629173559.18960061eggemoggin.niobe.net> 2016/6/29 1:51:16 -0700, Remi Forax : > I would like to agree on that proposal only on the ground that it > requires to change the classfile format thus allows in the same time > to fix the issues of the current classfile spec (encoding of the name > module as a Class instead of UTF8, wrong flag for requires public), I think that these are completely distinct issues (though your class-file issues are not yet in the issue list -- would you care to suggest some text?). > but i'm not sure this proposal support it's own weight. > > Without dynamic, a 'classical' export works the same way at runtime so > adding 'dynamic' to an export only restrict the usage at compile time. > Given that we can already selectively export packages to some chosen > modules which hide them for the rest of the other modules at compile > time, adding dynamic to the spec adds little value. Qualified exports aren't really a satisfactory substitute for dynamic exports. They still export packages at compile time, thereby tempting unwanted usage. They're also really meant only for use within a group of tightly-associated modules, where hashing can be used to ensure integrity, rather than amongst unrelated modules, between which names can all too easily be spoofed. More to the point, qualified exports are not adequate for all use cases. If you want to write a module that uses JPA but is independent of JPA implementation then you'll need to make your entity classes available to whichever JPA implementation you do wind up using (Hibernate or EclipseLink or ...), but you don't know the name of that implementation's module. In other words, you can't write exports com.foo.app.model to jpa; because `jpa` is not an actual module, it's a stand-in. We could introduce a level of indirection so that you could do that sort of thing, say some notion of "abstract module", but that'd be a very, very deep change and it doesn't fit at all well with existing practices around JAR files. The nice thing about `exports dynamic` is that it lets you say "just make the types in this package available for reflection by anybody, I'm okay with that". > And it has several drawbacks, the main one is that it weaken the > compiletime/runtime fidelity story, with a dynamic export the runtime > world is not the compiletime world anymore. The main motivation to > add dynamic seems to be for backward compatibility during the > transition between the classpath world and the full module world, for > several other cases, we use the command line for that, i don't see why > this case is so special that it should not work like the other > compatibility issues. Yes, it does weaken the fidelity story in order to accommodate existing practice, but at least it does so in an explicit manner so that what's going on is apparent just from reading the code. I don't think, however, that this change is just about compatibility and migration. Even in an idealistic, Utopian, fully-modular world I expect that we'll still want to use frameworks that use reflection on our own types to help us be more productive. > Moreover, an agent, an annotation processor and a jlink plugin are all > able to implement the same trick, add export after the compilation and > before the runtime, so i don't think it's a good idea to change the > spec for that. Then how do you express, in your source code, that you want an export to be added at run time? You could use an annotation, but that would require anybody who wants to do this sort of thing to configure their environment to include the module that defines that annotation and to add the required annotation processor at the appropriate point. Do we really want to tell people to do that when we can solve the problem for them so simply? - Mark From david.lloyd at redhat.com Thu Jun 30 11:30:07 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 30 Jun 2016 06:30:07 -0500 Subject: Proposal: #CustomizableAutomaticModuleNameMapping In-Reply-To: <20160629170308.510416918eggemoggin.niobe.net> References: <20160628212315.A247FB59D8@eggemoggin.niobe.net> <5773E478.9020903@redhat.com> <20160629170308.510416918eggemoggin.niobe.net> Message-ID: <577502BF.5080301@redhat.com> On 06/29/2016 07:03 PM, mark.reinhold at oracle.com wrote: > 2016/6/29 8:08:40 -0700, david.lloyd at redhat.com: >> On 06/28/2016 04:23 PM, Mark Reinhold wrote: >>> ... >>> >>> Proposal >>> -------- >>> >>> Extend the `java.lang.module.ModuleDescriptor.Builder` API so that it >>> can be used to create automatic as well as explicit modules. This will >>> allow a container application to use a custom `ModuleFinder` to manage >>> the mapping from artifact names to module names, version strings, and >>> so forth. >> >> Could you please clarify how, after this change, an automatic module >> would be different from a regular module, from the perspective of any >> code which is using the descriptor Builder API? > > The `isAutomatic()` method of an explicitly-built automatic > `ModuleDescriptor` instance will return `true`. Yeah, I get that, but I mean when looking at the final built module. I assumed that to be implied since building the descriptor is useless if you don't make a module out of it. Are automatic modules substantially different from regular modules in any way? If not, is there a reason to continue to have them? -- - DML From Tim_Ellison at uk.ibm.com Thu Jun 30 14:11:36 2016 From: Tim_Ellison at uk.ibm.com (Tim Ellison) Date: Thu, 30 Jun 2016 15:11:36 +0100 Subject: Proposal: #BootstrapClassLoaderSearchInJVMTI In-Reply-To: <20160629163100.841006773eggemoggin.niobe.net> References: <20160628212215.9E7D9B59D4@eggemoggin.niobe.net> <20160629163100.841006773eggemoggin.niobe.net> Message-ID: mark.reinhold at oracle.com wrote on 30/06/2016 00:31:00: > 2016/6/29 3:45:53 -0700, tim_ellison at uk.ibm.com: > > mark.reinhold at oracle.com (Mark Reinhold) wrote on 28/06/2016 22:22:15: > >> Issue summary > >> ------------- > >> > >> #BootstrapClassLoaderSearchInJVMTI --- The JVM TI API [1] defines a > >> function, `AddToBootstrapClassLoaderSearch` [2], by which an > >> instrumentation agent can append a file-system path element to the > >> bootstrap class loader's class path. ... > >> > >> Proposal > >> -------- > >> > >> Retain this method. Do not deprecate it. > > > > What would be the effect calling this function on a modular runtime when > > passed a directory search path segment? a modular JAR? a regular JAR? > > It will work just as it does today. > > If the bootstrap class loader cannot locate a suitable class file within > the run-time system then it will consult the path defined by calls to > this function. Each element of the path can be either a JAR file or an > "exploded" directory tree corresponding to a package hierarchy. Whether > a JAR file is a modular JAR is of no import -- any module descriptor > within it will be ignored. Just to check I fully understand, this function will affect the search path of the unnamed module associated with the bootstrap class loader, correct? If that's so then I'm happy with the proposed solution. Thanks, Tim Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From volker.berlin at goebel-clan.de Thu Jun 30 20:09:41 2016 From: volker.berlin at goebel-clan.de (Volker Berlin) Date: Thu, 30 Jun 2016 22:09:41 +0200 Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` In-Reply-To: <20160628211815.87E88B59C4@eggemoggin.niobe.net> References: <20160628211815.87E88B59C4@eggemoggin.niobe.net> Message-ID: Reflection should be enabled by default. My current launcher for Java 9 check if the needed refection is possible. If not possible then it restart the application with a very long list of -XaddExports switches. I think this is not what we want. Volker Berlin i-net software Am 28.06.2016 um 23:18 schrieb Mark Reinhold: > Issue summary > ------------- > > #ReflectiveAccessToNonExportedTypes --- Some kinds of framework > libraries require reflective access to members of the non-exported > types of other modules; examples include dependency injection (Guice), > persistence (JPA), debugging tools, code-automation tools, and > serialization (XStream). In some cases the particular library to be > used is not known until run time (e.g., Hibernate and EclipseLink both > implement JPA). This capability is also sometimes used to work around > bugs in unchangeable code. Access to non-exported packages can, at > present, only be done via command-line flags, which is extremely > awkward. Provide an easier way for reflective code to access such > non-exported types. [1] > > Proposal > -------- > > Extend the language of module declarations so that a package can be > declared to be exported at run time but not at compile time. This is, > roughly, the dual of the `requires static` construct proposed for > #CompileTimeDependences, hence we propose to introduce a new modifier, > `dynamic`, for use on the `exports` directive. It has the following > meanings: > > - At compile time, `exports dynamic P` does not cause the package `P` > to be exported, though it does require `P` to be a package defined > in the module. > > - In phases after compile time, `exports dynamic P` behaves in exactly > the same way as `exports P`. It therefore takes part fully in > resolution and configuration, and is subject to the same consistency > constraints as normally-exported packages (e.g., no split packages). > > Thus a module declaration of the form > > module com.foo.app { > requires hibernate.core; > requires hibernate.entitymanager; > exports dynamic com.foo.app.model; > } > > makes the types in the `com.foo.app.model` package accessible at run time > but not at compile time. In combination with the earlier change to > enable #ReflectionWithoutReadability [2] this means that frameworks that > today use core reflection to manipulate user classes at run time are more > likely to work out-of-the-box, without change, as automatic modules. If > the `com.foo.app.model` package in this example includes entity classes > to be managed by Hibernate then the framework will be able to access them > without further ado, but under normal circumstances an attempt to compile > code that refers directly to those classes will fail. > > The `dynamic` modifier can be applied to both unqualified and qualified > `exports` directives, though the caveats on using qualified exports [3] > still apply. > > Notes > ----- > > - To access a non-public member in a dynamically-exported package you > must still invoke the appropriate `setAccessible` method, just as > you do today to access non-public members in an exported package. > > - This proposal is similar to R??mi's suggestion to add a way to export > a package only for reflection, still requiring that `setAccessible` > be invoked [4]. This proposal differs in that it does not require > `setAccessible` to be invoked to access public members in a > dynamically-exported package. > > - This proposal primarily addresses "friendly" uses of reflection, such > as dependency injection and persistence, in which the author of a > module knows in advance that one or more packages must be exported at > run time for reflective access by frameworks. This proposal is also > convenient for frameworks that generate code at run time, since the > constant pool of a generated class file can include static references > to types in dynamically exported packages. > > - This proposal opens the door to a category of second-class APIs. If > a package is exported dynamically then you can still compile code > that refers to types in the package, by using `-XaddExports` or its > equivalent at compile time, and it will work as expected at run time. > It thus may be useful to use `exports dynamic` for packages that > contain legacy APIs whose use is strongly discouraged, e.g., those in > the `jdk.unsupported` module of the reference implementation, thereby > forcing anyone who wants to compile against them to go to the trouble > of using `-XaddExports`. > > - Intrusive access to arbitrary packages of arbitrary modules by, e.g., > debugging tools, will still require the use of sharp knives such as > the `-XaddExports` command-line option or its equivalent, or JVM TI. > > - Using the `-XaddExports` option or its equivalent remains awkward, > and sometimes it's the only way out. To ease migration I think it's > worth considering some way for an application packaged as a JAR file > to include such options in its `MANIFEST.MF` file, as suggested by > Simon Nash [5]. I'll create a separate issue for that. > > - New kinds of resolution failures are possible. If module `A` > requires `B`, and `B` requires `C`, then if `B` and `C` both declare > some package `P` to be exported dynamically, then a split-package > error will be reported. This may surprise, since the packages are > just internal implementation details that were exported dynamically > so that some framework could access their types. This is not, > however, all that different from the kinds of resolution and > layer-creation failures that are already possible due to package > collisions. It's unlikely to happen all that often in practice, > and the fix is straightforward: Just use reverse-DNS or otherwise > sufficiently-unique package names, as most people already do. It > is worth exploring whether javac can detect at least some of these > kinds of collisions and issue appropriate warnings. > > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccessToNonExportedTypes > [2] http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithoutReadability > [3] http://openjdk.java.net/projects/jigsaw/spec/sotms/#qualified-exports > [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-December/000205.html > [5] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005745.html > From alex.buckley at oracle.com Thu Jun 30 21:02:34 2016 From: alex.buckley at oracle.com (Alex Buckley) Date: Thu, 30 Jun 2016 14:02:34 -0700 Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` In-Reply-To: References: <20160628211815.87E88B59C4@eggemoggin.niobe.net> Message-ID: <577588EA.6060202@oracle.com> Volker, Please share details of why your application needs to reflect over the internals of another module. Alex On 6/30/2016 1:09 PM, Volker Berlin wrote: > Reflection should be enabled by default. My current launcher for Java 9 > check if the needed refection is possible. If not possible then it > restart the application with a very long list of -XaddExports switches. > I think this is not what we want. > > Volker Berlin > i-net software > > > Am 28.06.2016 um 23:18 schrieb Mark Reinhold: >> Issue summary >> ------------- >> >> #ReflectiveAccessToNonExportedTypes --- Some kinds of framework >> libraries require reflective access to members of the non-exported >> types of other modules; examples include dependency injection (Guice), >> persistence (JPA), debugging tools, code-automation tools, and >> serialization (XStream). In some cases the particular library to be >> used is not known until run time (e.g., Hibernate and EclipseLink both >> implement JPA). This capability is also sometimes used to work around >> bugs in unchangeable code. Access to non-exported packages can, at >> present, only be done via command-line flags, which is extremely >> awkward. Provide an easier way for reflective code to access such >> non-exported types. [1] >> >> Proposal >> -------- >> >> Extend the language of module declarations so that a package can be >> declared to be exported at run time but not at compile time. This is, >> roughly, the dual of the `requires static` construct proposed for >> #CompileTimeDependences, hence we propose to introduce a new modifier, >> `dynamic`, for use on the `exports` directive. It has the following >> meanings: >> >> - At compile time, `exports dynamic P` does not cause the package `P` >> to be exported, though it does require `P` to be a package defined >> in the module. >> >> - In phases after compile time, `exports dynamic P` behaves in exactly >> the same way as `exports P`. It therefore takes part fully in >> resolution and configuration, and is subject to the same consistency >> constraints as normally-exported packages (e.g., no split packages). From scolebourne at joda.org Thu Jun 30 21:29:24 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 30 Jun 2016 22:29:24 +0100 Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` In-Reply-To: <577588EA.6060202@oracle.com> References: <20160628211815.87E88B59C4@eggemoggin.niobe.net> <577588EA.6060202@oracle.com> Message-ID: For 20 years, reflection has been used as a low-level tool to access any code on the classpath. No developer up to this point has thought about modules being a relevant boundary for what they should or should not touch with reflection. To take one example from OpenGamma Strata: https://github.com/OpenGamma/Strata/blob/master/modules/collect/src/main/java/com/opengamma/strata/collect/named/ExtendedEnum.java#L181 ExtendedEnum is a mechanism to provide enum-like behaviour, where the set of constants is fixed at startup, but controlled dynamically based on what configuration files are present. One of the options it provides is for the configuration file to specify a Class name, which is then reflected on to find suitable enum-like constants. When designing this feature, at no stage did I think that the code would not work because some of the constants are in a different module. (Application users can add their own configuration files and constants, ie, in a separate module). This proposal essentially requires application users to specifically expose these types to reflection where they might otherwise lock them down in a non-exported package. While in this case, I could document that users of extended enums must make sure their code can be accessed if in a different module, it would at the very least be a faff, and a source of questions. IMO this goes against how reflection has been used and is simply not practical. I would propose the opposite change to this one. Provide a mechanism where a module can choose to block reflection on a package by package basis. I believe this could provide the security necessary without obstructing adoption. Stephen On 30 June 2016 at 22:02, Alex Buckley wrote: > Volker, > > Please share details of why your application needs to reflect over the > internals of another module. > > Alex > > > On 6/30/2016 1:09 PM, Volker Berlin wrote: >> >> Reflection should be enabled by default. My current launcher for Java 9 >> check if the needed refection is possible. If not possible then it >> restart the application with a very long list of -XaddExports switches. >> I think this is not what we want. >> >> Volker Berlin >> i-net software >> >> >> Am 28.06.2016 um 23:18 schrieb Mark Reinhold: >>> >>> Issue summary >>> ------------- >>> >>> #ReflectiveAccessToNonExportedTypes --- Some kinds of framework >>> libraries require reflective access to members of the non-exported >>> types of other modules; examples include dependency injection (Guice), >>> persistence (JPA), debugging tools, code-automation tools, and >>> serialization (XStream). In some cases the particular library to be >>> used is not known until run time (e.g., Hibernate and EclipseLink both >>> implement JPA). This capability is also sometimes used to work around >>> bugs in unchangeable code. Access to non-exported packages can, at >>> present, only be done via command-line flags, which is extremely >>> awkward. Provide an easier way for reflective code to access such >>> non-exported types. [1] >>> >>> Proposal >>> -------- >>> >>> Extend the language of module declarations so that a package can be >>> declared to be exported at run time but not at compile time. This is, >>> roughly, the dual of the `requires static` construct proposed for >>> #CompileTimeDependences, hence we propose to introduce a new modifier, >>> `dynamic`, for use on the `exports` directive. It has the following >>> meanings: >>> >>> - At compile time, `exports dynamic P` does not cause the package `P` >>> to be exported, though it does require `P` to be a package defined >>> in the module. >>> >>> - In phases after compile time, `exports dynamic P` behaves in exactly >>> the same way as `exports P`. It therefore takes part fully in >>> resolution and configuration, and is subject to the same consistency >>> constraints as normally-exported packages (e.g., no split packages). From nipa at codefx.org Thu Jun 30 21:36:34 2016 From: nipa at codefx.org (Nicolai Parlog) Date: Thu, 30 Jun 2016 23:36:34 +0200 Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` In-Reply-To: References: <20160628211815.87E88B59C4@eggemoggin.niobe.net> <1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr> Message-ID: <577590E2.7020907@codefx.org> Hi! > In my opinion, reflection should not apply any module-boundary > checks. Maybe this list is not a representative selection of Java developers, maybe I'm just the only one thinking this but, to paraphrase: Reflection over internals must die in a fire! Yes, yes, tools to great things, we need to see everything, ... Still, I think that modules need a safe space that they can use without anybody intruding. Maybe I'm a hardliner but I already view the command line parameters with suspicion. In that light, I like the "dynamic" proposal. Not the name, though. Having the symmetric pair "static vs dynamic" looks nice but it takes a lot of words to explain why exactly these terms apply here. We seem to be looking for qualifiers for "requires" and "exports". Maybe just using other terms solves the problem? requires & consumes exports & contains so long ... Nicolai On 30.06.2016 01:23, Rafael Winterhalter wrote: > I also agree here that the use of reflection should not require any > form of explicit permission by the user of a library. In a way, > this ties an implementation detail (reflection in this case) to the > usage of the library. The library cannot work without the explicit > configuration by a user and this only adds boilerplate to using > Hibernate. > > Unfortunately, many Java developers still do not know much about > reflection or about the inner workings of many libraries in the > market. If a user needed to explicitly export packages, this might > just lead to people "over exporting" their packages just to "make > Hibernate work". > > In my opinion, reflection should not apply any module-boundary > checks. > > Best regards, Rafael > > 2016-06-29 20:13 GMT+02:00 Paul Benedict : > >> Replying to "observers" list... >> >> I think the main problem here is that core reflection has become >> too restrictive. Reflection should be "magical" and have no >> knowledge of any kind of module and/or visibility boundaries. If >> you really want to control what can be reflected upon, that >> should be done via the Policy file of the Security Manager. >> >> This proposal is trying to hack out-of-the-box created by too >> strict of a design. It is very awkward to denote which packages >> must be reflectable ahead of time --- how can you figure that >> out?? The consumers who are doing the reflection are innumerable. >> I'd go as far to say it's it's impossible to devise a preemptive >> comprehensive strategy that may satisfy the kinds of tooling in >> the wild. >> >> It's best just to open up reflection so it can continue doing >> what it always has done well: let me see anything I want unless >> the Security Manager says no. >> >> Cheers, Paul >> >> On Wed, Jun 29, 2016 at 3:51 AM, Remi Forax >> wrote: >> >>> >>> >>> ----- Mail original ----- >>>> De: "Mark Reinhold" ?: >>>> jpms-spec-experts at openjdk.java.net Envoy?: Mardi 28 Juin 2016 >>>> 23:18:15 Objet: Proposal: >>>> #ReflectiveAccessToNonExportedTypes: `exports dynamic` >>>> >>>> Issue summary ------------- >>>> >>>> #ReflectiveAccessToNonExportedTypes --- Some kinds of >>>> framework libraries require reflective access to members of >>>> the non-exported types of other modules; examples include >>>> dependency injection >> (Guice), >>>> persistence (JPA), debugging tools, code-automation tools, >>>> and serialization (XStream). In some cases the particular >>>> library to be used is not known until run time (e.g., >>>> Hibernate and EclipseLink >> both >>>> implement JPA). This capability is also sometimes used to >>>> work >> around >>>> bugs in unchangeable code. Access to non-exported packages >>>> can, at present, only be done via command-line flags, which >>>> is extremely awkward. Provide an easier way for reflective >>>> code to access such non-exported types. [1] >>>> >>>> Proposal -------- >>>> >>>> Extend the language of module declarations so that a package >>>> can be declared to be exported at run time but not at compile >>>> time. This is, roughly, the dual of the `requires static` >>>> construct proposed for #CompileTimeDependences, hence we >>>> propose to introduce a new modifier, `dynamic`, for use on >>>> the `exports` directive. It has the following meanings: >>>> >>>> - At compile time, `exports dynamic P` does not cause the >>>> package `P` to be exported, though it does require `P` to be >>>> a package defined in the module. >>>> >>>> - In phases after compile time, `exports dynamic P` behaves >>>> in >> exactly >>>> the same way as `exports P`. It therefore takes part fully >>>> in resolution and configuration, and is subject to the same >> consistency >>>> constraints as normally-exported packages (e.g., no split >> packages). >>>> >>>> Thus a module declaration of the form >>>> >>>> module com.foo.app { requires hibernate.core; requires >>>> hibernate.entitymanager; exports dynamic com.foo.app.model; >>>> } >>>> >>>> makes the types in the `com.foo.app.model` package accessible >>>> at run >> time >>>> but not at compile time. In combination with the earlier >>>> change to enable #ReflectionWithoutReadability [2] this means >>>> that frameworks >> that >>>> today use core reflection to manipulate user classes at run >>>> time are >> more >>>> likely to work out-of-the-box, without change, as automatic >>>> modules. >> If >>>> the `com.foo.app.model` package in this example includes >>>> entity classes to be managed by Hibernate then the framework >>>> will be able to access >> them >>>> without further ado, but under normal circumstances an >>>> attempt to >> compile >>>> code that refers directly to those classes will fail. >>>> >>>> The `dynamic` modifier can be applied to both unqualified and >>>> qualified `exports` directives, though the caveats on using >>>> qualified exports [3] still apply. >>>> >>>> Notes ----- >>>> >>>> - To access a non-public member in a dynamically-exported >>>> package you must still invoke the appropriate `setAccessible` >>>> method, just as you do today to access non-public members in >>>> an exported package. >>>> >>>> - This proposal is similar to R?mi's suggestion to add a way >>>> to >> export >>>> a package only for reflection, still requiring that >>>> `setAccessible` be invoked [4]. This proposal differs in >>>> that it does not require `setAccessible` to be invoked to >>>> access public members in a dynamically-exported package. >>>> >>>> - This proposal primarily addresses "friendly" uses of >>>> reflection, >> such >>>> as dependency injection and persistence, in which the author >>>> of a module knows in advance that one or more packages must >>>> be exported >> at >>>> run time for reflective access by frameworks. This proposal >>>> is >> also >>>> convenient for frameworks that generate code at run time, >>>> since the constant pool of a generated class file can include >>>> static >> references >>>> to types in dynamically exported packages. >>>> >>>> - This proposal opens the door to a category of second-class >>>> APIs. >> If >>>> a package is exported dynamically then you can still compile >>>> code that refers to types in the package, by using >>>> `-XaddExports` or its equivalent at compile time, and it will >>>> work as expected at run >> time. >>>> It thus may be useful to use `exports dynamic` for packages >>>> that contain legacy APIs whose use is strongly discouraged, >>>> e.g., those >> in >>>> the `jdk.unsupported` module of the reference >>>> implementation, >> thereby >>>> forcing anyone who wants to compile against them to go to >>>> the >> trouble >>>> of using `-XaddExports`. >>>> >>>> - Intrusive access to arbitrary packages of arbitrary modules >>>> by, >> e.g., >>>> debugging tools, will still require the use of sharp knives >>>> such as the `-XaddExports` command-line option or its >>>> equivalent, or JVM >> TI. >>>> >>>> - Using the `-XaddExports` option or its equivalent remains >>>> awkward, and sometimes it's the only way out. To ease >>>> migration I think >> it's >>>> worth considering some way for an application packaged as a >>>> JAR >> file >>>> to include such options in its `MANIFEST.MF` file, as >>>> suggested by Simon Nash [5]. I'll create a separate issue >>>> for that. >>>> >>>> - New kinds of resolution failures are possible. If module >>>> `A` requires `B`, and `B` requires `C`, then if `B` and `C` >>>> both >> declare >>>> some package `P` to be exported dynamically, then a >>>> split-package error will be reported. This may surprise, >>>> since the packages are just internal implementation details >>>> that were exported dynamically so that some framework could >>>> access their types. This is not, however, all that different >>>> from the kinds of resolution and layer-creation failures that >>>> are already possible due to package collisions. It's >>>> unlikely to happen all that often in practice, and the fix is >>>> straightforward: Just use reverse-DNS or otherwise >>>> sufficiently-unique package names, as most people already do. >>>> It is worth exploring whether javac can detect at least some >>>> of these kinds of collisions and issue appropriate warnings. >>> >>> I would like to agree on that proposal only on the ground that >>> it >> requires >>> to change the classfile format thus allows in the same time to >>> fix the issues of the current classfile spec (encoding of the >>> name module as a Class instead of UTF8, wrong flag for requires >>> public), but i'm not sure this proposal support it's own >>> weight. >>> >>> Without dynamic, a 'classical' export works the same way at >>> runtime so adding 'dynamic' to an export only restrict the >>> usage at compile time. Given that we can already selectively >>> export packages to some chosen modules which hide them for the >>> rest of the other modules at compile >> time, >>> adding dynamic to the spec adds little value. >>> >>> And it has several drawbacks, the main one is that it weaken >>> the compiletime/runtime fidelity story, with a dynamic export >>> the runtime >> world >>> is not the compiletime world anymore. The main motivation to >>> add dynamic seems to be for backward compatibility during the >>> transition between the classpath world and the full module >> world, >>> for several other cases, we use the command line for that, i >>> don't see >> why >>> this case is so special that it should not work like the other >>> compatibility issues. >>> >>> Moreover, an agent, an annotation processor and a jlink plugin >>> are all able to implement the same trick, add export after the >>> compilation and before the runtime, so i don't think it's a >>> good idea to change the spec for that. >>> >>> R?mi >>> >>>> >>>> >>>> [1] >>>> >>> >> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccess ToNonExportedTypes >>>> >> [2] >>>> >>> >> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithou tReadability >>>> >> [3] >>> http://openjdk.java.net/projects/jigsaw/spec/sotms/#qualified-export s >>>> >>> [4] >>>> >>> >> http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-Decembe r/000205.html >>>> >> [5] >>>> >>> >> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/00574 5.html >>>> >>> >> > >> -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx From mark.reinhold at oracle.com Thu Jun 30 21:45:31 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 30 Jun 2016 14:45:31 -0700 Subject: Proposal: #CustomizableAutomaticModuleNameMapping In-Reply-To: <577502BF.5080301@redhat.com> References: <20160628212315.A247FB59D8@eggemoggin.niobe.net> <5773E478.9020903@redhat.com> <20160629170308.510416918eggemoggin.niobe.net> <577502BF.5080301@redhat.com> Message-ID: <20160630144531.546774815eggemoggin.niobe.net> 2016/6/30 4:30:07 -0700, david.lloyd at redhat.com: > On 06/29/2016 07:03 PM, mark.reinhold at oracle.com wrote: >> 2016/6/29 8:08:40 -0700, david.lloyd at redhat.com: >>> Could you please clarify how, after this change, an automatic module >>> would be different from a regular module, from the perspective of any >>> code which is using the descriptor Builder API? >> >> The `isAutomatic()` method of an explicitly-built automatic >> `ModuleDescriptor` instance will return `true`. > > Yeah, I get that, but I mean when looking at the final built module. I > assumed that to be implied since building the descriptor is useless if > you don't make a module out of it. Are automatic modules substantially > different from regular modules in any way? If not, is there a reason to > continue to have them? http://openjdk.java.net/projects/jigsaw/spec/sotms/#automatic-modules http://cr.openjdk.java.net/~mr/jigsaw/spec/api/java/lang/module/ModuleFinder.html#of-java.nio.file.Path...- - Mark From alex.buckley at oracle.com Thu Jun 30 21:58:42 2016 From: alex.buckley at oracle.com (Alex Buckley) Date: Thu, 30 Jun 2016 14:58:42 -0700 Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` In-Reply-To: References: <20160628211815.87E88B59C4@eggemoggin.niobe.net> <577588EA.6060202@oracle.com> Message-ID: <57759612.2080902@oracle.com> On 6/30/2016 2:29 PM, Stephen Colebourne wrote: > To take one example from OpenGamma Strata: > https://github.com/OpenGamma/Strata/blob/master/modules/collect/src/main/java/com/opengamma/strata/collect/named/ExtendedEnum.java#L181 (Line 50: I think "are" should be "as".) (Line 181: I notice that while "constants" and "lookup" providers are specified in the class-level javadoc, "instance" providers are not. Admittedly, reflection is used not only for the [unspecified] special field of an "instance" provider class but also for the [specified] no-args ctor of a "lookup" provider class.) > ExtendedEnum is a mechanism to provide enum-like behaviour, where the > set of constants is fixed at startup, but controlled dynamically based > on what configuration files are present. One of the options it > provides is for the configuration file to specify a Class name, which > is then reflected on to find suitable enum-like constants. > > When designing this feature, at no stage did I think that the code > would not work because some of the constants are in a different > module. (Application users can add their own configuration files and > constants, ie, in a separate module). > > This proposal essentially requires application users to specifically > expose these types to reflection where they might otherwise lock them > down in a non-exported package. While in this case, I could document > that users of extended enums must make sure their code can be accessed > if in a different module, it would at the very least be a faff, and a > source of questions. Counterpoint: the user's module-info.java is a great place to document the special relationship between one of their packages and one of their dependencies. module myapp { requires opengamma.strata; // Classes to drive Strata's ExtendedEnum instances exports dynamic com.foo.app.constants.tickers to opengamma.strata; } Alex From blackdrag at gmx.org Thu Jun 30 22:16:49 2016 From: blackdrag at gmx.org (Jochen Theodorou) Date: Fri, 1 Jul 2016 00:16:49 +0200 Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` In-Reply-To: <57759612.2080902@oracle.com> References: <20160628211815.87E88B59C4@eggemoggin.niobe.net> <577588EA.6060202@oracle.com> <57759612.2080902@oracle.com> Message-ID: <57759A51.3020102@gmx.org> On 30.06.2016 23:58, Alex Buckley wrote: [...] > Counterpoint: the user's module-info.java is a great place to document > the special relationship between one of their packages and one of their > dependencies. > > module myapp { > requires opengamma.strata; > > // Classes to drive Strata's ExtendedEnum instances > exports dynamic com.foo.app.constants.tickers to opengamma.strata; > } it would be btw interesting to have an export-all: module myapp { requires org.apache.groovy-core; exports static * to org.apache.groovy-core; } bye Jochen From rafael.wth at gmail.com Thu Jun 30 22:20:31 2016 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Fri, 1 Jul 2016 00:20:31 +0200 Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` In-Reply-To: <577590E2.7020907@codefx.org> References: <20160628211815.87E88B59C4@eggemoggin.niobe.net> <1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr> <577590E2.7020907@codefx.org> Message-ID: I disagree on two ends here. I do not think that shielding internals is a problem with regards to reflection. You want to shield internals in order to not break dependants by prohibiting their existance. With reflection, you do however typically react to the actual implementation by discovering it at runtime. If your use pattern was fixed, you would always require a static dependency over reflection in the first place. I do neither think that reflection only interests internals. For example, maybe you use Hibernate for your persistance layer without exposing those DTOs from your module. The problem is that you invert responsibilities. I do not care about how Hibernate works as long as it does work. Almost any library uses reflection and I would not want to dig into implementation details to find out how to set up my dynamic exports. I want to require a module and hopefully the compiler tells me about anything that's wrong. Reflection access errors would only be an awkward side effect that can be avoided by not applying modules alltogether. I think this limitation would therefore hurt Java 9 adoption. The second problem I have is the issue of attempting to fix an old regret. >From the perspective of Hibernate, it would be easiest to not attempt modules as their introduction would require its users to apply additional configuration. As a library implementor, you aim for the biggest convenience. Adding this restriction neither adds additional security where a security manager should be used. I again think this would hurt adoption. I agree with Stephen's argument even though I do not think that forbidding dynamic access would be a feature anybody would use so it should not be added. Rather, something should be integrated in the security manager, if this is considered necessary. Cheers, Rafael Am 30.06.2016 11:37 nachm. schrieb "Nicolai Parlog" : > > Hi! > > > In my opinion, reflection should not apply any module-boundary > > checks. > > Maybe this list is not a representative selection of Java developers, > maybe I'm just the only one thinking this but, to paraphrase: > > Reflection over internals must die in a fire! > > Yes, yes, tools to great things, we need to see everything, ... Still, > I think that modules need a safe space that they can use without > anybody intruding. Maybe I'm a hardliner but I already view the > command line parameters with suspicion. > > In that light, I like the "dynamic" proposal. > > Not the name, though. Having the symmetric pair "static vs dynamic" > looks nice but it takes a lot of words to explain why exactly these > terms apply here. We seem to be looking for qualifiers for "requires" > and "exports". Maybe just using other terms solves the problem? > > requires & consumes > exports & contains > > so long ... Nicolai > > > > On 30.06.2016 01:23, Rafael Winterhalter wrote: > > I also agree here that the use of reflection should not require any > > form of explicit permission by the user of a library. In a way, > > this ties an implementation detail (reflection in this case) to the > > usage of the library. The library cannot work without the explicit > > configuration by a user and this only adds boilerplate to using > > Hibernate. > > > > Unfortunately, many Java developers still do not know much about > > reflection or about the inner workings of many libraries in the > > market. If a user needed to explicitly export packages, this might > > just lead to people "over exporting" their packages just to "make > > Hibernate work". > > > > In my opinion, reflection should not apply any module-boundary > > checks. > > > > Best regards, Rafael > > > > 2016-06-29 20:13 GMT+02:00 Paul Benedict : > > > >> Replying to "observers" list... > >> > >> I think the main problem here is that core reflection has become > >> too restrictive. Reflection should be "magical" and have no > >> knowledge of any kind of module and/or visibility boundaries. If > >> you really want to control what can be reflected upon, that > >> should be done via the Policy file of the Security Manager. > >> > >> This proposal is trying to hack out-of-the-box created by too > >> strict of a design. It is very awkward to denote which packages > >> must be reflectable ahead of time --- how can you figure that > >> out?? The consumers who are doing the reflection are innumerable. > >> I'd go as far to say it's it's impossible to devise a preemptive > >> comprehensive strategy that may satisfy the kinds of tooling in > >> the wild. > >> > >> It's best just to open up reflection so it can continue doing > >> what it always has done well: let me see anything I want unless > >> the Security Manager says no. > >> > >> Cheers, Paul > >> > >> On Wed, Jun 29, 2016 at 3:51 AM, Remi Forax > >> wrote: > >> > >>> > >>> > >>> ----- Mail original ----- > >>>> De: "Mark Reinhold" ?: > >>>> jpms-spec-experts at openjdk.java.net Envoy?: Mardi 28 Juin 2016 > >>>> 23:18:15 Objet: Proposal: > >>>> #ReflectiveAccessToNonExportedTypes: `exports dynamic` > >>>> > >>>> Issue summary ------------- > >>>> > >>>> #ReflectiveAccessToNonExportedTypes --- Some kinds of > >>>> framework libraries require reflective access to members of > >>>> the non-exported types of other modules; examples include > >>>> dependency injection > >> (Guice), > >>>> persistence (JPA), debugging tools, code-automation tools, > >>>> and serialization (XStream). In some cases the particular > >>>> library to be used is not known until run time (e.g., > >>>> Hibernate and EclipseLink > >> both > >>>> implement JPA). This capability is also sometimes used to > >>>> work > >> around > >>>> bugs in unchangeable code. Access to non-exported packages > >>>> can, at present, only be done via command-line flags, which > >>>> is extremely awkward. Provide an easier way for reflective > >>>> code to access such non-exported types. [1] > >>>> > >>>> Proposal -------- > >>>> > >>>> Extend the language of module declarations so that a package > >>>> can be declared to be exported at run time but not at compile > >>>> time. This is, roughly, the dual of the `requires static` > >>>> construct proposed for #CompileTimeDependences, hence we > >>>> propose to introduce a new modifier, `dynamic`, for use on > >>>> the `exports` directive. It has the following meanings: > >>>> > >>>> - At compile time, `exports dynamic P` does not cause the > >>>> package `P` to be exported, though it does require `P` to be > >>>> a package defined in the module. > >>>> > >>>> - In phases after compile time, `exports dynamic P` behaves > >>>> in > >> exactly > >>>> the same way as `exports P`. It therefore takes part fully > >>>> in resolution and configuration, and is subject to the same > >> consistency > >>>> constraints as normally-exported packages (e.g., no split > >> packages). > >>>> > >>>> Thus a module declaration of the form > >>>> > >>>> module com.foo.app { requires hibernate.core; requires > >>>> hibernate.entitymanager; exports dynamic com.foo.app.model; > >>>> } > >>>> > >>>> makes the types in the `com.foo.app.model` package accessible > >>>> at run > >> time > >>>> but not at compile time. In combination with the earlier > >>>> change to enable #ReflectionWithoutReadability [2] this means > >>>> that frameworks > >> that > >>>> today use core reflection to manipulate user classes at run > >>>> time are > >> more > >>>> likely to work out-of-the-box, without change, as automatic > >>>> modules. > >> If > >>>> the `com.foo.app.model` package in this example includes > >>>> entity classes to be managed by Hibernate then the framework > >>>> will be able to access > >> them > >>>> without further ado, but under normal circumstances an > >>>> attempt to > >> compile > >>>> code that refers directly to those classes will fail. > >>>> > >>>> The `dynamic` modifier can be applied to both unqualified and > >>>> qualified `exports` directives, though the caveats on using > >>>> qualified exports [3] still apply. > >>>> > >>>> Notes ----- > >>>> > >>>> - To access a non-public member in a dynamically-exported > >>>> package you must still invoke the appropriate `setAccessible` > >>>> method, just as you do today to access non-public members in > >>>> an exported package. > >>>> > >>>> - This proposal is similar to R?mi's suggestion to add a way > >>>> to > >> export > >>>> a package only for reflection, still requiring that > >>>> `setAccessible` be invoked [4]. This proposal differs in > >>>> that it does not require `setAccessible` to be invoked to > >>>> access public members in a dynamically-exported package. > >>>> > >>>> - This proposal primarily addresses "friendly" uses of > >>>> reflection, > >> such > >>>> as dependency injection and persistence, in which the author > >>>> of a module knows in advance that one or more packages must > >>>> be exported > >> at > >>>> run time for reflective access by frameworks. This proposal > >>>> is > >> also > >>>> convenient for frameworks that generate code at run time, > >>>> since the constant pool of a generated class file can include > >>>> static > >> references > >>>> to types in dynamically exported packages. > >>>> > >>>> - This proposal opens the door to a category of second-class > >>>> APIs. > >> If > >>>> a package is exported dynamically then you can still compile > >>>> code that refers to types in the package, by using > >>>> `-XaddExports` or its equivalent at compile time, and it will > >>>> work as expected at run > >> time. > >>>> It thus may be useful to use `exports dynamic` for packages > >>>> that contain legacy APIs whose use is strongly discouraged, > >>>> e.g., those > >> in > >>>> the `jdk.unsupported` module of the reference > >>>> implementation, > >> thereby > >>>> forcing anyone who wants to compile against them to go to > >>>> the > >> trouble > >>>> of using `-XaddExports`. > >>>> > >>>> - Intrusive access to arbitrary packages of arbitrary modules > >>>> by, > >> e.g., > >>>> debugging tools, will still require the use of sharp knives > >>>> such as the `-XaddExports` command-line option or its > >>>> equivalent, or JVM > >> TI. > >>>> > >>>> - Using the `-XaddExports` option or its equivalent remains > >>>> awkward, and sometimes it's the only way out. To ease > >>>> migration I think > >> it's > >>>> worth considering some way for an application packaged as a > >>>> JAR > >> file > >>>> to include such options in its `MANIFEST.MF` file, as > >>>> suggested by Simon Nash [5]. I'll create a separate issue > >>>> for that. > >>>> > >>>> - New kinds of resolution failures are possible. If module > >>>> `A` requires `B`, and `B` requires `C`, then if `B` and `C` > >>>> both > >> declare > >>>> some package `P` to be exported dynamically, then a > >>>> split-package error will be reported. This may surprise, > >>>> since the packages are just internal implementation details > >>>> that were exported dynamically so that some framework could > >>>> access their types. This is not, however, all that different > >>>> from the kinds of resolution and layer-creation failures that > >>>> are already possible due to package collisions. It's > >>>> unlikely to happen all that often in practice, and the fix is > >>>> straightforward: Just use reverse-DNS or otherwise > >>>> sufficiently-unique package names, as most people already do. > >>>> It is worth exploring whether javac can detect at least some > >>>> of these kinds of collisions and issue appropriate warnings. > >>> > >>> I would like to agree on that proposal only on the ground that > >>> it > >> requires > >>> to change the classfile format thus allows in the same time to > >>> fix the issues of the current classfile spec (encoding of the > >>> name module as a Class instead of UTF8, wrong flag for requires > >>> public), but i'm not sure this proposal support it's own > >>> weight. > >>> > >>> Without dynamic, a 'classical' export works the same way at > >>> runtime so adding 'dynamic' to an export only restrict the > >>> usage at compile time. Given that we can already selectively > >>> export packages to some chosen modules which hide them for the > >>> rest of the other modules at compile > >> time, > >>> adding dynamic to the spec adds little value. > >>> > >>> And it has several drawbacks, the main one is that it weaken > >>> the compiletime/runtime fidelity story, with a dynamic export > >>> the runtime > >> world > >>> is not the compiletime world anymore. The main motivation to > >>> add dynamic seems to be for backward compatibility during the > >>> transition between the classpath world and the full module > >> world, > >>> for several other cases, we use the command line for that, i > >>> don't see > >> why > >>> this case is so special that it should not work like the other > >>> compatibility issues. > >>> > >>> Moreover, an agent, an annotation processor and a jlink plugin > >>> are all able to implement the same trick, add export after the > >>> compilation and before the runtime, so i don't think it's a > >>> good idea to change the spec for that. > >>> > >>> R?mi > >>> > >>>> > >>>> > >>>> [1] > >>>> > >>> > >> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccess > ToNonExportedTypes > >>>> > >> > [2] > >>>> > >>> > >> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithou > tReadability > >>>> > >> > [3] > >>> http://openjdk.java.net/projects/jigsaw/spec/sotms/#qualified-export > s > >>>> > >>> > [4] > >>>> > >>> > >> http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-Decembe > r/000205.html > >>>> > >> > [5] > >>>> > >>> > >> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/00574 > 5.html > >>>> > >>> > >> > > > >> > -- > > PGP Key: > http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 > > Web: > http://codefx.org > a blog about software development > http://do-foss.de > Free and Open Source Software for the City of Dortmund > > Twitter: > https://twitter.com/nipafx > From pbenedict at apache.org Thu Jun 30 22:33:13 2016 From: pbenedict at apache.org (Paul Benedict) Date: Thu, 30 Jun 2016 17:33:13 -0500 Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` In-Reply-To: <57759612.2080902@oracle.com> References: <20160628211815.87E88B59C4@eggemoggin.niobe.net> <577588EA.6060202@oracle.com> <57759612.2080902@oracle.com> Message-ID: As I have opined in an email a few days back [1], listing out who can reflect over you is counterintuitive. The proposed "dynamic" feature only works when you believe your analysis is sound, and you can absolutely identify who may reflect over you. If after you publish to the world, and find out that you're wrong, you're completely hosed forever. There's no way to go back into the published library to fix the mistake -- you'd have to publish a new version with an updated module descriptor.... (which, by the way, ties in to David Llyod's frequent objection that the descriptor should be something that is easily editable... and by that, it probably needs to be OUTSIDE the module so as not to affect the SHA1 checksums of published artifacts.... but I digress.) To my main point, it's not really reasonable to know who may want to reflect your internals. If you're publishing yourself as a common library serving as middleware, how can you guess at that? You just can't. I fully believe that the compiler and JRE should respect the inability to use non-exported types under normal circumstances, but reflection is not a normal circumstance. To enforce this restriction on reflection is very much unpalatable because it puts developers in a guessing game. The proposed "dynamic" is asking programmers to divine how their application will be consumed in the future. It's just not possible. Alex, in your counter example, pretend "myapp" has been published to the Maven repository in 2017 and ten years has passed. In 2017, it's great that "myapp" can use "opengamma.strata", but it becomes absolutely useless to "opengamma.strata2" in 2027 despite the fact "myapp" should be perfectly usable. See the problem? Let's take this example closer to the real world: Struts: (1.x) org.apache.struts (2.x) org.apache.struts2 (3.x) ??? Log4J: (1.x) org.apache.log4j (2.x) org.apache.logging.log4j (3.x) ??? JBoss Community/AS: (<7) org.jboss (8-11) org.wildfly (12+) ??? * not implying a change but not presuming either [1] http://mail.openjdk.java.net/pipermail/jpms-spec-observers/2016-June/000416.html Cheers, Paul On Thu, Jun 30, 2016 at 4:58 PM, Alex Buckley wrote: > On 6/30/2016 2:29 PM, Stephen Colebourne wrote: > >> To take one example from OpenGamma Strata: >> >> https://github.com/OpenGamma/Strata/blob/master/modules/collect/src/main/java/com/opengamma/strata/collect/named/ExtendedEnum.java#L181 >> > > (Line 50: I think "are" should be "as".) > > (Line 181: I notice that while "constants" and "lookup" providers are > specified in the class-level javadoc, "instance" providers are not. > Admittedly, reflection is used not only for the [unspecified] special field > of an "instance" provider class but also for the [specified] no-args ctor > of a "lookup" provider class.) > > ExtendedEnum is a mechanism to provide enum-like behaviour, where the >> set of constants is fixed at startup, but controlled dynamically based >> on what configuration files are present. One of the options it >> provides is for the configuration file to specify a Class name, which >> is then reflected on to find suitable enum-like constants. >> >> When designing this feature, at no stage did I think that the code >> would not work because some of the constants are in a different >> module. (Application users can add their own configuration files and >> constants, ie, in a separate module). >> >> This proposal essentially requires application users to specifically >> expose these types to reflection where they might otherwise lock them >> down in a non-exported package. While in this case, I could document >> that users of extended enums must make sure their code can be accessed >> if in a different module, it would at the very least be a faff, and a >> source of questions. >> > > Counterpoint: the user's module-info.java is a great place to document the > special relationship between one of their packages and one of their > dependencies. > > module myapp { > requires opengamma.strata; > > // Classes to drive Strata's ExtendedEnum instances > exports dynamic com.foo.app.constants.tickers to opengamma.strata; > } > > Alex > From alex.buckley at oracle.com Thu Jun 30 22:53:43 2016 From: alex.buckley at oracle.com (Alex Buckley) Date: Thu, 30 Jun 2016 15:53:43 -0700 Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` In-Reply-To: References: <20160628211815.87E88B59C4@eggemoggin.niobe.net> <577588EA.6060202@oracle.com> <57759612.2080902@oracle.com> Message-ID: <5775A2F7.4040508@oracle.com> On 6/30/2016 3:33 PM, Paul Benedict wrote: > To my main point, it's not really reasonable to know who may want to > reflect your internals. If you're publishing yourself as a common > library serving as middleware, how can you guess at that? You just > can't. Stephen's case involved a user module whose code is written to follow a specific idiom offered by a library. Since the library specifies that it implements the idiom through reflection, it's fine for the user module to have a qualified 'exports dynamic ... to strata'. > Alex, in your counter example, pretend "myapp" has been published to the > Maven repository in 2017 and ten years has passed. In 2017, it's great > that "myapp" can use "opengamma.strata", but it becomes absolutely > useless to "opengamma.strata2" in 2027 despite the fact "myapp" should > be perfectly usable. See the problem? Remember that module myapp requires opengamma.strata so opengamma.strata2 is irrelevant. If opengamma.strata2 offers a SuperDuperExtendedEnum idiom in 2027, that's great, but myapp's enum provider classes weren't written to follow that idiom, so it's right for myapp to export its tickers package solely to opengamma.strata. Alex > Let's take this example closer to the real world: > > Struts: > (1.x) org.apache.struts > (2.x) org.apache.struts2 > (3.x) ??? > > Log4J: > (1.x) org.apache.log4j > (2.x) org.apache.logging.log4j > (3.x) ??? > > JBoss Community/AS: > (<7) org.jboss > (8-11) org.wildfly > (12+) ??? * not implying a change but not presuming either > > [1] > http://mail.openjdk.java.net/pipermail/jpms-spec-observers/2016-June/000416.html > > Cheers, > Paul > > On Thu, Jun 30, 2016 at 4:58 PM, Alex Buckley > wrote: > > On 6/30/2016 2:29 PM, Stephen Colebourne wrote: > > To take one example from OpenGamma Strata: > https://github.com/OpenGamma/Strata/blob/master/modules/collect/src/main/java/com/opengamma/strata/collect/named/ExtendedEnum.java#L181 > > > (Line 50: I think "are" should be "as".) > > (Line 181: I notice that while "constants" and "lookup" providers > are specified in the class-level javadoc, "instance" providers are > not. Admittedly, reflection is used not only for the [unspecified] > special field of an "instance" provider class but also for the > [specified] no-args ctor of a "lookup" provider class.) > > ExtendedEnum is a mechanism to provide enum-like behaviour, > where the > set of constants is fixed at startup, but controlled dynamically > based > on what configuration files are present. One of the options it > provides is for the configuration file to specify a Class name, > which > is then reflected on to find suitable enum-like constants. > > When designing this feature, at no stage did I think that the code > would not work because some of the constants are in a different > module. (Application users can add their own configuration files and > constants, ie, in a separate module). > > This proposal essentially requires application users to specifically > expose these types to reflection where they might otherwise lock > them > down in a non-exported package. While in this case, I could document > that users of extended enums must make sure their code can be > accessed > if in a different module, it would at the very least be a faff, > and a > source of questions. > > > Counterpoint: the user's module-info.java is a great place to > document the special relationship between one of their packages and > one of their dependencies. > > module myapp { > requires opengamma.strata; > > // Classes to drive Strata's ExtendedEnum instances > exports dynamic com.foo.app.constants.tickers to opengamma.strata; > } > > Alex > > From mark.reinhold at oracle.com Thu Jun 30 23:07:58 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 30 Jun 2016 16:07:58 -0700 Subject: Proposal: #CompileTimeDependences: `requires static` In-Reply-To: References: <20160628211715.84610B59BF@eggemoggin.niobe.net> Message-ID: <20160630160758.482519193eggemoggin.niobe.net> 2016/6/29 2:30:28 -0700, tim_ellison at uk.ibm.com: > mark.reinhold at oracle.com wrote on 28/06/2016 22:17:15: >> >> Issue summary >> ------------- >> >> #CompileTimeDependences --- Provide a means to specify a module >> dependence that is mandatory at compile time but optional at run time, >> for use with libraries that are not strictly necessary but can be >> leveraged if present at run time. (Detail: If a dependence is >> "optional at run time" then does the module system try to satisfy it >> during resolution but fail silently if it cannot, or does it ignore it >> during resolution but add the corresponding read edge to that module if >> it was resolved for some other reason?) [1] >> >> Use cases for this feature include annotations that need not be present >> at run time [2] and "auto-configuring" libraries and frameworks such as >> Joda Beans [3] and Spring [4]. > > For some reason I can't quite explain, this feels like we are giving up > on one of the fundamental completeness assurances of a coherent modular > application. Is this a new failure mode for link/run time modular systems > that has not previously existed? This proposal reflects a fairly common existing practice, at least in Java. Today it's most often expressed via optional dependencies in Maven [1] and carefully-written code that either uses reflection or is prepared to catch the appropriate exceptions in case an optional component is not available at run time. I wouldn't characterize this practice as a failure mode but, rather, a flexible way to configure software systems. > It may be that this proposal satisfies an important compatibility story for > existing code bases, and that is the justification, because on first reading > my reaction is that the code has to be written to deal with the absence of > the optional module anyway, more details below. Yes, that's correct, but see below. >> Proposal >> -------- >> >> Extend the language of module declarations to allow the `static` modifier >> to be used on a `requires` directive, with the following meanings: >> >> - At compile time, `requires static M` expresses a mandatory >> dependence. It is an error if a suitable module cannot be found >> amongst the observable modules and resolved. >> >> - In phases after compile time, `requires static M` expresses an >> optional dependence. The module system will not search the >> observable modules for a suitable module during resolution, but >> if the resulting module graph contains a suitable module then it >> will add the appropriate readability edge prior to doing the usual >> post-resolution sanity checks. (The parenthetical question in the >> issue summary is, in other words, answered in the negative.) > > I read the parenthetical question as an either-or ... Quite right -- I should've written "answered by taking the second alternative". >> ... >> >> The code in `joda.beans` that refers to types in `joda.collect` must, of >> course, be written defensively so that it fails gracefully at run time >> when the `joda.collect` module is not present. > > This is a key point. Since the application code has to be robust in the face > of a missing "requires static" module, wouldn't it be better served by coding > the behvior of the required module as a service that has a "fail gracefully" > implementation and can be replaced by a productive implementation if required > at link/run time? Some uses of compile-time dependences could be better expressed as services, but that's not always the case. If you're trying to leverage a large third-party library (e.g., Guava) when it's present at run time, but can live without it when it's not, then doing that with services would require you to abstract the Guava functionality that you want to use into one or more service types, which is apt to be awkward at best. The situation gets even worse with large auto-configuring frameworks such as Spring [2]. >> ... >> >> Notes >> ----- >> >> - In phases after compile time, why doesn't the module system search >> the observable modules in an attempt to satisfy an optional >> dependence? >> >> This would not be difficult to arrange but it could lead to surprising >> behavior in which optional dependences silently cause many additional >> modules to be resolved. Given the use cases we have it does not seem >> unreasonable to expect an end user to indicate explicitly, either in >> source code or out-of-band (e.g., with a command-line option such as >> `-addmods`), that an otherwise-optional module is actually needed. > > Another option would be to provide a command-line option that explicitly allows > the linker/launcher to ignore missing modules that are required where there is > an assurance that it will not actually be required in the particular context; > that is, if I design a single module with "requires" or "requires static" > dependencies if it is used in a stand-alone or container context, for example. > > I'd prefer to fail because of an explicit -ignoremods option on the command > line rather than a missing -addmods. The existing common practice with JAR files is to trust end users to place desired optional JAR files on their run-time class paths. The current proposal reflects that practice, by asking end users to add desired optional modules via -addmods. We could reverse the sense of all this, change the resolver to try to satisfy optional dependences at link time and run time, and tell people to use -ignoremods when they specifically don't want an optional dependence to be resolved, but I think that would be significantly harder to explain and to understand, and hence a burden to migration. - Mark [1] https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html [2] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005814.html From alex.buckley at oracle.com Thu Jun 30 23:12:37 2016 From: alex.buckley at oracle.com (Alex Buckley) Date: Thu, 30 Jun 2016 16:12:37 -0700 Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` In-Reply-To: References: <20160628211815.87E88B59C4@eggemoggin.niobe.net> <1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr> <577590E2.7020907@codefx.org> Message-ID: <5775A765.7090301@oracle.com> On 6/30/2016 3:20 PM, Rafael Winterhalter wrote: > I do not think that shielding internals is a problem with regards to > reflection. You want to shield internals in order to not break dependants > by prohibiting their existance. With reflection, you do however typically > react to the actual implementation by discovering it at runtime. If your > use pattern was fixed, you would always require a static dependency over > reflection in the first place. I do neither think that reflection only > interests internals. For example, maybe you use Hibernate for your > persistance layer without exposing those DTOs from your module. The problem > is that you invert responsibilities. I do not care about how Hibernate > works as long as it does work. Almost any library uses reflection and I > would not want to dig into implementation details to find out how to set up > my dynamic exports. I agree that no-one should have to look at a framework's implementation to figure out what to export dynamically. It's up to the framework's specification to describe where it plans to inspect the internals of modules requiring the framework. In Stephen's example, the Strata specification did that admirably. That said, as soon as you mention inverting responsibilities, it becomes clear that services are the answer. Rather than the framework reflecting over the internals of a user module to discover suitable classes, the framework should define a service interface and a user module should provide a service implementation. It seems like this is a style of OO programming that has passed many framework developers by, but there are some big benefits: 1. The user module knows statically that it's following the idiom expected by the framework. There's no "oops, I forgot to name my static field correctly in the DTO class." 2. The framework delegates the boring work of discovery to ServiceLoader. There's no need for framework code to crawl the classpath or modulepath looking for assignable classes. 3. The framework, by no longer working with Class objects, avoids the deathly assumption that it has visibility of the same classes as the user module. Alex > I want to require a module and hopefully the compiler > tells me about anything that's wrong. Reflection access errors would only > be an awkward side effect that can be avoided by not applying modules > alltogether. I think this limitation would therefore hurt Java 9 adoption. > From mark.reinhold at oracle.com Thu Jun 30 23:52:50 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 30 Jun 2016 16:52:50 -0700 Subject: Proposal: #ResourceEncapsulation and #ClassFilesAsResources In-Reply-To: References: <20160628212015.99A06B59CC@eggemoggin.niobe.net> Message-ID: <20160630165250.707143965eggemoggin.niobe.net> 2016/6/29 3:15:38 -0700, tim_ellison at uk.ibm.com: > mark.reinhold at oracle.com wrote on 28/06/2016 22:20:15: >> >> ... >> >> Proposal >> -------- >> >> Drop the agreed resource-encapsulation requirement, which reads: >> >> Resource encapsulation --- The run-time system must ensure that the >> static resource files within a module are directly accessible only by >> code within that module. The existing resource-access APIs should >> continue to work as they do today when used to access a module's own >> resource files. [2] >> >> ... > > The proposal to drop this requirement leaves no way for a module to encapsulate > its resources. Have you considered an alternative that preserves the compatibility > of existing APIs which I agree is desirable to support existing applications, > but introduces a module-level resource API that allows those modules that want > to hide their implementation details? There are certainly other alternatives; what I proposed is just the simplest. One alternative is just to address the two specific use cases. Make META-INF/**, WEB-INF/**, and the class file for any class visible to the caller available via the resource-access APIs, but make all other resources module-private. This wouldn't address other use cases that require non-class-file resources in normal package directories to be available, of which the Wicket web framework is an example (pointed out by Rafael Winterhalter [1]). So a second alternative would be to go further and define a way to declare which packages of a module are considered to "export" their resources, similar to what Peter Levart suggested [2]). Maybe something like this: module com.foo.app { requires wicket.core; requires wicket.utils; publishes com.foo.app.pages.main; publishes com.foo.app.pages.aux; } would make resources in the com/foo/app/pages/{main,aux} directories of the module's artifact available for anyone to load. By fiat we'd make everything under {META,WEB}-INF available, for compatibility. Whether access to the class files for visible classes should be limited further, or perhaps solely, by this mechanism, I'm not sure. So yes, there are some alternatives here. Thoughts welcome. - Mark [1] http://mail.openjdk.java.net/pipermail/jpms-spec-observers/2016-June/000417.html [2] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-June/008355.html