From david.lloyd at redhat.com Tue Jun 21 22:11:36 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Tue, 21 Jun 2016 17:11:36 -0500 Subject: JDK 9 is not (yet) Feature Complete -- how will we get there? In-Reply-To: <20160621145607.146493062eggemoggin.niobe.net> References: <20160610072437.975638333eggemoggin.niobe.net> <20160621145607.146493062eggemoggin.niobe.net> Message-ID: <5769BB98.1000902@redhat.com> On 06/21/2016 04:56 PM, mark.reinhold at oracle.com wrote: > 2016/6/10 7:24:37 -0700, mark.reinhold at oracle.com: >> ... >> >> To manage the remaining JEPs and small enhancements so that we can reach >> the Feature Complete state in a timely fashion I hereby propose the >> following process: >> >> ... >> >> JDK 9 Committers are invited to comment on this process proposal. If no >> serious objections are raised in one week's time, by 15:00 UTC on 17 June >> 2016, then this is the process that we'll use. > > Hearing no objections, this process is now adopted. > > I've summarized it here: > > http://openjdk.java.net/projects/jdk9/fc-extension-process > > Link to JBS query for pending requests: > > http://j.mp/jdk9-fc-pending I have one question about this - how are the remaining JPMS issues fitting in to this process, or does this fall into some different category? -- - DML From mark.reinhold at oracle.com Tue Jun 21 22:37:07 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Tue, 21 Jun 2016 15:37:07 -0700 Subject: JDK 9 is not (yet) Feature Complete -- how will we get there? In-Reply-To: <5769BB98.1000902@redhat.com> References: <20160610072437.975638333eggemoggin.niobe.net> <20160621145607.146493062eggemoggin.niobe.net> <5769BB98.1000902@redhat.com> Message-ID: <20160621153707.716452057eggemoggin.niobe.net> 2016/6/21 15:11:36 -0700, david.lloyd at redhat.com: > On 06/21/2016 04:56 PM, mark.reinhold at oracle.com wrote: >> 2016/6/10 7:24:37 -0700, mark.reinhold at oracle.com: >>> ... >>> >>> To manage the remaining JEPs and small enhancements so that we can reach >>> the Feature Complete state in a timely fashion I hereby propose the >>> following process: >>> >>> ... >>> >>> JDK 9 Committers are invited to comment on this process proposal. If no >>> serious objections are raised in one week's time, by 15:00 UTC on 17 June >>> 2016, then this is the process that we'll use. >> >> Hearing no objections, this process is now adopted. >> >> I've summarized it here: >> >> http://openjdk.java.net/projects/jdk9/fc-extension-process >> >> Link to JBS query for pending requests: >> >> http://j.mp/jdk9-fc-pending > > I have one question about this - how are the remaining JPMS issues > fitting in to this process, or does this fall into some different category? Insofar as the resolution of those issues requires changes in JDK 9 then those changes will be subject to the above process, but I don't expect that to present any difficulties. All of the relevant JEPs should shortly be granted FC extensions. As to the issues themselves, I'm drafting a few proposals which I hope to post here shortly. - Mark From david.lloyd at redhat.com Wed Jun 22 13:03:31 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Wed, 22 Jun 2016 08:03:31 -0500 Subject: Minor confusion on the issues doc Message-ID: <576A8CA3.6060701@redhat.com> The #ReflectionWithoutReadability issue resolution is not indented to the same level as the bullet point for that issue, leading multiple people to ask me how that resolution applies to the whole section. Is there any way to either fix the indent, or barring that, indicate that the resolution applies to just that issue? http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithoutReadability -- - DML From mark.reinhold at oracle.com Thu Jun 23 16:53:52 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 23 Jun 2016 09:53:52 -0700 Subject: Minor confusion on the issues doc In-Reply-To: <576A8CA3.6060701@redhat.com> References: <576A8CA3.6060701@redhat.com> Message-ID: <20160623095352.405957043eggemoggin.niobe.net> 2016/6/22 6:03:31 -0700, david.lloyd at redhat.com: > The #ReflectionWithoutReadability issue resolution is not indented to > the same level as the bullet point for that issue, leading multiple > people to ask me how that resolution applies to the whole section. Is > there any way to either fix the indent, or barring that, indicate that > the resolution applies to just that issue? The issue list is formatted in the same way as the requirements list. Each item is summarized in indented text, with applicable commentary following the item in non-indented text. I can see how that's confusing though, so I'll think about how to fix it in the next revision. - Mark From mark.reinhold at oracle.com Tue Jun 28 21:16:15 2016 From: mark.reinhold at oracle.com (Mark Reinhold) Date: Tue, 28 Jun 2016 14:16:15 -0700 (PDT) Subject: JSR 376 schedule Message-ID: <20160628211615.81862B59BA@eggemoggin.niobe.net> At this point we have quite a number of issues to work through [1]. Rather than spend time generating an Early Draft Review specification I suggest that we instead focus on the issues themselves, aiming to finish up and publish an EDR around the end of the summer. Thus our new schedule would be: Early Draft Review September 2016 Public Review November 2016 Proposed Final Draft January 2017 Final Release March 2017 I realize that this is a bit compressed, but given that the draft specification and prototype RI have been publicly available and regularly updated since last September, and that many developers have sent feedback on them via several different channels, it's not clear to me that doing an EDR any earlier would have had, or would now have, any practical benefit in terms of transparency. Please let me know by this time next week if you have any concerns with this new schedule. In the meantime I'll post specific proposals for some of the open issues in messages that follow, and then update the issue list with links thereto. If you comment on one of these proposals then please either reply on the existing thread or else include the hashtag of the relevant issue(s) in the subject line of a new thread, to simplify tracking. - Mark [1] http://openjdk.java.net/projects/jigsaw/spec/issues/ From mark.reinhold at oracle.com Tue Jun 28 21:17:15 2016 From: mark.reinhold at oracle.com (Mark Reinhold) Date: Tue, 28 Jun 2016 14:17:15 -0700 (PDT) Subject: Proposal: #CompileTimeDependences: `requires static` Message-ID: <20160628211715.84610B59BF@eggemoggin.niobe.net> Issue summary ------------- #CompileTimeDependences --- Provide a means to specify a module dependence that is mandatory at compile time but optional at run time, for use with libraries that are not strictly necessary but can be leveraged if present at run time. (Detail: If a dependence is "optional at run time" then does the module system try to satisfy it during resolution but fail silently if it cannot, or does it ignore it during resolution but add the corresponding read edge to that module if it was resolved for some other reason?) [1] Use cases for this feature include annotations that need not be present at run time [2] and "auto-configuring" libraries and frameworks such as Joda Beans [3] and Spring [4]. Proposal -------- Extend the language of module declarations to allow the `static` modifier to be used on a `requires` directive, with the following meanings: - At compile time, `requires static M` expresses a mandatory dependence. It is an error if a suitable module cannot be found amongst the observable modules and resolved. - In phases after compile time, `requires static M` expresses an optional dependence. The module system will not search the observable modules for a suitable module during resolution, but if the resulting module graph contains a suitable module then it will add the appropriate readability edge prior to doing the usual post-resolution sanity checks. (The parenthetical question in the issue summary is, in other words, answered in the negative.) Thus a hypothetical module declaration of the form module joda.beans { requires static joda.collect; ... } would ensure that the `joda.collect` module is available at compile time, so that code in the `joda.beans` module that refers to `joda.collect` can be compiled without any fuss. It would not, however, guarantee that `joda.collect` is available at link time or run time. A user who wants to use `joda.beans` together with `joda.collect` in those later phases must ensure that `joda.collect` is added to the module graph by, e.g., declaring a dependence upon it in some other module, or by using the `-addmods` option or its equivalent. The code in `joda.beans` that refers to types in `joda.collect` must, of course, be written defensively so that it fails gracefully at run time when the `joda.collect` module is not present. It is possible to combine the `public` and `static` modifiers on a `requires` directive: module joda.beans { requires static joda.collect; requires public static freemarker; ... } This is useful in this particular case since `joda.beans` defines two exported public classes that extend classes defined in the `freemarker` module, and so those classes are logically part of the `joda.beans` API. In general, however, this idiom leads to fragile APIs and is therefore not advised except when dealing with legacy code. Notes ----- - In phases after compile time, why doesn't the module system search the observable modules in an attempt to satisfy an optional dependence? This would not be difficult to arrange but it could lead to surprising behavior in which optional dependences silently cause many additional modules to be resolved. Given the use cases we have it does not seem unreasonable to expect an end user to indicate explicitly, either in source code or out-of-band (e.g., with a command-line option such as `-addmods`), that an otherwise-optional module is actually needed. [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#CompileTimeDependences [2] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-October/004915.html [3] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005462.html [4] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005814.html From mark.reinhold at oracle.com Tue Jun 28 21:18:15 2016 From: mark.reinhold at oracle.com (Mark Reinhold) Date: Tue, 28 Jun 2016 14:18:15 -0700 (PDT) Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` Message-ID: <20160628211815.87E88B59C4@eggemoggin.niobe.net> Issue summary ------------- #ReflectiveAccessToNonExportedTypes --- Some kinds of framework libraries require reflective access to members of the non-exported types of other modules; examples include dependency injection (Guice), persistence (JPA), debugging tools, code-automation tools, and serialization (XStream). In some cases the particular library to be used is not known until run time (e.g., Hibernate and EclipseLink both implement JPA). This capability is also sometimes used to work around bugs in unchangeable code. Access to non-exported packages can, at present, only be done via command-line flags, which is extremely awkward. Provide an easier way for reflective code to access such non-exported types. [1] Proposal -------- Extend the language of module declarations so that a package can be declared to be exported at run time but not at compile time. This is, roughly, the dual of the `requires static` construct proposed for #CompileTimeDependences, hence we propose to introduce a new modifier, `dynamic`, for use on the `exports` directive. It has the following meanings: - At compile time, `exports dynamic P` does not cause the package `P` to be exported, though it does require `P` to be a package defined in the module. - In phases after compile time, `exports dynamic P` behaves in exactly the same way as `exports P`. It therefore takes part fully in resolution and configuration, and is subject to the same consistency constraints as normally-exported packages (e.g., no split packages). Thus a module declaration of the form module com.foo.app { requires hibernate.core; requires hibernate.entitymanager; exports dynamic com.foo.app.model; } makes the types in the `com.foo.app.model` package accessible at run time but not at compile time. In combination with the earlier change to enable #ReflectionWithoutReadability [2] this means that frameworks that today use core reflection to manipulate user classes at run time are more likely to work out-of-the-box, without change, as automatic modules. If the `com.foo.app.model` package in this example includes entity classes to be managed by Hibernate then the framework will be able to access them without further ado, but under normal circumstances an attempt to compile code that refers directly to those classes will fail. The `dynamic` modifier can be applied to both unqualified and qualified `exports` directives, though the caveats on using qualified exports [3] still apply. Notes ----- - To access a non-public member in a dynamically-exported package you must still invoke the appropriate `setAccessible` method, just as you do today to access non-public members in an exported package. - This proposal is similar to R??mi's suggestion to add a way to export a package only for reflection, still requiring that `setAccessible` be invoked [4]. This proposal differs in that it does not require `setAccessible` to be invoked to access public members in a dynamically-exported package. - This proposal primarily addresses "friendly" uses of reflection, such as dependency injection and persistence, in which the author of a module knows in advance that one or more packages must be exported at run time for reflective access by frameworks. This proposal is also convenient for frameworks that generate code at run time, since the constant pool of a generated class file can include static references to types in dynamically exported packages. - This proposal opens the door to a category of second-class APIs. If a package is exported dynamically then you can still compile code that refers to types in the package, by using `-XaddExports` or its equivalent at compile time, and it will work as expected at run time. It thus may be useful to use `exports dynamic` for packages that contain legacy APIs whose use is strongly discouraged, e.g., those in the `jdk.unsupported` module of the reference implementation, thereby forcing anyone who wants to compile against them to go to the trouble of using `-XaddExports`. - Intrusive access to arbitrary packages of arbitrary modules by, e.g., debugging tools, will still require the use of sharp knives such as the `-XaddExports` command-line option or its equivalent, or JVM TI. - Using the `-XaddExports` option or its equivalent remains awkward, and sometimes it's the only way out. To ease migration I think it's worth considering some way for an application packaged as a JAR file to include such options in its `MANIFEST.MF` file, as suggested by Simon Nash [5]. I'll create a separate issue for that. - New kinds of resolution failures are possible. If module `A` requires `B`, and `B` requires `C`, then if `B` and `C` both declare some package `P` to be exported dynamically, then a split-package error will be reported. This may surprise, since the packages are just internal implementation details that were exported dynamically so that some framework could access their types. This is not, however, all that different from the kinds of resolution and layer-creation failures that are already possible due to package collisions. It's unlikely to happen all that often in practice, and the fix is straightforward: Just use reverse-DNS or otherwise sufficiently-unique package names, as most people already do. It is worth exploring whether javac can detect at least some of these kinds of collisions and issue appropriate warnings. [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccessToNonExportedTypes [2] http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithoutReadability [3] http://openjdk.java.net/projects/jigsaw/spec/sotms/#qualified-exports [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-December/000205.html [5] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005745.html From mark.reinhold at oracle.com Tue Jun 28 21:19:15 2016 From: mark.reinhold at oracle.com (Mark Reinhold) Date: Tue, 28 Jun 2016 14:19:15 -0700 (PDT) Subject: Proposal: #ModuleAnnotations and #ModuleDeprecation Message-ID: <20160628211915.9706BB59C8@eggemoggin.niobe.net> Issue summaries --------------- #ModuleAnnotations --- Should it be possible to annotate module declarations? As previously discussed this is technically feasible but it would add significant complexity. [1] #ModuleDeprecation -- If module declarations cannot be annotated, should there be some other way to deprecate modules? [2] Proposal -------- Allow arbitrary annotations to be placed on module declarations, and revise the `java.lang.Deprecated` annotation so that it can be applied to modules. - Add a new value, `MODULE`, to `java.lang.annotation.ElementType`, to denote the top level of a module declaration. - Extend `javax.lang.model` and `java.lang.reflect` to provide access to module annotations. Repeating annotations are supported, subject to the usual rules for repeatable annotation types. (This proposal will not add similar support to `java.lang.module.ModuleDescriptor`; that would add yet more complexity, and we haven't seen a use case for it.) - Annotations on modules will never affect the operation of the module system itself. - Revise `java.lang.Deprecated` and `java.lang.SuppressWarnings` to allow `ElementType.MODULE` as a target. For now, no other existing Java SE annotations will apply to modules. Also, allow `import` declarations [3] to be used in module compilation units. - Any `import` declarations must appear before the `module` declaration. - As a consequence of supporting `import` declarations, simple type names can now be used in the `uses` and `provides` directives of a module declaration. Type annotations on these type names are not supported. Rationale --------- When we first discussed this topic I wrote [4]: > We haven't seen a compelling use case (so far) for annotations on module > declarations, or on clauses in module declarations, and they aren't a > documented requirement, so for simplicity we left them out of the initial > design and the prototype. The compelling use case is, of course, deprecation, as R??mi pointed out [5]. In Java SE 9, in particular, we very much want to be able to write something like this: @Deprecated(since = "9", forRemoval = true) module java.corba { ... } The most natural way to do this is to allow module declarations to be annotated in the same way as other top-level declarations (i.e., packages and classes). If we don't allow the `@Deprecated` annotation to be used on modules then we'd have to invent some other way to deprecate modules (another new keyword?), and that would just be confusing. Like most language features, annotations have sometimes been over-used and even abused [6]. Allowing modules to be annotated opens up a whole new avenue for unfortunate design decisions that lead to unreadable code. We could try to limit the damage by allowing just a few annotations on module declarations (e.g., only those in the `java.lang` package), or by not supporting `import` declarations so that the use of non-platform annotations is discouragingly verbose. Such an approach would lead to unpleasant asymmetries in the language, however, so instead we propose to support arbitrary annotations on module declarations in the obvious way. I think the guidance I wrote earlier [4] still holds. Annotations are useful for extra-linguistic statements, such as deprecation, that make sense in source code and can usefully be carried through the tool chain into the resulting class file. I still don't think they're appropriate for information that is more appropriately the domain of build systems, e.g., version strings. Notes ----- - Should it be possible to annotate the individual directives in a module declaration? This would allow you to deprecate individual exports and requires-public directives, which could be useful in the process of evolving an API: If you plan to remove a package, or remove a public dependence upon some other module, then you could deprecate the corresponding `exports` or `requires public` directive (presumably with `forRemoval = true`) to warn your users that the API is going to change. There are no cases in the JDK itself where we need this ability, at least not desperately. Designing and implementing this would be a non-trivial exercise, and it is a separable feature that could be added later on if justified by experience. [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleAnnotations [2] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleDeprecation [3] http://docs.oracle.com/javase/specs/jls/se8/html/jls-7.html#jls-7.5 [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-October/000153.html [5] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-March/000263.html [6] http://annotatiomania.com/ From mark.reinhold at oracle.com Tue Jun 28 21:20:15 2016 From: mark.reinhold at oracle.com (Mark Reinhold) Date: Tue, 28 Jun 2016 14:20:15 -0700 (PDT) Subject: Proposal: #ResourceEncapsulation and #ClassFilesAsResources Message-ID: <20160628212015.99A06B59CC@eggemoggin.niobe.net> Issue summaries --------------- #ClassFilesAsResources --- If a type is visible and was loaded from a class file then it should be possible to read that file by invoking the `getResourceAsStream` method of the type's class loader, as it is in earlier releases. [1] #ResourceEncapsulation --- The `Module::getResourceAsStream` method can be used to read the resources of any named module, without restriction, which violates the resource-encapsulation requirement [2]. This method should be restricted somehow so that only "suitably-privileged" code (for some definition of that term) can access resources in a named module other than its own. An alternative is to drop this requirement. [3] Proposal -------- Drop the agreed resource-encapsulation requirement, which reads: Resource encapsulation --- The run-time system must ensure that the static resource files within a module are directly accessible only by code within that module. The existing resource-access APIs should continue to work as they do today when used to access a module's own resource files. [2] Make the following changes to the various `getResource*` methods: - The `ClassLoader::getResource*` methods will first delegate to the loader's parent, if any, and will then search for resources in the named modules defined to the loader, and will then search in the loader's unnamed module, typically defined by a class path. In the case of the `getResources` method the order of the resources in the returned enumeration will be unspecified, except that the values of `cl.getResources(name).nextElement()` and `cl.getResource(name)` will be equal. - The `Class::getResource*` methods will only search for resources in the module that defines the class. In the case of a class in an unnamed module this will typically result in searching the class path of the corresponding class loader. The `java.lang.reflect.Module::getResourceAsStream` method will remain, so that it's possible to look up a resource in a module without having a reference to a class defined in that module. It will always be the case that, for a given a class `c`, `c.getResourceAsStream(name)` will be equivalent to `c.getModule().getResourceAsStream(name)`. Rationale --------- The encapsulation of resources was intended to be part of the overall encapsulation story, on the view that a module's resources are just internal implementation details that need not, and should not, be available to code in other modules. Allowing external code to depend upon a module's internal resources is just as problematic as allowing such code to depend upon internal APIs. The reality of existing code today, however, is rather different, with two common use cases: - Resources are often used intentionally to convey information to external code, e.g., as a way to publish configuration files such as `persistence.xml`. This sort of thing should, ideally, be done via services [4][5], but that is not the current practice and it will take years to migrate to it. - Various popular tools and byte-code manipulation frameworks expect to be able to load class files as resources, i.e., to access the class file that defines a class `p.q.C` by invoking `getResource("p.q.C")` on that class's loader. With the API as it stands today these use cases can still be supported, but the code that looks up the resource must be converted to use the `Module::getResourceAsStream` method. This is a barrier to migration, since it means that fewer existing frameworks will work out-of-the-box as automatic modules. We should not drop agreed requirements lightly, but now that we have some practical experience with an implementation of this one I think it's clear that the pain outweighs the gain. [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ClassFilesAsResources [2] http://openjdk.java.net/projects/jigsaw/spec/reqs/#resource-encapsulation [3] http://openjdk.java.net/projects/jigsaw/spec/issues/#ResourceEncapsulation [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-October/000163.html [5] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-December/000207.html [6] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-September/004486.html [7] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-November/005240.html From mark.reinhold at oracle.com Tue Jun 28 21:21:15 2016 From: mark.reinhold at oracle.com (Mark Reinhold) Date: Tue, 28 Jun 2016 14:21:15 -0700 (PDT) Subject: Proposal: #ReflectiveAccessByInstrumentationAgents Message-ID: <20160628212115.9BDC4B59D0@eggemoggin.niobe.net> Issue summary ------------- #ReflectiveAccessByInstrumentationAgents --- As noted in the related issue #ReflectiveAccessToNonExportedTypes [1], the core reflection API cannot be used to gain access to members of packages that are not exported by their defining modules. Even if that issue is addressed for the use cases cited, a distinct use case is that of instrumentation agents that need to enable injected code to invoke non-public methods and access non-public fields. A possible solution for this use case is specifically to provide instrumentation agents with the ability to use core reflection, or some equivalent, even on elements of non-exported packages. [2] Proposal -------- Extend the `java.lang.instrument.Instrumentation` interface, and the corresponding JVM TI native API, to support a `redefineModule` method. This method is roughly analogous to the existing `redefineClasses` method. It will allow instrumentation agents to update an existing module, represented by a `java.lang.reflect.Module` object, to read additional modules, export additional packages, use additional services, and provide additional services. Remove the `addModuleReads` method from the `Instrumentation` interface, and remove the `AddModuleReads` and `CanReadModule` functions from JNI, since they are no longer needed. Notes ----- - This new API will not allow read, export, uses, and provides relationships to be dropped at run time. The use cases do not require this, and implementing it would be very difficult. - A sufficiently-skilled instrumentation-agent developer can already do some of this with the existing API, by injecting new classes into target modules with initializers that invoke the necessary `Module` methods. This is, however, awkward at best. - Objects that implement the `Instrumentation` interface are already very powerful, since they can be used to redefine arbitrary classes. Just as is the case today, in order to preserve platform integrity instrumentation agents must be coded very carefully so that they do not leak such objects to user code. [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccessToNonExportedTypes [2] http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccessByInstrumentationAgents From mark.reinhold at oracle.com Tue Jun 28 21:22:15 2016 From: mark.reinhold at oracle.com (Mark Reinhold) Date: Tue, 28 Jun 2016 14:22:15 -0700 (PDT) Subject: Proposal: #BootstrapClassLoaderSearchInJVMTI Message-ID: <20160628212215.9E7D9B59D4@eggemoggin.niobe.net> Issue summary ------------- #BootstrapClassLoaderSearchInJVMTI --- The JVM TI API [1] defines a function, `AddToBootstrapClassLoaderSearch` [2], by which an instrumentation agent can append a file-system path element to the bootstrap class loader's class path. The bootstrap class path, as such, no longer exists, but this function remains supported for now. Should this function be deprecated, with the intent to remove it in a future release, or should it remain supported? It appears to be critical to the operation of at least one popular instrumentation agent. [3] Proposal -------- Retain this method. Do not deprecate it. [1] http://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html [2] http://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html#AddToBootstrapClassLoaderSearch [3] http://openjdk.java.net/projects/jigsaw/spec/issues/#BootstrapClassLoaderSearchInJVMTI From mark.reinhold at oracle.com Tue Jun 28 21:23:15 2016 From: mark.reinhold at oracle.com (Mark Reinhold) Date: Tue, 28 Jun 2016 14:23:15 -0700 (PDT) Subject: Proposal: #CustomizableAutomaticModuleNameMapping Message-ID: <20160628212315.A247FB59D8@eggemoggin.niobe.net> Issue summary ------------- #CustomizableAutomaticModuleNameMapping --- Provide a means to customize the algorithm that computes the names and version strings of automatic modules. [1] Proposal -------- Extend the `java.lang.module.ModuleDescriptor.Builder` API so that it can be used to create automatic as well as explicit modules. This will allow a container application to use a custom `ModuleFinder` to manage the mapping from artifact names to module names, version strings, and so forth. [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#CustomizableAutomaticModuleNameMapping From forax at univ-mlv.fr Tue Jun 28 22:50:52 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 29 Jun 2016 00:50:52 +0200 (CEST) Subject: Proposal: #CompileTimeDependences: `requires static` In-Reply-To: <20160628211715.84610B59BF@eggemoggin.niobe.net> References: <20160628211715.84610B59BF@eggemoggin.niobe.net> Message-ID: <1808187852.884711.1467154252074.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Mark Reinhold" > ?: jpms-spec-experts at openjdk.java.net > Envoy?: Mardi 28 Juin 2016 23:17:15 > Objet: Proposal: #CompileTimeDependences: `requires static` > > Issue summary > ------------- > > #CompileTimeDependences --- Provide a means to specify a module > dependence that is mandatory at compile time but optional at run time, > for use with libraries that are not strictly necessary but can be > leveraged if present at run time. (Detail: If a dependence is > "optional at run time" then does the module system try to satisfy it > during resolution but fail silently if it cannot, or does it ignore it > during resolution but add the corresponding read edge to that module if > it was resolved for some other reason?) [1] > > Use cases for this feature include annotations that need not be present > at run time [2] and "auto-configuring" libraries and frameworks such as > Joda Beans [3] and Spring [4]. > > Proposal > -------- > > Extend the language of module declarations to allow the `static` modifier > to be used on a `requires` directive, with the following meanings: > > - At compile time, `requires static M` expresses a mandatory > dependence. It is an error if a suitable module cannot be found > amongst the observable modules and resolved. > > - In phases after compile time, `requires static M` expresses an > optional dependence. The module system will not search the > observable modules for a suitable module during resolution, but > if the resulting module graph contains a suitable module then it > will add the appropriate readability edge prior to doing the usual > post-resolution sanity checks. (The parenthetical question in the > issue summary is, in other words, answered in the negative.) > > Thus a hypothetical module declaration of the form > > module joda.beans { > requires static joda.collect; > ... > } > > would ensure that the `joda.collect` module is available at compile time, > so that code in the `joda.beans` module that refers to `joda.collect` can > be compiled without any fuss. It would not, however, guarantee that > `joda.collect` is available at link time or run time. A user who wants > to use `joda.beans` together with `joda.collect` in those later phases > must ensure that `joda.collect` is added to the module graph by, e.g., > declaring a dependence upon it in some other module, or by using the > `-addmods` option or its equivalent. > > The code in `joda.beans` that refers to types in `joda.collect` must, of > course, be written defensively so that it fails gracefully at run time > when the `joda.collect` module is not present. > > It is possible to combine the `public` and `static` modifiers on a > `requires` directive: > > module joda.beans { > requires static joda.collect; > requires public static freemarker; > ... > } > > This is useful in this particular case since `joda.beans` defines two > exported public classes that extend classes defined in the `freemarker` > module, and so those classes are logically part of the `joda.beans` API. > In general, however, this idiom leads to fragile APIs and is therefore > not advised except when dealing with legacy code. > > Notes > ----- > > - In phases after compile time, why doesn't the module system search > the observable modules in an attempt to satisfy an optional > dependence? > > This would not be difficult to arrange but it could lead to surprising > behavior in which optional dependences silently cause many additional > modules to be resolved. Given the use cases we have it does not seem > unreasonable to expect an end user to indicate explicitly, either in > source code or out-of-band (e.g., with a command-line option such as > `-addmods`), that an otherwise-optional module is actually needed. Apart the fact that 'static' should be spelled 'optional', there is no reason to reuse static as it doesn't convey the semantics we want, i.e. optional at runtime, i fully agree with this proposal. R?mi > > > [1] > http://openjdk.java.net/projects/jigsaw/spec/issues/#CompileTimeDependences > [2] > http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-October/004915.html > [3] > http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005462.html > [4] > http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005814.html > From mark.reinhold at oracle.com Tue Jun 28 23:31:50 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Tue, 28 Jun 2016 16:31:50 -0700 Subject: Proposal: #CompileTimeDependences: `requires static` In-Reply-To: <1808187852.884711.1467154252074.JavaMail.zimbra@u-pem.fr> References: <20160628211715.84610B59BF@eggemoggin.niobe.net> <1808187852.884711.1467154252074.JavaMail.zimbra@u-pem.fr> Message-ID: <20160628163150.929695476eggemoggin.niobe.net> 2016/6/28 15:50:52 -0700, Remi Forax : > Apart the fact that 'static' should be spelled 'optional', there is no > reason to reuse static as it doesn't convey the semantics we want, > i.e. optional at runtime, i fully agree with this proposal. In this context `static` is intended to mean "at compile time" or, equivalently, before the more "dynamic" phases of run time and link time (the latter of which is more like run time than compile time). I agree that `static` is potentially confusing since its meaning here is different from what `static` means when used on a member of a class. It does, however, fit nicely with its (nearly) dual directive, `exports dynamic`, proposed for #ReflectiveAccessToNonExportedTypes. I think `optional` is a non-starter, since `requires optional` reads as an oxymoron, and it's optional at run time but mandatory at compile time, so in what sense is it, really, "optional"? Suggestions of other alternatives are welcome ... - Mark From forax at univ-mlv.fr Wed Jun 29 08:51:16 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 29 Jun 2016 10:51:16 +0200 (CEST) Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` In-Reply-To: <20160628211815.87E88B59C4@eggemoggin.niobe.net> References: <20160628211815.87E88B59C4@eggemoggin.niobe.net> Message-ID: <1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Mark Reinhold" > ?: jpms-spec-experts at openjdk.java.net > Envoy?: Mardi 28 Juin 2016 23:18:15 > Objet: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` > > Issue summary > ------------- > > #ReflectiveAccessToNonExportedTypes --- Some kinds of framework > libraries require reflective access to members of the non-exported > types of other modules; examples include dependency injection (Guice), > persistence (JPA), debugging tools, code-automation tools, and > serialization (XStream). In some cases the particular library to be > used is not known until run time (e.g., Hibernate and EclipseLink both > implement JPA). This capability is also sometimes used to work around > bugs in unchangeable code. Access to non-exported packages can, at > present, only be done via command-line flags, which is extremely > awkward. Provide an easier way for reflective code to access such > non-exported types. [1] > > Proposal > -------- > > Extend the language of module declarations so that a package can be > declared to be exported at run time but not at compile time. This is, > roughly, the dual of the `requires static` construct proposed for > #CompileTimeDependences, hence we propose to introduce a new modifier, > `dynamic`, for use on the `exports` directive. It has the following > meanings: > > - At compile time, `exports dynamic P` does not cause the package `P` > to be exported, though it does require `P` to be a package defined > in the module. > > - In phases after compile time, `exports dynamic P` behaves in exactly > the same way as `exports P`. It therefore takes part fully in > resolution and configuration, and is subject to the same consistency > constraints as normally-exported packages (e.g., no split packages). > > Thus a module declaration of the form > > module com.foo.app { > requires hibernate.core; > requires hibernate.entitymanager; > exports dynamic com.foo.app.model; > } > > makes the types in the `com.foo.app.model` package accessible at run time > but not at compile time. In combination with the earlier change to > enable #ReflectionWithoutReadability [2] this means that frameworks that > today use core reflection to manipulate user classes at run time are more > likely to work out-of-the-box, without change, as automatic modules. If > the `com.foo.app.model` package in this example includes entity classes > to be managed by Hibernate then the framework will be able to access them > without further ado, but under normal circumstances an attempt to compile > code that refers directly to those classes will fail. > > The `dynamic` modifier can be applied to both unqualified and qualified > `exports` directives, though the caveats on using qualified exports [3] > still apply. > > Notes > ----- > > - To access a non-public member in a dynamically-exported package you > must still invoke the appropriate `setAccessible` method, just as > you do today to access non-public members in an exported package. > > - This proposal is similar to R?mi's suggestion to add a way to export > a package only for reflection, still requiring that `setAccessible` > be invoked [4]. This proposal differs in that it does not require > `setAccessible` to be invoked to access public members in a > dynamically-exported package. > > - This proposal primarily addresses "friendly" uses of reflection, such > as dependency injection and persistence, in which the author of a > module knows in advance that one or more packages must be exported at > run time for reflective access by frameworks. This proposal is also > convenient for frameworks that generate code at run time, since the > constant pool of a generated class file can include static references > to types in dynamically exported packages. > > - This proposal opens the door to a category of second-class APIs. If > a package is exported dynamically then you can still compile code > that refers to types in the package, by using `-XaddExports` or its > equivalent at compile time, and it will work as expected at run time. > It thus may be useful to use `exports dynamic` for packages that > contain legacy APIs whose use is strongly discouraged, e.g., those in > the `jdk.unsupported` module of the reference implementation, thereby > forcing anyone who wants to compile against them to go to the trouble > of using `-XaddExports`. > > - Intrusive access to arbitrary packages of arbitrary modules by, e.g., > debugging tools, will still require the use of sharp knives such as > the `-XaddExports` command-line option or its equivalent, or JVM TI. > > - Using the `-XaddExports` option or its equivalent remains awkward, > and sometimes it's the only way out. To ease migration I think it's > worth considering some way for an application packaged as a JAR file > to include such options in its `MANIFEST.MF` file, as suggested by > Simon Nash [5]. I'll create a separate issue for that. > > - New kinds of resolution failures are possible. If module `A` > requires `B`, and `B` requires `C`, then if `B` and `C` both declare > some package `P` to be exported dynamically, then a split-package > error will be reported. This may surprise, since the packages are > just internal implementation details that were exported dynamically > so that some framework could access their types. This is not, > however, all that different from the kinds of resolution and > layer-creation failures that are already possible due to package > collisions. It's unlikely to happen all that often in practice, > and the fix is straightforward: Just use reverse-DNS or otherwise > sufficiently-unique package names, as most people already do. It > is worth exploring whether javac can detect at least some of these > kinds of collisions and issue appropriate warnings. I would like to agree on that proposal only on the ground that it requires to change the classfile format thus allows in the same time to fix the issues of the current classfile spec (encoding of the name module as a Class instead of UTF8, wrong flag for requires public), but i'm not sure this proposal support it's own weight. Without dynamic, a 'classical' export works the same way at runtime so adding 'dynamic' to an export only restrict the usage at compile time. Given that we can already selectively export packages to some chosen modules which hide them for the rest of the other modules at compile time, adding dynamic to the spec adds little value. And it has several drawbacks, the main one is that it weaken the compiletime/runtime fidelity story, with a dynamic export the runtime world is not the compiletime world anymore. The main motivation to add dynamic seems to be for backward compatibility during the transition between the classpath world and the full module world, for several other cases, we use the command line for that, i don't see why this case is so special that it should not work like the other compatibility issues. Moreover, an agent, an annotation processor and a jlink plugin are all able to implement the same trick, add export after the compilation and before the runtime, so i don't think it's a good idea to change the spec for that. R?mi > > > [1] > http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccessToNonExportedTypes > [2] > http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithoutReadability > [3] http://openjdk.java.net/projects/jigsaw/spec/sotms/#qualified-exports > [4] > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-December/000205.html > [5] > http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005745.html > From forax at univ-mlv.fr Wed Jun 29 09:08:38 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 29 Jun 2016 11:08:38 +0200 (CEST) Subject: Proposal: #ModuleAnnotations and #ModuleDeprecation In-Reply-To: <20160628211915.9706BB59C8@eggemoggin.niobe.net> References: <20160628211915.9706BB59C8@eggemoggin.niobe.net> Message-ID: <1683195038.1023897.1467191318948.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Mark Reinhold" > ?: jpms-spec-experts at openjdk.java.net > Envoy?: Mardi 28 Juin 2016 23:19:15 > Objet: Proposal: #ModuleAnnotations and #ModuleDeprecation > > Issue summaries > --------------- > > #ModuleAnnotations --- Should it be possible to annotate module > declarations? As previously discussed this is technically feasible > but it would add significant complexity. [1] > > #ModuleDeprecation -- If module declarations cannot be annotated, > should there be some other way to deprecate modules? [2] > > Proposal > -------- > > Allow arbitrary annotations to be placed on module declarations, and > revise the `java.lang.Deprecated` annotation so that it can be applied > to modules. > > - Add a new value, `MODULE`, to `java.lang.annotation.ElementType`, > to denote the top level of a module declaration. > > - Extend `javax.lang.model` and `java.lang.reflect` to provide access > to module annotations. Repeating annotations are supported, subject > to the usual rules for repeatable annotation types. (This proposal > will not add similar support to `java.lang.module.ModuleDescriptor`; > that would add yet more complexity, and we haven't seen a use case > for it.) > > - Annotations on modules will never affect the operation of the module > system itself. > > - Revise `java.lang.Deprecated` and `java.lang.SuppressWarnings` to > allow `ElementType.MODULE` as a target. For now, no other existing > Java SE annotations will apply to modules. > > Also, allow `import` declarations [3] to be used in module compilation > units. > > - Any `import` declarations must appear before the `module` declaration. > > - As a consequence of supporting `import` declarations, simple type > names can now be used in the `uses` and `provides` directives of a > module declaration. Type annotations on these type names are not > supported. > > Rationale > --------- > > When we first discussed this topic I wrote [4]: > > > We haven't seen a compelling use case (so far) for annotations on module > > declarations, or on clauses in module declarations, and they aren't a > > documented requirement, so for simplicity we left them out of the initial > > design and the prototype. > > The compelling use case is, of course, deprecation, as R?mi pointed out > [5]. In Java SE 9, in particular, we very much want to be able to write > something like this: > > @Deprecated(since = "9", forRemoval = true) > module java.corba { > ... > } > > The most natural way to do this is to allow module declarations to be > annotated in the same way as other top-level declarations (i.e., packages > and classes). If we don't allow the `@Deprecated` annotation to be used > on modules then we'd have to invent some other way to deprecate modules > (another new keyword?), and that would just be confusing. > > Like most language features, annotations have sometimes been over-used > and even abused [6]. Allowing modules to be annotated opens up a whole > new avenue for unfortunate design decisions that lead to unreadable code. > We could try to limit the damage by allowing just a few annotations on > module declarations (e.g., only those in the `java.lang` package), or by > not supporting `import` declarations so that the use of non-platform > annotations is discouragingly verbose. Such an approach would lead to > unpleasant asymmetries in the language, however, so instead we propose to > support arbitrary annotations on module declarations in the obvious way. > > I think the guidance I wrote earlier [4] still holds. Annotations are > useful for extra-linguistic statements, such as deprecation, that make > sense in source code and can usefully be carried through the tool chain > into the resulting class file. I still don't think they're appropriate > for information that is more appropriately the domain of build systems, > e.g., version strings. > > Notes > ----- > > - Should it be possible to annotate the individual directives in a > module declaration? This would allow you to deprecate individual > exports and requires-public directives, which could be useful in the > process of evolving an API: If you plan to remove a package, or > remove a public dependence upon some other module, then you could > deprecate the corresponding `exports` or `requires public` directive > (presumably with `forRemoval = true`) to warn your users that the > API is going to change. > > There are no cases in the JDK itself where we need this ability, at least > not desperately. Designing and implementing this would be a non-trivial > exercise, and it is a separable feature that could be added later on if > justified by experience. I like both proposals, adding annotations and imports on module. I disagree that those annotations should not be visible at runtime, i don't see why we need a special case here, it's not complex to implement because the VM already provide the hook to access to those annotations at runtime, @Deprecated is an annotation visible at runtime for a good reason, it allows by example to implement policy like disallowing or emit a warning when calling a deprecated code at runtime. R?mi > > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleAnnotations > [2] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleDeprecation > [3] http://docs.oracle.com/javase/specs/jls/se8/html/jls-7.html#jls-7.5 > [4] > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-October/000153.html > [5] > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-March/000263.html > [6] http://annotatiomania.com/ > From forax at univ-mlv.fr Wed Jun 29 09:11:12 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 29 Jun 2016 11:11:12 +0200 (CEST) Subject: Proposal: #ReflectiveAccessByInstrumentationAgents In-Reply-To: <20160628212115.9BDC4B59D0@eggemoggin.niobe.net> References: <20160628212115.9BDC4B59D0@eggemoggin.niobe.net> Message-ID: <1551556214.1025252.1467191472326.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Mark Reinhold" > ?: jpms-spec-experts at openjdk.java.net > Envoy?: Mardi 28 Juin 2016 23:21:15 > Objet: Proposal: #ReflectiveAccessByInstrumentationAgents > > Issue summary > ------------- > > #ReflectiveAccessByInstrumentationAgents --- As noted in the related > issue #ReflectiveAccessToNonExportedTypes [1], the core reflection API > cannot be used to gain access to members of packages that are not > exported by their defining modules. Even if that issue is addressed > for the use cases cited, a distinct use case is that of instrumentation > agents that need to enable injected code to invoke non-public methods > and access non-public fields. A possible solution for this use case is > specifically to provide instrumentation agents with the ability to use > core reflection, or some equivalent, even on elements of non-exported > packages. [2] > > Proposal > -------- > > Extend the `java.lang.instrument.Instrumentation` interface, and the > corresponding JVM TI native API, to support a `redefineModule` method. > This method is roughly analogous to the existing `redefineClasses` > method. It will allow instrumentation agents to update an existing > module, represented by a `java.lang.reflect.Module` object, to read > additional modules, export additional packages, use additional services, > and provide additional services. > > Remove the `addModuleReads` method from the `Instrumentation` interface, > and remove the `AddModuleReads` and `CanReadModule` functions from JNI, > since they are no longer needed. > I like this proposal ! R?mi > > Notes > ----- > > - This new API will not allow read, export, uses, and provides > relationships to be dropped at run time. The use cases do not > require this, and implementing it would be very difficult. > > - A sufficiently-skilled instrumentation-agent developer can already do > some of this with the existing API, by injecting new classes into > target modules with initializers that invoke the necessary `Module` > methods. This is, however, awkward at best. > > - Objects that implement the `Instrumentation` interface are already > very powerful, since they can be used to redefine arbitrary classes. > Just as is the case today, in order to preserve platform integrity > instrumentation agents must be coded very carefully so that they do > not leak such objects to user code. > > > [1] > http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccessToNonExportedTypes > [2] > http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccessByInstrumentationAgents > From forax at univ-mlv.fr Wed Jun 29 09:14:21 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 29 Jun 2016 11:14:21 +0200 (CEST) Subject: Proposal: #BootstrapClassLoaderSearchInJVMTI In-Reply-To: <20160628212215.9E7D9B59D4@eggemoggin.niobe.net> References: <20160628212215.9E7D9B59D4@eggemoggin.niobe.net> Message-ID: <466027723.1027081.1467191661253.JavaMail.zimbra@u-pem.fr> I like this proposal. R?mi ----- Mail original ----- > De: "Mark Reinhold" > ?: jpms-spec-experts at openjdk.java.net > Envoy?: Mardi 28 Juin 2016 23:22:15 > Objet: Proposal: #BootstrapClassLoaderSearchInJVMTI > > Issue summary > ------------- > > #BootstrapClassLoaderSearchInJVMTI --- The JVM TI API [1] defines a > function, `AddToBootstrapClassLoaderSearch` [2], by which an > instrumentation agent can append a file-system path element to the > bootstrap class loader's class path. The bootstrap class path, as > such, no longer exists, but this function remains supported for now. > Should this function be deprecated, with the intent to remove it in a > future release, or should it remain supported? It appears to be > critical to the operation of at least one popular instrumentation > agent. [3] > > Proposal > -------- > > Retain this method. Do not deprecate it. > > > [1] http://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html > [2] > http://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html#AddToBootstrapClassLoaderSearch > [3] > http://openjdk.java.net/projects/jigsaw/spec/issues/#BootstrapClassLoaderSearchInJVMTI > From forax at univ-mlv.fr Wed Jun 29 09:14:44 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 29 Jun 2016 11:14:44 +0200 (CEST) Subject: Proposal: #CustomizableAutomaticModuleNameMapping In-Reply-To: <20160628212315.A247FB59D8@eggemoggin.niobe.net> References: <20160628212315.A247FB59D8@eggemoggin.niobe.net> Message-ID: <1490559688.1027428.1467191684507.JavaMail.zimbra@u-pem.fr> A fine idea. R?mi ----- Mail original ----- > De: "Mark Reinhold" > ?: jpms-spec-experts at openjdk.java.net > Envoy?: Mardi 28 Juin 2016 23:23:15 > Objet: Proposal: #CustomizableAutomaticModuleNameMapping > > Issue summary > ------------- > > #CustomizableAutomaticModuleNameMapping --- Provide a means to > customize the algorithm that computes the names and version strings > of automatic modules. [1] > > Proposal > -------- > > Extend the `java.lang.module.ModuleDescriptor.Builder` API so that it > can be used to create automatic as well as explicit modules. This will > allow a container application to use a custom `ModuleFinder` to manage > the mapping from artifact names to module names, version strings, and > so forth. > > > [1] > http://openjdk.java.net/projects/jigsaw/spec/issues/#CustomizableAutomaticModuleNameMapping > From Tim_Ellison at uk.ibm.com Wed Jun 29 09:30:28 2016 From: Tim_Ellison at uk.ibm.com (Tim Ellison) Date: Wed, 29 Jun 2016 10:30:28 +0100 Subject: Proposal: #CompileTimeDependences: `requires static` In-Reply-To: <20160628211715.84610B59BF@eggemoggin.niobe.net> References: <20160628211715.84610B59BF@eggemoggin.niobe.net> Message-ID: mark.reinhold at oracle.com (Mark Reinhold) wrote on 28/06/2016 22:17:15: > Issue summary > ------------- > > #CompileTimeDependences --- Provide a means to specify a module > dependence that is mandatory at compile time but optional at run time, > for use with libraries that are not strictly necessary but can be > leveraged if present at run time. (Detail: If a dependence is > "optional at run time" then does the module system try to satisfy it > during resolution but fail silently if it cannot, or does it ignore it > during resolution but add the corresponding read edge to that module if > it was resolved for some other reason?) [1] > > Use cases for this feature include annotations that need not be present > at run time [2] and "auto-configuring" libraries and frameworks such as > Joda Beans [3] and Spring [4]. For some reason I can't quite explain, this feels like we are giving up on one of the fundamental completeness assurances of a coherent modular application. Is this a new failure mode for link/run time modular systems that has not previously existed? It may be that this proposal satisfies an important compatibility story for existing code bases, and that is the justification, because on first reading my reaction is that the code has to be written to deal with the absence of the optional module anyway, more details below. > Proposal > -------- > > Extend the language of module declarations to allow the `static` modifier > to be used on a `requires` directive, with the following meanings: > > - At compile time, `requires static M` expresses a mandatory > dependence. It is an error if a suitable module cannot be found > amongst the observable modules and resolved. > > - In phases after compile time, `requires static M` expresses an > optional dependence. The module system will not search the > observable modules for a suitable module during resolution, but > if the resulting module graph contains a suitable module then it > will add the appropriate readability edge prior to doing the usual > post-resolution sanity checks. (The parenthetical question in the > issue summary is, in other words, answered in the negative.) I read the parenthetical question as an either-or ... > Thus a hypothetical module declaration of the form > > module joda.beans { > requires static joda.collect; > ... > } > > would ensure that the `joda.collect` module is available at compile time, > so that code in the `joda.beans` module that refers to `joda.collect` can > be compiled without any fuss. It would not, however, guarantee that > `joda.collect` is available at link time or run time. A user who wants > to use `joda.beans` together with `joda.collect` in those later phases > must ensure that `joda.collect` is added to the module graph by, e.g., > declaring a dependence upon it in some other module, or by using the > `-addmods` option or its equivalent. > > The code in `joda.beans` that refers to types in `joda.collect` must, of > course, be written defensively so that it fails gracefully at run time > when the `joda.collect` module is not present. This is a key point. Since the application code has to be robust in the face of a missing "requires static" module, wouldn't it be better served by coding the behvior of the required module as a service that has a "fail gracefully" implementation and can be replaced by a productive implementation if required at link/run time? I guess a weak analogy would be using compile against JARs that have no runtime equivalent, and are used to produce code that can fail at runtime, vs. having a type of service or strategy pattern that encompasses the fail gracefully / be productive implementations. > It is possible to combine the `public` and `static` modifiers on a > `requires` directive: > > module joda.beans { > requires static joda.collect; > requires public static freemarker; > ... > } > > This is useful in this particular case since `joda.beans` defines two > exported public classes that extend classes defined in the `freemarker` > module, and so those classes are logically part of the `joda.beans` API. > In general, however, this idiom leads to fragile APIs and is therefore > not advised except when dealing with legacy code. > > Notes > ----- > > - In phases after compile time, why doesn't the module system search > the observable modules in an attempt to satisfy an optional > dependence? > > This would not be difficult to arrange but it could lead to surprising > behavior in which optional dependences silently cause many additional > modules to be resolved. Given the use cases we have it does not seem > unreasonable to expect an end user to indicate explicitly, either in > source code or out-of-band (e.g., with a command-line option such as > `-addmods`), that an otherwise-optional module is actually needed. Another option would be to provide a command-line option that explicitly allows the linker/launcher to ignore missing modules that are required where there is an assurance that it will not actually be required in the particular context; that is, if I design a single module with "requires" or "requires static" dependencies if it is used in a stand-alone or container context, for example. I'd prefer to fail because of an explicit -ignoremods option on the command line rather than a missing -addmods. Regards, Tim > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/ > #CompileTimeDependences > [2] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-October/004915.html > [3] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/ > 005462.html > [4] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/ > 005814.html > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Tim_Ellison at uk.ibm.com Wed Jun 29 09:45:31 2016 From: Tim_Ellison at uk.ibm.com (Tim Ellison) Date: Wed, 29 Jun 2016 10:45:31 +0100 Subject: Proposal: #ModuleAnnotations and #ModuleDeprecation In-Reply-To: <20160628211915.9706BB59C8@eggemoggin.niobe.net> References: <20160628211915.9706BB59C8@eggemoggin.niobe.net> Message-ID: mark.reinhold at oracle.com (Mark Reinhold) wrote on 28/06/2016 22:19:15: > Issue summaries > --------------- > > #ModuleAnnotations --- Should it be possible to annotate module > declarations? As previously discussed this is technically feasible > but it would add significant complexity. [1] > > #ModuleDeprecation -- If module declarations cannot be annotated, > should there be some other way to deprecate modules? [2] > > Proposal > -------- > > Allow arbitrary annotations to be placed on module declarations, and > revise the `java.lang.Deprecated` annotation so that it can be applied > to modules. > > - Add a new value, `MODULE`, to `java.lang.annotation.ElementType`, > to denote the top level of a module declaration. > > - Extend `javax.lang.model` and `java.lang.reflect` to provide access > to module annotations. Repeating annotations are supported, subject > to the usual rules for repeatable annotation types. (This proposal > will not add similar support to `java.lang.module.ModuleDescriptor`; > that would add yet more complexity, and we haven't seen a use case > for it.) > > - Annotations on modules will never affect the operation of the module > system itself. > > - Revise `java.lang.Deprecated` and `java.lang.SuppressWarnings` to > allow `ElementType.MODULE` as a target. For now, no other existing > Java SE annotations will apply to modules. > > Also, allow `import` declarations [3] to be used in module compilation > units. > > - Any `import` declarations must appear before the `module` declaration. > > - As a consequence of supporting `import` declarations, simple type > names can now be used in the `uses` and `provides` directives of a > module declaration. Type annotations on these type names are not > supported. > > Rationale > --------- > > When we first discussed this topic I wrote [4]: > > > We haven't seen a compelling use case (so far) for annotations on module > > declarations, or on clauses in module declarations, and they aren't a > > documented requirement, so for simplicity we left them out of the initial > > design and the prototype. > > The compelling use case is, of course, deprecation, as R??mi pointed out > [5]. In Java SE 9, in particular, we very much want to be able to write > something like this: > > @Deprecated(since = "9", forRemoval = true) > module java.corba { > ... > } > > The most natural way to do this is to allow module declarations to be > annotated in the same way as other top-level declarations (i.e., packages > and classes). If we don't allow the `@Deprecated` annotation to be used > on modules then we'd have to invent some other way to deprecate modules > (another new keyword?), and that would just be confusing. > > Like most language features, annotations have sometimes been over-used > and even abused [6]. Allowing modules to be annotated opens up a whole > new avenue for unfortunate design decisions that lead to unreadable code. > We could try to limit the damage by allowing just a few annotations on > module declarations (e.g., only those in the `java.lang` package), or by > not supporting `import` declarations so that the use of non-platform > annotations is discouragingly verbose. Such an approach would lead to > unpleasant asymmetries in the language, however, so instead we propose to > support arbitrary annotations on module declarations in the obvious way. > > I think the guidance I wrote earlier [4] still holds. Annotations are > useful for extra-linguistic statements, such as deprecation, that make > sense in source code and can usefully be carried through the tool chain > into the resulting class file. I still don't think they're appropriate > for information that is more appropriately the domain of build systems, > e.g., version strings. > > Notes > ----- > > - Should it be possible to annotate the individual directives in a > module declaration? This would allow you to deprecate individual > exports and requires-public directives, which could be useful in the > process of evolving an API: If you plan to remove a package, or > remove a public dependence upon some other module, then you could > deprecate the corresponding `exports` or `requires public` directive > (presumably with `forRemoval = true`) to warn your users that the > API is going to change. > > There are no cases in the JDK itself where we need this ability, at least > not desperately. Designing and implementing this would be a non-trivial > exercise, and it is a separable feature that could be added later on if > justified by experience. > > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleAnnotations > [2] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleDeprecation > [3] http://docs.oracle.com/javase/specs/jls/se8/html/jls-7.html#jls-7.5 > [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015- > October/000153.html > [5] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016- > March/000263.html > [6] http://annotatiomania.com/ > I'm +1 to this proposal, and agree with all that is written above about rationale and scope (though, of course, you know the module annotations *will* be abused immediately!). Regards, Tim Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Tim_Ellison at uk.ibm.com Wed Jun 29 10:15:38 2016 From: Tim_Ellison at uk.ibm.com (Tim Ellison) Date: Wed, 29 Jun 2016 11:15:38 +0100 Subject: Proposal: #ResourceEncapsulation and #ClassFilesAsResources In-Reply-To: <20160628212015.99A06B59CC@eggemoggin.niobe.net> References: <20160628212015.99A06B59CC@eggemoggin.niobe.net> Message-ID: mark.reinhold at oracle.com (Mark Reinhold) wrote on 28/06/2016 22:20:15: > Issue summaries > --------------- > > #ClassFilesAsResources --- If a type is visible and was loaded from a > class file then it should be possible to read that file by invoking the > `getResourceAsStream` method of the type's class loader, as it is in > earlier releases. [1] > > #ResourceEncapsulation --- The `Module::getResourceAsStream` method can > be used to read the resources of any named module, without restriction, > which violates the resource-encapsulation requirement [2]. This method > should be restricted somehow so that only "suitably-privileged" code > (for some definition of that term) can access resources in a named > module other than its own. An alternative is to drop this > requirement. [3] > > Proposal > -------- > > Drop the agreed resource-encapsulation requirement, which reads: > > Resource encapsulation --- The run-time system must ensure that the > static resource files within a module are directly accessible only by > code within that module. The existing resource-access APIs should > continue to work as they do today when used to access a module's own > resource files. [2] > > Make the following changes to the various `getResource*` methods: > > - The `ClassLoader::getResource*` methods will first delegate to the > loader's parent, if any, and will then search for resources in the > named modules defined to the loader, and will then search in the > loader's unnamed module, typically defined by a class path. In the > case of the `getResources` method the order of the resources in the > returned enumeration will be unspecified, except that the values of > `cl.getResources(name).nextElement()` and `cl.getResource(name)` > will be equal. > > - The `Class::getResource*` methods will only search for resources in > the module that defines the class. In the case of a class in an > unnamed module this will typically result in searching the class > path of the corresponding class loader. > > The `java.lang.reflect.Module::getResourceAsStream` method will remain, > so that it's possible to look up a resource in a module without having a > reference to a class defined in that module. It will always be the case > that, for a given a class `c`, `c.getResourceAsStream(name)` will be > equivalent to `c.getModule().getResourceAsStream(name)`. > > > Rationale > --------- > > The encapsulation of resources was intended to be part of the overall > encapsulation story, on the view that a module's resources are just > internal implementation details that need not, and should not, be > available to code in other modules. Allowing external code to depend > upon a module's internal resources is just as problematic as allowing > such code to depend upon internal APIs. > > The reality of existing code today, however, is rather different, with > two common use cases: > > - Resources are often used intentionally to convey information to > external code, e.g., as a way to publish configuration files such as > `persistence.xml`. This sort of thing should, ideally, be done via > services [4][5], but that is not the current practice and it will > take years to migrate to it. > > - Various popular tools and byte-code manipulation frameworks expect to > be able to load class files as resources, i.e., to access the class > file that defines a class `p.q.C` by invoking `getResource("p.q.C")` > on that class's loader. > > With the API as it stands today these use cases can still be supported, > but the code that looks up the resource must be converted to use the > `Module::getResourceAsStream` method. This is a barrier to migration, > since it means that fewer existing frameworks will work out-of-the-box > as automatic modules. > > We should not drop agreed requirements lightly, but now that we have > some practical experience with an implementation of this one I think > it's clear that the pain outweighs the gain. > > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ClassFilesAsResources > [2] http://openjdk.java.net/projects/jigsaw/spec/reqs/#resource-encapsulation > [3] http://openjdk.java.net/projects/jigsaw/spec/issues/#ResourceEncapsulation > [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015- > October/000163.html > [5] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015- > December/000207.html > [6] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015- > September/004486.html > [7] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-November/ > 005240.html The proposal to drop this requirement leaves no way for a module to encapsulate its resources. Have you considered an alternative that preserves the compatibility of existing APIs which I agree is desirable to support existing applications, but introduces a module-level resource API that allows those modules that want to hide their implementation details? Regards, Tim Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Tim_Ellison at uk.ibm.com Wed Jun 29 10:30:45 2016 From: Tim_Ellison at uk.ibm.com (Tim Ellison) Date: Wed, 29 Jun 2016 11:30:45 +0100 Subject: Proposal: #ReflectiveAccessByInstrumentationAgents In-Reply-To: <20160628212115.9BDC4B59D0@eggemoggin.niobe.net> References: <20160628212115.9BDC4B59D0@eggemoggin.niobe.net> Message-ID: mark.reinhold at oracle.com (Mark Reinhold) wrote on 28/06/2016 22:21:15: > Issue summary > ------------- > > #ReflectiveAccessByInstrumentationAgents --- As noted in the related > issue #ReflectiveAccessToNonExportedTypes [1], the core reflection API > cannot be used to gain access to members of packages that are not > exported by their defining modules. Even if that issue is addressed > for the use cases cited, a distinct use case is that of instrumentation > agents that need to enable injected code to invoke non-public methods > and access non-public fields. A possible solution for this use case is > specifically to provide instrumentation agents with the ability to use > core reflection, or some equivalent, even on elements of non-exported > packages. [2] > > Proposal > -------- > > Extend the `java.lang.instrument.Instrumentation` interface, and the > corresponding JVM TI native API, to support a `redefineModule` method. > This method is roughly analogous to the existing `redefineClasses` > method. It will allow instrumentation agents to update an existing > module, represented by a `java.lang.reflect.Module` object, to read > additional modules, export additional packages, use additional services, > and provide additional services. > > Remove the `addModuleReads` method from the `Instrumentation` interface, > and remove the `AddModuleReads` and `CanReadModule` functions from JNI, > since they are no longer needed. > > > Notes > ----- > > - This new API will not allow read, export, uses, and provides > relationships to be dropped at run time. The use cases do not > require this, and implementing it would be very difficult. > > - A sufficiently-skilled instrumentation-agent developer can already do > some of this with the existing API, by injecting new classes into > target modules with initializers that invoke the necessary `Module` > methods. This is, however, awkward at best. > > - Objects that implement the `Instrumentation` interface are already > very powerful, since they can be used to redefine arbitrary classes. > Just as is the case today, in order to preserve platform integrity > instrumentation agents must be coded very carefully so that they do > not leak such objects to user code. > > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/ > #ReflectiveAccessToNonExportedTypes > [2] http://openjdk.java.net/projects/jigsaw/spec/issues/ > #ReflectiveAccessByInstrumentationAgents I agree with this proposal. As already pointed out, instrumentation agents are a powerful, somewhat specialist case that suits the full set of manipulation capabilities. Regards, Tim Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Tim_Ellison at uk.ibm.com Wed Jun 29 10:45:53 2016 From: Tim_Ellison at uk.ibm.com (Tim Ellison) Date: Wed, 29 Jun 2016 11:45:53 +0100 Subject: Proposal: #BootstrapClassLoaderSearchInJVMTI In-Reply-To: <20160628212215.9E7D9B59D4@eggemoggin.niobe.net> References: <20160628212215.9E7D9B59D4@eggemoggin.niobe.net> Message-ID: mark.reinhold at oracle.com (Mark Reinhold) wrote on 28/06/2016 22:22:15: > Issue summary > ------------- > > #BootstrapClassLoaderSearchInJVMTI --- The JVM TI API [1] defines a > function, `AddToBootstrapClassLoaderSearch` [2], by which an > instrumentation agent can append a file-system path element to the > bootstrap class loader's class path. The bootstrap class path, as > such, no longer exists, but this function remains supported for now. > Should this function be deprecated, with the intent to remove it in a > future release, or should it remain supported? It appears to be > critical to the operation of at least one popular instrumentation > agent. [3] > > Proposal > -------- > > Retain this method. Do not deprecate it. What would be the effect calling this function on a modular runtime when passed a directory search path segment? a modular JAR? a regular JAR? Regards, Tim > [1] http://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html > [2] http://docs.oracle.com/javase/8/docs/platform/jvmti/ > jvmti.html#AddToBootstrapClassLoaderSearch > [3] http://openjdk.java.net/projects/jigsaw/spec/issues/ > #BootstrapClassLoaderSearchInJVMTI > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From david.lloyd at redhat.com Wed Jun 29 12:25:34 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Wed, 29 Jun 2016 07:25:34 -0500 Subject: Proposal: #ModuleAnnotations and #ModuleDeprecation In-Reply-To: <20160628211915.9706BB59C8@eggemoggin.niobe.net> References: <20160628211915.9706BB59C8@eggemoggin.niobe.net> Message-ID: <5773BE3E.7060104@redhat.com> I like the proposal but I agree with R?mi - it would be useful to allow run-time access. On 06/28/2016 04:19 PM, Mark Reinhold wrote: > Issue summaries > --------------- > > #ModuleAnnotations --- Should it be possible to annotate module > declarations? As previously discussed this is technically feasible > but it would add significant complexity. [1] > > #ModuleDeprecation -- If module declarations cannot be annotated, > should there be some other way to deprecate modules? [2] > > Proposal > -------- > > Allow arbitrary annotations to be placed on module declarations, and > revise the `java.lang.Deprecated` annotation so that it can be applied > to modules. > > - Add a new value, `MODULE`, to `java.lang.annotation.ElementType`, > to denote the top level of a module declaration. > > - Extend `javax.lang.model` and `java.lang.reflect` to provide access > to module annotations. Repeating annotations are supported, subject > to the usual rules for repeatable annotation types. (This proposal > will not add similar support to `java.lang.module.ModuleDescriptor`; > that would add yet more complexity, and we haven't seen a use case > for it.) > > - Annotations on modules will never affect the operation of the module > system itself. > > - Revise `java.lang.Deprecated` and `java.lang.SuppressWarnings` to > allow `ElementType.MODULE` as a target. For now, no other existing > Java SE annotations will apply to modules. > > Also, allow `import` declarations [3] to be used in module compilation > units. > > - Any `import` declarations must appear before the `module` declaration. > > - As a consequence of supporting `import` declarations, simple type > names can now be used in the `uses` and `provides` directives of a > module declaration. Type annotations on these type names are not > supported. > > Rationale > --------- > > When we first discussed this topic I wrote [4]: > >> We haven't seen a compelling use case (so far) for annotations on module >> declarations, or on clauses in module declarations, and they aren't a >> documented requirement, so for simplicity we left them out of the initial >> design and the prototype. > > The compelling use case is, of course, deprecation, as R?mi pointed out > [5]. In Java SE 9, in particular, we very much want to be able to write > something like this: > > @Deprecated(since = "9", forRemoval = true) > module java.corba { > ... > } > > The most natural way to do this is to allow module declarations to be > annotated in the same way as other top-level declarations (i.e., packages > and classes). If we don't allow the `@Deprecated` annotation to be used > on modules then we'd have to invent some other way to deprecate modules > (another new keyword?), and that would just be confusing. > > Like most language features, annotations have sometimes been over-used > and even abused [6]. Allowing modules to be annotated opens up a whole > new avenue for unfortunate design decisions that lead to unreadable code. > We could try to limit the damage by allowing just a few annotations on > module declarations (e.g., only those in the `java.lang` package), or by > not supporting `import` declarations so that the use of non-platform > annotations is discouragingly verbose. Such an approach would lead to > unpleasant asymmetries in the language, however, so instead we propose to > support arbitrary annotations on module declarations in the obvious way. > > I think the guidance I wrote earlier [4] still holds. Annotations are > useful for extra-linguistic statements, such as deprecation, that make > sense in source code and can usefully be carried through the tool chain > into the resulting class file. I still don't think they're appropriate > for information that is more appropriately the domain of build systems, > e.g., version strings. > > Notes > ----- > > - Should it be possible to annotate the individual directives in a > module declaration? This would allow you to deprecate individual > exports and requires-public directives, which could be useful in the > process of evolving an API: If you plan to remove a package, or > remove a public dependence upon some other module, then you could > deprecate the corresponding `exports` or `requires public` directive > (presumably with `forRemoval = true`) to warn your users that the > API is going to change. > > There are no cases in the JDK itself where we need this ability, at least > not desperately. Designing and implementing this would be a non-trivial > exercise, and it is a separable feature that could be added later on if > justified by experience. > > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleAnnotations > [2] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleDeprecation > [3] http://docs.oracle.com/javase/specs/jls/se8/html/jls-7.html#jls-7.5 > [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-October/000153.html > [5] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-March/000263.html > [6] http://annotatiomania.com/ > -- - DML From david.lloyd at redhat.com Wed Jun 29 12:37:34 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Wed, 29 Jun 2016 07:37:34 -0500 Subject: Proposal: #CompileTimeDependences: `requires static` In-Reply-To: <20160628163150.929695476eggemoggin.niobe.net> References: <20160628211715.84610B59BF@eggemoggin.niobe.net> <1808187852.884711.1467154252074.JavaMail.zimbra@u-pem.fr> <20160628163150.929695476eggemoggin.niobe.net> Message-ID: <5773C10E.7000108@redhat.com> On 06/28/2016 06:31 PM, mark.reinhold at oracle.com wrote: > 2016/6/28 15:50:52 -0700, Remi Forax : >> Apart the fact that 'static' should be spelled 'optional', there is no >> reason to reuse static as it doesn't convey the semantics we want, >> i.e. optional at runtime, i fully agree with this proposal. I agree that 'static' really doesn't have any existing meaning in this context, so we'd be inventing another connotation for that term. > In this context `static` is intended to mean "at compile time" or, > equivalently, before the more "dynamic" phases of run time and link > time (the latter of which is more like run time than compile time). > > I agree that `static` is potentially confusing since its meaning here > is different from what `static` means when used on a member of a class. > > It does, however, fit nicely with its (nearly) dual directive, `exports > dynamic`, proposed for #ReflectiveAccessToNonExportedTypes. > > I think `optional` is a non-starter, since `requires optional` reads as > an oxymoron, and it's optional at run time but mandatory at compile > time, so in what sense is it, really, "optional"? > > Suggestions of other alternatives are welcome ... Some random ideas: "optional_at_runtime" - hey, it's descriptive. Use "compile" or "compile_time" instead, i.e. indicate where the dependency is required, not where it isn't required. Use "optional" instead of "requires"; this has a grammatical problem though because "requires" is a verb but "optional" is an adjective. You could use "required" instead, but that's weird I guess. Drop "requires" completely and describe dependencies in a different way, like "dependenc[ey] joda.collect run(optional) compile(required)". Annotations? Just let build tools rewrite the descriptor after the fact, as seems inevitable anyway. -- - DML From Tim_Ellison at uk.ibm.com Wed Jun 29 13:01:23 2016 From: Tim_Ellison at uk.ibm.com (Tim Ellison) Date: Wed, 29 Jun 2016 14:01:23 +0100 Subject: Proposal: #CustomizableAutomaticModuleNameMapping In-Reply-To: <20160628212315.A247FB59D8@eggemoggin.niobe.net> References: <20160628212315.A247FB59D8@eggemoggin.niobe.net> Message-ID: mark.reinhold at oracle.com (Mark Reinhold) wrote on 28/06/2016 22:23:15: > Issue summary > ------------- > > #CustomizableAutomaticModuleNameMapping --- Provide a means to > customize the algorithm that computes the names and version strings > of automatic modules. [1] > > Proposal > -------- > > Extend the `java.lang.module.ModuleDescriptor.Builder` API so that it > can be used to create automatic as well as explicit modules. This will > allow a container application to use a custom `ModuleFinder` to manage > the mapping from artifact names to module names, version strings, and > so forth. > > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/ > #CustomizableAutomaticModuleNameMapping Enthusiastic +1 for this proposal. I envisage a number of places from which such auto information may be derived. Regards, Tim Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From david.lloyd at redhat.com Wed Jun 29 13:21:39 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Wed, 29 Jun 2016 08:21:39 -0500 Subject: Proposal: #ResourceEncapsulation and #ClassFilesAsResources In-Reply-To: <20160628212015.99A06B59CC@eggemoggin.niobe.net> References: <20160628212015.99A06B59CC@eggemoggin.niobe.net> Message-ID: <5773CB63.6030308@redhat.com> Responses inline & at bottom. On 06/28/2016 04:20 PM, Mark Reinhold wrote: > Issue summaries > --------------- > > #ClassFilesAsResources --- If a type is visible and was loaded from a > class file then it should be possible to read that file by invoking the > `getResourceAsStream` method of the type's class loader, as it is in > earlier releases. [1] > > #ResourceEncapsulation --- The `Module::getResourceAsStream` method can > be used to read the resources of any named module, without restriction, > which violates the resource-encapsulation requirement [2]. This method > should be restricted somehow so that only "suitably-privileged" code > (for some definition of that term) can access resources in a named > module other than its own. An alternative is to drop this > requirement. [3] > > Proposal > -------- > > Drop the agreed resource-encapsulation requirement, which reads: > > Resource encapsulation --- The run-time system must ensure that the > static resource files within a module are directly accessible only by > code within that module. The existing resource-access APIs should > continue to work as they do today when used to access a module's own > resource files. [2] OK so far. > Make the following changes to the various `getResource*` methods: > > - The `ClassLoader::getResource*` methods will first delegate to the > loader's parent, if any, and will then search for resources in the > named modules defined to the loader, and will then search in the > loader's unnamed module, typically defined by a class path. In the > case of the `getResources` method the order of the resources in the > returned enumeration will be unspecified, except that the values of > `cl.getResources(name).nextElement()` and `cl.getResource(name)` > will be equal. I am really not a fan of unspecified resource order, especially when you can easily derive a stable order from the graph topology. Resources however should always load from the child-most loader first. This isn't a problem in Jigsaw for classes because packages cannot be duplicated between modules (in a visible manner), but it becomes a problem with resources, especially if/when run-time cycles are permitted: you can run into an awkward situation where two mutually dependent modules each may see the other's resource "first" before their own, which is almost always undesirable. > - The `Class::getResource*` methods will only search for resources in > the module that defines the class. In the case of a class in an > unnamed module this will typically result in searching the class > path of the corresponding class loader. Does this also apply to automatic modules? > The `java.lang.reflect.Module::getResourceAsStream` method will remain, > so that it's possible to look up a resource in a module without having a > reference to a class defined in that module. It will always be the case > that, for a given a class `c`, `c.getResourceAsStream(name)` will be > equivalent to `c.getModule().getResourceAsStream(name)`. I believe that in the former case, the "name" was historically relative to the location of the class. Is that behavior changing? > Rationale > --------- > > The encapsulation of resources was intended to be part of the overall > encapsulation story, on the view that a module's resources are just > internal implementation details that need not, and should not, be > available to code in other modules. Allowing external code to depend > upon a module's internal resources is just as problematic as allowing > such code to depend upon internal APIs. > > The reality of existing code today, however, is rather different, with > two common use cases: > > - Resources are often used intentionally to convey information to > external code, e.g., as a way to publish configuration files such as > `persistence.xml`. This sort of thing should, ideally, be done via > services [4][5], but that is not the current practice and it will > take years to migrate to it. In many cases services are too limiting. It is useful for users to have a more powerful, general facility available that can use the same basic mechanism as services, but for arbitrary resources. For example, one case that came up quite recently (on Jigsaw-dev I think) was that it is useful to acquire not the service instance, but the class name, so that it could be constructed in various ways. Services as they are now require another front-end class to encapsulate this kind of variation. Also it is not uncommon for a situation to arise where many classes must be produced for variant services which all have the same basic behavior but differ only in details, which is awkward as well as it can result in large quantities of similar classes. It would be nice if service loading and resource loading continued to follow a similar set of rules. When Jigsaw switched ServiceLoader from a convenience API over resources to a separate concept, it lost the concept of using resources *as* an API component or participant, which is a powerful feature. This change will restore that to some degree, but it is still strange that loading a META-INF/services/com.foo.Service file will have different search semantics than using ServiceLoader. Basically we are taking this capability from users unless they do it the way that Jigsaw wants you to: via ServiceLoader, and no other way; it almost seems like Jigsaw is attempting to take on the characteristics of a container (not a very powerful one though because it only supports linking via services (a pretty thin concept when you think about it), not by (for example) annotations or anything else). Anyway I hope you can extract some kind of cogency from all that. > - Various popular tools and byte-code manipulation frameworks expect to > be able to load class files as resources, i.e., to access the class > file that defines a class `p.q.C` by invoking `getResource("p.q.C")` > on that class's loader. > > With the API as it stands today these use cases can still be supported, > but the code that looks up the resource must be converted to use the > `Module::getResourceAsStream` method. This is a barrier to migration, > since it means that fewer existing frameworks will work out-of-the-box > as automatic modules. > > We should not drop agreed requirements lightly, but now that we have > some practical experience with an implementation of this one I think > it's clear that the pain outweighs the gain. Overall I think this is a step in the right direction, but I think the discussion needs to carry on a bit longer. -- - DML From david.lloyd at redhat.com Wed Jun 29 15:08:40 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Wed, 29 Jun 2016 10:08:40 -0500 Subject: Proposal: #CustomizableAutomaticModuleNameMapping In-Reply-To: <20160628212315.A247FB59D8@eggemoggin.niobe.net> References: <20160628212315.A247FB59D8@eggemoggin.niobe.net> Message-ID: <5773E478.9020903@redhat.com> On 06/28/2016 04:23 PM, Mark Reinhold wrote: > Issue summary > ------------- > > #CustomizableAutomaticModuleNameMapping --- Provide a means to > customize the algorithm that computes the names and version strings > of automatic modules. [1] > > Proposal > -------- > > Extend the `java.lang.module.ModuleDescriptor.Builder` API so that it > can be used to create automatic as well as explicit modules. This will > allow a container application to use a custom `ModuleFinder` to manage > the mapping from artifact names to module names, version strings, and > so forth. Could you please clarify how, after this change, an automatic module would be different from a regular module, from the perspective of any code which is using the descriptor Builder API? -- - DML From mark.reinhold at oracle.com Wed Jun 29 23:31:00 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 29 Jun 2016 16:31:00 -0700 Subject: Proposal: #BootstrapClassLoaderSearchInJVMTI In-Reply-To: References: <20160628212215.9E7D9B59D4@eggemoggin.niobe.net> Message-ID: <20160629163100.841006773eggemoggin.niobe.net> 2016/6/29 3:45:53 -0700, tim_ellison at uk.ibm.com: > mark.reinhold at oracle.com (Mark Reinhold) wrote on 28/06/2016 22:22:15: >> Issue summary >> ------------- >> >> #BootstrapClassLoaderSearchInJVMTI --- The JVM TI API [1] defines a >> function, `AddToBootstrapClassLoaderSearch` [2], by which an >> instrumentation agent can append a file-system path element to the >> bootstrap class loader's class path. ... >> >> Proposal >> -------- >> >> Retain this method. Do not deprecate it. > > What would be the effect calling this function on a modular runtime when > passed a directory search path segment? a modular JAR? a regular JAR? It will work just as it does today. If the bootstrap class loader cannot locate a suitable class file within the run-time system then it will consult the path defined by calls to this function. Each element of the path can be either a JAR file or an "exploded" directory tree corresponding to a package hierarchy. Whether a JAR file is a modular JAR is of no import -- any module descriptor within it will be ignored. - Mark From mark.reinhold at oracle.com Wed Jun 29 23:37:18 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 29 Jun 2016 16:37:18 -0700 Subject: Proposal: #ModuleAnnotations and #ModuleDeprecation In-Reply-To: <1683195038.1023897.1467191318948.JavaMail.zimbra@u-pem.fr> References: <20160628211915.9706BB59C8@eggemoggin.niobe.net> <1683195038.1023897.1467191318948.JavaMail.zimbra@u-pem.fr> Message-ID: <20160629163718.727221014eggemoggin.niobe.net> 2016/6/29 2:08:38 -0700, Remi Forax : > ... > > I like both proposals, adding annotations and imports on module. > > I disagree that those annotations should not be visible at runtime, i > don't see why we need a special case here, it's not complex to > implement because the VM already provide the hook to access to those > annotations at runtime, @Deprecated is an annotation visible at > runtime for a good reason, it allows by example to implement policy > like disallowing or emit a warning when calling a deprecated code at > runtime. I completely agree, and the intent of the proposal is to make annotations on module declarations available at run time -- that's why it specifically mentions extending the `java.lang.reflect` API. If there's text in the proposal that suggests otherwise then please point it out and I'll fix it. Thanks. - Mark From mark.reinhold at oracle.com Thu Jun 30 00:03:08 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 29 Jun 2016 17:03:08 -0700 Subject: Proposal: #CustomizableAutomaticModuleNameMapping In-Reply-To: <5773E478.9020903@redhat.com> References: <20160628212315.A247FB59D8@eggemoggin.niobe.net> <5773E478.9020903@redhat.com> Message-ID: <20160629170308.510416918eggemoggin.niobe.net> 2016/6/29 8:08:40 -0700, david.lloyd at redhat.com: > On 06/28/2016 04:23 PM, Mark Reinhold wrote: >> ... >> >> Proposal >> -------- >> >> Extend the `java.lang.module.ModuleDescriptor.Builder` API so that it >> can be used to create automatic as well as explicit modules. This will >> allow a container application to use a custom `ModuleFinder` to manage >> the mapping from artifact names to module names, version strings, and >> so forth. > > Could you please clarify how, after this change, an automatic module > would be different from a regular module, from the perspective of any > code which is using the descriptor Builder API? The `isAutomatic()` method of an explicitly-built automatic `ModuleDescriptor` instance will return `true`. - Mark From mark.reinhold at oracle.com Thu Jun 30 00:35:59 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 29 Jun 2016 17:35:59 -0700 Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic` In-Reply-To: <1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr> References: <20160628211815.87E88B59C4@eggemoggin.niobe.net> <1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr> Message-ID: <20160629173559.18960061eggemoggin.niobe.net> 2016/6/29 1:51:16 -0700, Remi Forax : > I would like to agree on that proposal only on the ground that it > requires to change the classfile format thus allows in the same time > to fix the issues of the current classfile spec (encoding of the name > module as a Class instead of UTF8, wrong flag for requires public), I think that these are completely distinct issues (though your class-file issues are not yet in the issue list -- would you care to suggest some text?). > but i'm not sure this proposal support it's own weight. > > Without dynamic, a 'classical' export works the same way at runtime so > adding 'dynamic' to an export only restrict the usage at compile time. > Given that we can already selectively export packages to some chosen > modules which hide them for the rest of the other modules at compile > time, adding dynamic to the spec adds little value. Qualified exports aren't really a satisfactory substitute for dynamic exports. They still export packages at compile time, thereby tempting unwanted usage. They're also really meant only for use within a group of tightly-associated modules, where hashing can be used to ensure integrity, rather than amongst unrelated modules, between which names can all too easily be spoofed. More to the point, qualified exports are not adequate for all use cases. If you want to write a module that uses JPA but is independent of JPA implementation then you'll need to make your entity classes available to whichever JPA implementation you do wind up using (Hibernate or EclipseLink or ...), but you don't know the name of that implementation's module. In other words, you can't write exports com.foo.app.model to jpa; because `jpa` is not an actual module, it's a stand-in. We could introduce a level of indirection so that you could do that sort of thing, say some notion of "abstract module", but that'd be a very, very deep change and it doesn't fit at all well with existing practices around JAR files. The nice thing about `exports dynamic` is that it lets you say "just make the types in this package available for reflection by anybody, I'm okay with that". > And it has several drawbacks, the main one is that it weaken the > compiletime/runtime fidelity story, with a dynamic export the runtime > world is not the compiletime world anymore. The main motivation to > add dynamic seems to be for backward compatibility during the > transition between the classpath world and the full module world, for > several other cases, we use the command line for that, i don't see why > this case is so special that it should not work like the other > compatibility issues. Yes, it does weaken the fidelity story in order to accommodate existing practice, but at least it does so in an explicit manner so that what's going on is apparent just from reading the code. I don't think, however, that this change is just about compatibility and migration. Even in an idealistic, Utopian, fully-modular world I expect that we'll still want to use frameworks that use reflection on our own types to help us be more productive. > Moreover, an agent, an annotation processor and a jlink plugin are all > able to implement the same trick, add export after the compilation and > before the runtime, so i don't think it's a good idea to change the > spec for that. Then how do you express, in your source code, that you want an export to be added at run time? You could use an annotation, but that would require anybody who wants to do this sort of thing to configure their environment to include the module that defines that annotation and to add the required annotation processor at the appropriate point. Do we really want to tell people to do that when we can solve the problem for them so simply? - Mark From david.lloyd at redhat.com Thu Jun 30 11:30:07 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 30 Jun 2016 06:30:07 -0500 Subject: Proposal: #CustomizableAutomaticModuleNameMapping In-Reply-To: <20160629170308.510416918eggemoggin.niobe.net> References: <20160628212315.A247FB59D8@eggemoggin.niobe.net> <5773E478.9020903@redhat.com> <20160629170308.510416918eggemoggin.niobe.net> Message-ID: <577502BF.5080301@redhat.com> On 06/29/2016 07:03 PM, mark.reinhold at oracle.com wrote: > 2016/6/29 8:08:40 -0700, david.lloyd at redhat.com: >> On 06/28/2016 04:23 PM, Mark Reinhold wrote: >>> ... >>> >>> Proposal >>> -------- >>> >>> Extend the `java.lang.module.ModuleDescriptor.Builder` API so that it >>> can be used to create automatic as well as explicit modules. This will >>> allow a container application to use a custom `ModuleFinder` to manage >>> the mapping from artifact names to module names, version strings, and >>> so forth. >> >> Could you please clarify how, after this change, an automatic module >> would be different from a regular module, from the perspective of any >> code which is using the descriptor Builder API? > > The `isAutomatic()` method of an explicitly-built automatic > `ModuleDescriptor` instance will return `true`. Yeah, I get that, but I mean when looking at the final built module. I assumed that to be implied since building the descriptor is useless if you don't make a module out of it. Are automatic modules substantially different from regular modules in any way? If not, is there a reason to continue to have them? -- - DML From Tim_Ellison at uk.ibm.com Thu Jun 30 14:11:36 2016 From: Tim_Ellison at uk.ibm.com (Tim Ellison) Date: Thu, 30 Jun 2016 15:11:36 +0100 Subject: Proposal: #BootstrapClassLoaderSearchInJVMTI In-Reply-To: <20160629163100.841006773eggemoggin.niobe.net> References: <20160628212215.9E7D9B59D4@eggemoggin.niobe.net> <20160629163100.841006773eggemoggin.niobe.net> Message-ID: mark.reinhold at oracle.com wrote on 30/06/2016 00:31:00: > 2016/6/29 3:45:53 -0700, tim_ellison at uk.ibm.com: > > mark.reinhold at oracle.com (Mark Reinhold) wrote on 28/06/2016 22:22:15: > >> Issue summary > >> ------------- > >> > >> #BootstrapClassLoaderSearchInJVMTI --- The JVM TI API [1] defines a > >> function, `AddToBootstrapClassLoaderSearch` [2], by which an > >> instrumentation agent can append a file-system path element to the > >> bootstrap class loader's class path. ... > >> > >> Proposal > >> -------- > >> > >> Retain this method. Do not deprecate it. > > > > What would be the effect calling this function on a modular runtime when > > passed a directory search path segment? a modular JAR? a regular JAR? > > It will work just as it does today. > > If the bootstrap class loader cannot locate a suitable class file within > the run-time system then it will consult the path defined by calls to > this function. Each element of the path can be either a JAR file or an > "exploded" directory tree corresponding to a package hierarchy. Whether > a JAR file is a modular JAR is of no import -- any module descriptor > within it will be ignored. Just to check I fully understand, this function will affect the search path of the unnamed module associated with the bootstrap class loader, correct? If that's so then I'm happy with the proposed solution. Thanks, Tim Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From mark.reinhold at oracle.com Thu Jun 30 21:45:31 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 30 Jun 2016 14:45:31 -0700 Subject: Proposal: #CustomizableAutomaticModuleNameMapping In-Reply-To: <577502BF.5080301@redhat.com> References: <20160628212315.A247FB59D8@eggemoggin.niobe.net> <5773E478.9020903@redhat.com> <20160629170308.510416918eggemoggin.niobe.net> <577502BF.5080301@redhat.com> Message-ID: <20160630144531.546774815eggemoggin.niobe.net> 2016/6/30 4:30:07 -0700, david.lloyd at redhat.com: > On 06/29/2016 07:03 PM, mark.reinhold at oracle.com wrote: >> 2016/6/29 8:08:40 -0700, david.lloyd at redhat.com: >>> Could you please clarify how, after this change, an automatic module >>> would be different from a regular module, from the perspective of any >>> code which is using the descriptor Builder API? >> >> The `isAutomatic()` method of an explicitly-built automatic >> `ModuleDescriptor` instance will return `true`. > > Yeah, I get that, but I mean when looking at the final built module. I > assumed that to be implied since building the descriptor is useless if > you don't make a module out of it. Are automatic modules substantially > different from regular modules in any way? If not, is there a reason to > continue to have them? http://openjdk.java.net/projects/jigsaw/spec/sotms/#automatic-modules http://cr.openjdk.java.net/~mr/jigsaw/spec/api/java/lang/module/ModuleFinder.html#of-java.nio.file.Path...- - Mark From mark.reinhold at oracle.com Thu Jun 30 23:07:58 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 30 Jun 2016 16:07:58 -0700 Subject: Proposal: #CompileTimeDependences: `requires static` In-Reply-To: References: <20160628211715.84610B59BF@eggemoggin.niobe.net> Message-ID: <20160630160758.482519193eggemoggin.niobe.net> 2016/6/29 2:30:28 -0700, tim_ellison at uk.ibm.com: > mark.reinhold at oracle.com wrote on 28/06/2016 22:17:15: >> >> Issue summary >> ------------- >> >> #CompileTimeDependences --- Provide a means to specify a module >> dependence that is mandatory at compile time but optional at run time, >> for use with libraries that are not strictly necessary but can be >> leveraged if present at run time. (Detail: If a dependence is >> "optional at run time" then does the module system try to satisfy it >> during resolution but fail silently if it cannot, or does it ignore it >> during resolution but add the corresponding read edge to that module if >> it was resolved for some other reason?) [1] >> >> Use cases for this feature include annotations that need not be present >> at run time [2] and "auto-configuring" libraries and frameworks such as >> Joda Beans [3] and Spring [4]. > > For some reason I can't quite explain, this feels like we are giving up > on one of the fundamental completeness assurances of a coherent modular > application. Is this a new failure mode for link/run time modular systems > that has not previously existed? This proposal reflects a fairly common existing practice, at least in Java. Today it's most often expressed via optional dependencies in Maven [1] and carefully-written code that either uses reflection or is prepared to catch the appropriate exceptions in case an optional component is not available at run time. I wouldn't characterize this practice as a failure mode but, rather, a flexible way to configure software systems. > It may be that this proposal satisfies an important compatibility story for > existing code bases, and that is the justification, because on first reading > my reaction is that the code has to be written to deal with the absence of > the optional module anyway, more details below. Yes, that's correct, but see below. >> Proposal >> -------- >> >> Extend the language of module declarations to allow the `static` modifier >> to be used on a `requires` directive, with the following meanings: >> >> - At compile time, `requires static M` expresses a mandatory >> dependence. It is an error if a suitable module cannot be found >> amongst the observable modules and resolved. >> >> - In phases after compile time, `requires static M` expresses an >> optional dependence. The module system will not search the >> observable modules for a suitable module during resolution, but >> if the resulting module graph contains a suitable module then it >> will add the appropriate readability edge prior to doing the usual >> post-resolution sanity checks. (The parenthetical question in the >> issue summary is, in other words, answered in the negative.) > > I read the parenthetical question as an either-or ... Quite right -- I should've written "answered by taking the second alternative". >> ... >> >> The code in `joda.beans` that refers to types in `joda.collect` must, of >> course, be written defensively so that it fails gracefully at run time >> when the `joda.collect` module is not present. > > This is a key point. Since the application code has to be robust in the face > of a missing "requires static" module, wouldn't it be better served by coding > the behvior of the required module as a service that has a "fail gracefully" > implementation and can be replaced by a productive implementation if required > at link/run time? Some uses of compile-time dependences could be better expressed as services, but that's not always the case. If you're trying to leverage a large third-party library (e.g., Guava) when it's present at run time, but can live without it when it's not, then doing that with services would require you to abstract the Guava functionality that you want to use into one or more service types, which is apt to be awkward at best. The situation gets even worse with large auto-configuring frameworks such as Spring [2]. >> ... >> >> Notes >> ----- >> >> - In phases after compile time, why doesn't the module system search >> the observable modules in an attempt to satisfy an optional >> dependence? >> >> This would not be difficult to arrange but it could lead to surprising >> behavior in which optional dependences silently cause many additional >> modules to be resolved. Given the use cases we have it does not seem >> unreasonable to expect an end user to indicate explicitly, either in >> source code or out-of-band (e.g., with a command-line option such as >> `-addmods`), that an otherwise-optional module is actually needed. > > Another option would be to provide a command-line option that explicitly allows > the linker/launcher to ignore missing modules that are required where there is > an assurance that it will not actually be required in the particular context; > that is, if I design a single module with "requires" or "requires static" > dependencies if it is used in a stand-alone or container context, for example. > > I'd prefer to fail because of an explicit -ignoremods option on the command > line rather than a missing -addmods. The existing common practice with JAR files is to trust end users to place desired optional JAR files on their run-time class paths. The current proposal reflects that practice, by asking end users to add desired optional modules via -addmods. We could reverse the sense of all this, change the resolver to try to satisfy optional dependences at link time and run time, and tell people to use -ignoremods when they specifically don't want an optional dependence to be resolved, but I think that would be significantly harder to explain and to understand, and hence a burden to migration. - Mark [1] https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html [2] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005814.html From mark.reinhold at oracle.com Thu Jun 30 23:52:50 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 30 Jun 2016 16:52:50 -0700 Subject: Proposal: #ResourceEncapsulation and #ClassFilesAsResources In-Reply-To: References: <20160628212015.99A06B59CC@eggemoggin.niobe.net> Message-ID: <20160630165250.707143965eggemoggin.niobe.net> 2016/6/29 3:15:38 -0700, tim_ellison at uk.ibm.com: > mark.reinhold at oracle.com wrote on 28/06/2016 22:20:15: >> >> ... >> >> Proposal >> -------- >> >> Drop the agreed resource-encapsulation requirement, which reads: >> >> Resource encapsulation --- The run-time system must ensure that the >> static resource files within a module are directly accessible only by >> code within that module. The existing resource-access APIs should >> continue to work as they do today when used to access a module's own >> resource files. [2] >> >> ... > > The proposal to drop this requirement leaves no way for a module to encapsulate > its resources. Have you considered an alternative that preserves the compatibility > of existing APIs which I agree is desirable to support existing applications, > but introduces a module-level resource API that allows those modules that want > to hide their implementation details? There are certainly other alternatives; what I proposed is just the simplest. One alternative is just to address the two specific use cases. Make META-INF/**, WEB-INF/**, and the class file for any class visible to the caller available via the resource-access APIs, but make all other resources module-private. This wouldn't address other use cases that require non-class-file resources in normal package directories to be available, of which the Wicket web framework is an example (pointed out by Rafael Winterhalter [1]). So a second alternative would be to go further and define a way to declare which packages of a module are considered to "export" their resources, similar to what Peter Levart suggested [2]). Maybe something like this: module com.foo.app { requires wicket.core; requires wicket.utils; publishes com.foo.app.pages.main; publishes com.foo.app.pages.aux; } would make resources in the com/foo/app/pages/{main,aux} directories of the module's artifact available for anyone to load. By fiat we'd make everything under {META,WEB}-INF available, for compatibility. Whether access to the class files for visible classes should be limited further, or perhaps solely, by this mechanism, I'm not sure. So yes, there are some alternatives here. Thoughts welcome. - Mark [1] http://mail.openjdk.java.net/pipermail/jpms-spec-observers/2016-June/000417.html [2] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-June/008355.html