From mark.reinhold at oracle.com Mon Apr 3 16:29:15 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Mon, 3 Apr 2017 09:29:15 -0700 (PDT) Subject: Alternatives for naming automatic modules (#AutomaticModuleNames) Message-ID: <20170403162915.89F9EA1983@eggemoggin.niobe.net> Assume, for the purpose of this note, that we decide to retain automatic modules more or less as-is. How should the automatic-module name of an existing JAR file be computed? The fundamental problem here is that we're trying to infer a sensible name from very little information, so we have to do the best we can with what we've got. * * * Robert Scholte and Brian Fox suggest computing the automatic-module name of a JAR-file artifact from the Maven group and artifact identifiers in the `pom.properties` file of the artifact itself, when available. They observe that 94% of the artifacts in the Maven Central repository contain such a file, since Maven inserts them by default. They conclude that this approach would both "nearly eliminate" module-name conflicts and, via the default effect, establish what they consider to be a desirable pattern for module names [1]. I argued previously that module names neither need be nor should be globally unique, since modules are more abstract than the artifacts that define them [2]. Not everyone found that persuasive, however, so here are some further thoughts on the Scholte/Fox approach, backed up by data. (a) That 94% of the artifacts in Central contain `pom.properties` files does not imply that this approach would be applicable to 94% of the artifacts commonly used in applications today. Less than half of the artifacts in the most popular projects in Central, according to three different "top 100" surveys [3][4][5], contain such files (see [6] for a list of these projects). The ratio doesn't change if you examine just the artifacts with the highest version number for each project, so it's not the case that only newer artifacts have this file. It appears, then, that many popular projects are not built with Maven, or if they are then the default insertion of the `pom.properties` file is disabled. (b) Scholte and Fox don't say exactly how module names should be formed from group-id/artifact-id pairs. The obvious method is just to concatenate them and then replace non-identifier characters with periods. This can, however, result in very long module names with redundant tokens. Here are a few examples from testing this method against the popular artifacts mentioned above (from the "verbose" column in [6]): com.fasterxml.jackson.core.jackson.core com.google.guava.guava commons.lang.commons.lang org.apache.maven.plugin.tools.maven.plugin.annotations org.slf4j.slf4j.api org.springframework.boot.spring.boot.starter.web Not only are these module names annoyingly verbose, but some module authors would strongly object to such names being established for their own modules via the default effect. (c) A more sophisticated method would use heuristics to produce shorter module names. A trick that works surprisingly well, at least for these artifacts, is simply to scan the name computed in (b) from left to right and remove redundant tokens (the "automagic" column in [6]). This leads to module names that often more readable, but they're sometimes quite different from the original coordinates. Any similar heuristic is likely suffer from the same problem, and any such approach is likely to be rejected by module authors who prefer a different style of name. (d) Making the Java SE Platform Specification depend upon the behavior of a particular build tool is inappropriate on principle, no matter how popular that tool. If we were to adopt this kind of approach then the naming algorithm would have to be demoted to the realm of the "host system", in the parlance of the JLS [7], and therefore established not by standard but only by convention. (e) Maven may be the most popular Java build tool today, but it's not the only popular such tool. Gradle, for one, is fairly popular yet it does not generate `pom.properties` files, at least not by default. Even if this naming algorithm were merely a convention it would be questionable -- though perhaps defensible -- for the JDK to establish a convention that is, essentially, specific to one build tool. Considering all these issues, I don't think the Scholte/Fox approach is viable. Available data suggests that it would help with less than half of the most popular projects in use today. It would result in module names that are either annoyingly verbose or not so obviously related to their artifacts' original coordinates, and in either case objectionable to some module authors. It would, finally, raise non-trivial issues in terms of standardization and convention. * * * Stephen Colebourne argues that the best name for a module, whether it's automatic or explicit, is the name of its "highest" package [8]. We could, in theory, scan a JAR file, determine its highest package, and use that as the name of the corresponding automatic module. (The module system must scan the JAR file anyway, in order to identify its packages and service providers, so this would be no less efficient in terms of I/O operations.) This would seem to be an attractive option, but when tested against the popular artifacts mentioned above it fares only a little better than the Scholte/Fox approach. Only about 70% of those artifacts have a single "highest" package. The remainder have multiple such packages, and there's no obvious way to disambiguate them. * * * The least-bad method that we've found to compute the name of an automatic module is to do a simple transformation of its JAR-file name. This has been in the JPMS specification and the prototype RI for over a year now. This method is not ideal, but the concerns raised about it so far can be addressed effectively by tools and documentation or else are really concerns about all kinds of modules, whether automatic or explicit. Herewith a summary of those concerns, and responses to them. (1) It's difficult for the user of an automatic module to figure out the name of the module. This is contrary to my experience, having presented this method to a wide variety of developers at many conferences over the last few years. They immediately understand how it works after being shown an example or two. This concern applies at least as well to any other method for naming automatic modules, and in any case I expect the IDEs eventually to offer completion hints for all the observable modules on a project's module path, whether automatic or explicit. (2) A module name computed by this method was not chosen by the author of the component. A module name produced by this method almost always corresponds clearly to the JAR file's Maven artifact identifier. It therefore is a name that was chosen by the author, though for a somewhat different purpose, i.e., as an artifact name rather than a module name. (3) Module names computed by this method do not follow the popular reverse domain-name convention [9]. This concern applies to all kinds of modules, not just automatic modules. It also assumes agreement with the reverse domain-name convention; many developers prefer that convention but some do not, and it is not the place of a module system, a build tool, or an artifact repository to enforce such conventions. (4) A module name computed by this method is based not upon intrinsic properties of the module's class-file content but, rather, the more ephemeral name of the artifact in which it's packaged. True. A method that uses only intrinsic information found in the classes defined in the module would be preferable in principle. We have not yet, however, found such a method. (5) Suppose that I publish my own explicit module, and its declaration requires some other module, and that module is automatic since it's really just a JAR-file component that hasn't yet been modularized. I'm being rude to the author of that component: If my module is popular then this could establish the automatic-module name as the module name of their component via the default effect, and they might prefer a different name. Yes, that would be rude. We should strongly advise module authors not to publish explicit modules that depend upon components that have not yet been modularized. It's perfectly fine to create and use such modules in a limited fashion, e.g., within a team or organization as part of a long-term modularization effort. Such modules should not, however, be published to Maven Central or any similar public repository [a]. (6) It's risky to require an automatic module in my own explicit module. If the author of the underlying JAR-file component explicitly modularizes it later on then they might choose a name that's different from the automatic name, and they might choose not to export all of its packages, and then I'll have to change my module, and anyone who uses my module might have to upgrade. This concern applies to all kinds of modules, not just automatic modules. The author of an explicit module could, at any time, change the module's name or its set of exported packages. Such changes may be more likely for automatic modules until they're explicitly modularized, but that's just one more reason to treat automatic modules as a transitional tool rather than a long-term solution, and never to publish an explicit module that depends upon automatic modules. (7) If the names of two automatic modules conflict then they cannot both be placed on the same module path. A simple workaround for this problem is for the user of such modules to rename one of the relevant JAR files. [1] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-January/000537.html [2] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-February/000582.html [3] http://blog.takipi.com/the-top-100-java-libraries-in-2016-after-analyzing-47251-dependencies/ [4] https://javalibs.com/charts/dependencies [5] http://mvnrepository.com/popular [6] http://cr.openjdk.java.net/~mr/jigsaw/notes/module-naming-alternatives.html [7] http://docs.oracle.com/javase/specs/jls/se8/html/jls-7.html#jls-7.2 [8] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-February/011520.html [9] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-February/011434.html [a] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-January/011106.html From mark.reinhold at oracle.com Mon Apr 3 16:30:15 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Mon, 3 Apr 2017 09:30:15 -0700 (PDT) Subject: Proposal: #AutomaticModuleNames Message-ID: <20170403163015.90385A1987@eggemoggin.niobe.net> Issue summary ------------- #AutomaticModuleNames -- Revise the algorithm that computes the names of automatic modules to include the Maven group identifier, when available in a JAR file's `pom.properties` file, so that module names are less likely to collide, or else drop the automatic-modules feature entirely, since it may be more trouble than it's worth. [1] Proposal -------- - Do not revise the algorithm that computes the names of automatic modules. The suggestion to use Maven coordinates when available [2] would help less than half of the most popular projects in use today, as determined by three different surveys. It would result in module names that are either annoyingly verbose or not so obviously related to their artifacts' original coordinates, and in either case known to be objectionable to some module authors. It would, finally, raise non-trivial issues with regard to standardization and convention. No algorithm better than the current one has been proposed. [3] - Do not drop the automatic-modules feature. It's a critical part of the overall JPMS migration story, and it's been well-received by a wide variety of developers despite the fact that some expert users are not comfortable with it. - To increase awareness of when automatic modules are used, and of the consequences of their use, encourage Java language compilers to issue two new types of warnings, and implement these warnings in the RI: - An opt-in (i.e., off-by-default) warning on a reference to an automatic module in a `requires` directive in a module declaration, and - An opt-out (i.e., on-by-default) warning on a reference to an automatic module in a `requires transitive` directive in a module declaration. The first warning allows developers to be maximally hygienic if they so choose. The second helps make the author of an explicit module aware that they're putting the users of that module at risk by establishing implied readability to an automatic module. - Encourage Java run-time systems to make it easy to identify via, e.g., command-line options, which observable modules are automatic, which observable modules would establish implied readability to automatic modules if resolved, and which automatic observable modules are actually resolved. Implement these capabilities in the RI. - Strongly advise developers never to publish, for broad use, explicit modules that require automatic modules. That's risky: An automatic module is unreliable, since it can depend on types on the class path, and its name and exported packages could change if and when it's converted into an explicit module. It's fine to declare and use explicit modules that require automatic modules in limited settings, but they should never be published to Maven Central or any similar public repository. - Do not implement the previous suggestion to define a `Module-Name` manifest attribute [4][5]. It would be confusing, because it would establish two ways to name a module, one of which is rooted in the past. It would also make it easy for people other than the author of a module, e.g., tool maintainers [6], to try to establish the module's name via the default effect, a move to which many module authors would understandably object. Not defining this attribute does mean that authors and maintainers who want to modularize existing components yet continue to support those components on earlier releases will have to do a bit more work in order to create `module-info.class` files. It's straightforward to create standalone tools to address this need for those who are unable or unwilling to build on JDK 9. [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#AutomaticModuleNames [2] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-January/000537.html [3] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000666.html [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-February/000582.html [5] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleNameInManifest [6] http://mail-archives.apache.org/mod_mbox/maven-dev/201702.mbox/%3CCAMk_Beg2ir9e65hXG9RSNVFdWNYMr02aTh%3Dd%3DfCWL%2BWof-aXpw%40mail.gmail.com%3E From sander.mak at luminis.eu Mon Apr 3 18:01:54 2017 From: sander.mak at luminis.eu (Sander Mak) Date: Mon, 3 Apr 2017 18:01:54 +0000 Subject: Proposal: #AutomaticModuleNames In-Reply-To: <20170403163015.90385A1987@eggemoggin.niobe.net> References: <20170403163015.90385A1987@eggemoggin.niobe.net> Message-ID: > - Do not implement the previous suggestion to define a `Module-Name` > manifest attribute [4][5]. I was already wondering why this feature hadn't made it to the jigsaw builds yet. However, I disagree that creating `module-info.class` files is just a bit more work than adding a Module-Name attribute. Apart from the logistics, making a named module imposes more constraints. Yes, the maintainer can just create an open module descriptor exporting all packages for the time being, mimicking its use as automatic module. But it won't be able to read the classpath like an automatic module. Nor will it get automatic readability to its dependencies, possibly deployed as (automatic) modules like it got with automatic modules. So, all the library's dependencies must be required in the module descriptor. Some of which may be not yet modularized themselves, so how would the library refer to them? Also, this approach makes it look as if the library author officially sanctions the use (reflective or otherwise) of all types by creating this open module with all packages exported. On the other hand, using the library as automatic module before it is modularized is a deliberate choice made by the library consumer. With the implied expectation that things might change once the library becomes a true module. If the library is published in the proposed 'in-between' form (with just a module descriptor to claim the module name), this implied expectation doesn't exist. A named module is a named module. I can already see the discussions emerge once the library maintainer starts with the real modularization. Which hopefully goes beyond the 'bit more work' done earlier to create the initial module descriptor to claim the module name, involving possible code changes to encapsulate internals and expose a nice API. Being able to 'reserve' a module name with a one-line Pull Request sounds like a great feature to me, even if some people plot to use the same mechanism to hijack module names. Seems to me the baby gets thrown out with the bathwater here, weakening the migration story for library authors. Sander From forax at univ-mlv.fr Tue Apr 4 10:33:28 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 4 Apr 2017 12:33:28 +0200 (CEST) Subject: Proposal: #AutomaticModuleNames In-Reply-To: <20170403163015.90385A1987@eggemoggin.niobe.net> References: <20170403163015.90385A1987@eggemoggin.niobe.net> Message-ID: <1623657670.701980.1491302008159.JavaMail.zimbra@u-pem.fr> Hi Mark, hi all, it seems it will be hard to have a consensus on this issue, but i think we can try to find a common ground, ----- Mail original ----- > De: "mark reinhold" > ?: jpms-spec-experts at openjdk.java.net > Envoy?: Lundi 3 Avril 2017 18:30:15 > Objet: Proposal: #AutomaticModuleNames > Issue summary > ------------- > > #AutomaticModuleNames -- Revise the algorithm that computes the names > of automatic modules to include the Maven group identifier, when > available in a JAR file's `pom.properties` file, so that module names > are less likely to collide, or else drop the automatic-modules feature > entirely, since it may be more trouble than it's worth. [1] > > Proposal > -------- > > - Do not revise the algorithm that computes the names of automatic > modules. The suggestion to use Maven coordinates when available [2] > would help less than half of the most popular projects in use today, > as determined by three different surveys. It would result in module > names that are either annoyingly verbose or not so obviously related > to their artifacts' original coordinates, and in either case known to > be objectionable to some module authors. It would, finally, raise > non-trivial issues with regard to standardization and convention. > No algorithm better than the current one has been proposed. [3] I agree with the conclusion but not the text above. We should not read any property file generated by Maven just because Maven sit on top of the JDK and not the opposite, the dependency (in general meaning) goes in the wrong direction. > > - Do not drop the automatic-modules feature. It's a critical part of > the overall JPMS migration story, and it's been well-received by a > wide variety of developers despite the fact that some expert users > are not comfortable with it. i fully agree. > > - To increase awareness of when automatic modules are used, and of the > consequences of their use, encourage Java language compilers to issue > two new types of warnings, and implement these warnings in the RI: > > - An opt-in (i.e., off-by-default) warning on a reference to > an automatic module in a `requires` directive in a module > declaration, and it should be an opt-out one, automatic module are cool in short term and evil in long term, so you need a remainder. > > - An opt-out (i.e., on-by-default) warning on a reference to > an automatic module in a `requires transitive` directive in > a module declaration. > > The first warning allows developers to be maximally hygienic if they > so choose. The second helps make the author of an explicit module > aware that they're putting the users of that module at risk by > establishing implied readability to an automatic module. > > - Encourage Java run-time systems to make it easy to identify via, > e.g., command-line options, which observable modules are automatic, > which observable modules would establish implied readability to > automatic modules if resolved, and which automatic observable modules > are actually resolved. Implement these capabilities in the RI. yes, one simple way to do that is to considere all automatic modules as deprecated (forRemoval=false), so the tool already exist, it's jdeprscan. > > - Strongly advise developers never to publish, for broad use, explicit > modules that require automatic modules. That's risky: An automatic > module is unreliable, since it can depend on types on the class path, > and its name and exported packages could change if and when it's > converted into an explicit module. It's fine to declare and use > explicit modules that require automatic modules in limited settings, > but they should never be published to Maven Central or any similar > public repository. yes, you can formally ask Sonatype to never allow to publish an automatic module ? > > - Do not implement the previous suggestion to define a `Module-Name` > manifest attribute [4][5]. It would be confusing, because it would > establish two ways to name a module, one of which is rooted in the > past. It would also make it easy for people other than the author > of a module, e.g., tool maintainers [6], to try to establish the > module's name via the default effect, a move to which many module > authors would understandably object. > > Not defining this attribute does mean that authors and maintainers > who want to modularize existing components yet continue to support > those components on earlier releases will have to do a bit more work > in order to create `module-info.class` files. It's straightforward > to create standalone tools to address this need for those who are > unable or unwilling to build on JDK 9. Yes, the idea of using the "Module-Name" manifest attribute is an idea of the past, but it's a quick fix that library author can use to reserve the name of their module before anyone choose a name. Creating a module-info only works if all your dependencies are modular too, that's why we have automatic module. While migrating should be done as fast as possible, this will not be done in one night, so i think we still need the "Module-Name" attribute to not punish people that want to run their application on Java 9, because otherwise they will have to change their application again each time one dependency becomes modular. Perhaps a middle ground is to allow 'Module-Name' in 9 and emit a warning like --permit-illegal-access if an automatic module with a "Module-Name" is used in 10. R?mi > > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#AutomaticModuleNames > [2] > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-January/000537.html > [3] > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000666.html > [4] > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-February/000582.html > [5] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleNameInManifest > [6] > http://mail-archives.apache.org/mod_mbox/maven-dev/201702.mbox/%3CCAMk_Beg2ir9e65hXG9RSNVFdWNYMr02aTh%3Dd%3DfCWL%2BWof-aXpw%40mail.gmail.com%3E From brianf at infinity.nu Tue Apr 4 12:43:42 2017 From: brianf at infinity.nu (Brian Fox) Date: Tue, 4 Apr 2017 13:43:42 +0100 Subject: Proposal: #AutomaticModuleNames In-Reply-To: <1623657670.701980.1491302008159.JavaMail.zimbra@u-pem.fr> References: <20170403163015.90385A1987@eggemoggin.niobe.net> <1623657670.701980.1491302008159.JavaMail.zimbra@u-pem.fr> Message-ID: The Module-Name was enough of a hook to allow us to have Maven "fix" this problem automatically by inserting a name based on the coordinates. Taking that away I think again ties our hands. On Tue, Apr 4, 2017 at 11:33 AM, Remi Forax wrote: > Hi Mark, hi all, > it seems it will be hard to have a consensus on this issue, but i think we > can try to find a common ground, > > ----- Mail original ----- > > De: "mark reinhold" > > ?: jpms-spec-experts at openjdk.java.net > > Envoy?: Lundi 3 Avril 2017 18:30:15 > > Objet: Proposal: #AutomaticModuleNames > > > Issue summary > > ------------- > > > > #AutomaticModuleNames -- Revise the algorithm that computes the names > > of automatic modules to include the Maven group identifier, when > > available in a JAR file's `pom.properties` file, so that module names > > are less likely to collide, or else drop the automatic-modules feature > > entirely, since it may be more trouble than it's worth. [1] > > > > Proposal > > -------- > > > > - Do not revise the algorithm that computes the names of automatic > > modules. The suggestion to use Maven coordinates when available [2] > > would help less than half of the most popular projects in use today, > > as determined by three different surveys. It would result in module > > names that are either annoyingly verbose or not so obviously related > > to their artifacts' original coordinates, and in either case known to > > be objectionable to some module authors. It would, finally, raise > > non-trivial issues with regard to standardization and convention. > > No algorithm better than the current one has been proposed. [3] > > I agree with the conclusion but not the text above. > We should not read any property file generated by Maven just because Maven > sit on top of the JDK and not the opposite, > the dependency (in general meaning) goes in the wrong direction. > > > > > - Do not drop the automatic-modules feature. It's a critical part of > > the overall JPMS migration story, and it's been well-received by a > > wide variety of developers despite the fact that some expert users > > are not comfortable with it. > > i fully agree. > > > > > - To increase awareness of when automatic modules are used, and of the > > consequences of their use, encourage Java language compilers to issue > > two new types of warnings, and implement these warnings in the RI: > > > > - An opt-in (i.e., off-by-default) warning on a reference to > > an automatic module in a `requires` directive in a module > > declaration, and > > it should be an opt-out one, automatic module are cool in short term and > evil in long term, so you need a remainder. > > > > > - An opt-out (i.e., on-by-default) warning on a reference to > > an automatic module in a `requires transitive` directive in > > a module declaration. > > > > The first warning allows developers to be maximally hygienic if they > > so choose. The second helps make the author of an explicit module > > aware that they're putting the users of that module at risk by > > establishing implied readability to an automatic module. > > > > - Encourage Java run-time systems to make it easy to identify via, > > e.g., command-line options, which observable modules are automatic, > > which observable modules would establish implied readability to > > automatic modules if resolved, and which automatic observable modules > > are actually resolved. Implement these capabilities in the RI. > > yes, one simple way to do that is to considere all automatic modules as > deprecated (forRemoval=false), > so the tool already exist, it's jdeprscan. > > > > > - Strongly advise developers never to publish, for broad use, explicit > > modules that require automatic modules. That's risky: An automatic > > module is unreliable, since it can depend on types on the class path, > > and its name and exported packages could change if and when it's > > converted into an explicit module. It's fine to declare and use > > explicit modules that require automatic modules in limited settings, > > but they should never be published to Maven Central or any similar > > public repository. > > yes, > you can formally ask Sonatype to never allow to publish an automatic > module ? > > > > > - Do not implement the previous suggestion to define a `Module-Name` > > manifest attribute [4][5]. It would be confusing, because it would > > establish two ways to name a module, one of which is rooted in the > > past. It would also make it easy for people other than the author > > of a module, e.g., tool maintainers [6], to try to establish the > > module's name via the default effect, a move to which many module > > authors would understandably object. > > > > Not defining this attribute does mean that authors and maintainers > > who want to modularize existing components yet continue to support > > those components on earlier releases will have to do a bit more work > > in order to create `module-info.class` files. It's straightforward > > to create standalone tools to address this need for those who are > > unable or unwilling to build on JDK 9. > > Yes, the idea of using the "Module-Name" manifest attribute is an idea of > the past, > but it's a quick fix that library author can use to reserve the name of > their module before anyone choose a name. > > Creating a module-info only works if all your dependencies are modular > too, that's why we have automatic module. > > While migrating should be done as fast as possible, this will not be done > in one night, so i think we still need the "Module-Name" attribute to not > punish people that want to run their application on Java 9, because > otherwise they will have to change their application again each time one > dependency becomes modular. > > Perhaps a middle ground is to allow 'Module-Name' in 9 and emit a warning > like --permit-illegal-access if an automatic module with a "Module-Name" is > used in 10. > > R?mi > > > > > > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#Automat > icModuleNames > > [2] > > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/201 > 7-January/000537.html > > [3] > > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/201 > 7-April/000666.html > > [4] > > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/201 > 7-February/000582.html > > [5] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleN > ameInManifest > > [6] > > http://mail-archives.apache.org/mod_mbox/maven-dev/201702.mb > ox/%3CCAMk_Beg2ir9e65hXG9RSNVFdWNYMr02aTh%3Dd%3DfCWL%2BWof- > aXpw%40mail.gmail.com%3E > From mark.reinhold at oracle.com Tue Apr 4 23:53:30 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Tue, 04 Apr 2017 16:53:30 -0700 Subject: Proposal: #AutomaticModuleNames In-Reply-To: <1623657670.701980.1491302008159.JavaMail.zimbra@u-pem.fr> References: <20170403163015.90385A1987@eggemoggin.niobe.net> <1623657670.701980.1491302008159.JavaMail.zimbra@u-pem.fr> Message-ID: <20170404165330.587372716@eggemoggin.niobe.net> 2017/4/4 3:33:28 -0700, Remi Forax : > 2017/4/3 9:30:15 -0700, mark.reinhold at oracle.com: >> ... >> >> Proposal >> -------- >> >> - Do not revise the algorithm that computes the names of automatic >> modules. The suggestion to use Maven coordinates when available [2] >> would help less than half of the most popular projects in use today, >> as determined by three different surveys. It would result in module >> names that are either annoyingly verbose or not so obviously related >> to their artifacts' original coordinates, and in either case known to >> be objectionable to some module authors. It would, finally, raise >> non-trivial issues with regard to standardization and convention. >> No algorithm better than the current one has been proposed. [3] > > I agree with the conclusion but not the text above. > We should not read any property file generated by Maven just because > Maven sit on top of the JDK and not the opposite, the dependency (in > general meaning) goes in the wrong direction. I agree with that line of reasoning too, but I think it's essentially equivalent to the points I made about standardization and convention. >> - Do not drop the automatic-modules feature. It's a critical part of >> the overall JPMS migration story, and it's been well-received by a >> wide variety of developers despite the fact that some expert users >> are not comfortable with it. > > i fully agree. Glad to hear it. >> - To increase awareness of when automatic modules are used, and of the >> consequences of their use, encourage Java language compilers to issue >> two new types of warnings, and implement these warnings in the RI: >> >> - An opt-in (i.e., off-by-default) warning on a reference to >> an automatic module in a `requires` directive in a module >> declaration, and > > it should be an opt-out one, automatic module are cool in short term > and evil in long term, so you need a remainder. I'm not convinced. We want developers just getting started to feel free to experiment with automatic modules without shame. Confronting them with warnings right away could be counterproductive. >> - An opt-out (i.e., on-by-default) warning on a reference to >> an automatic module in a `requires transitive` directive in >> a module declaration. >> >> The first warning allows developers to be maximally hygienic if they >> so choose. The second helps make the author of an explicit module >> aware that they're putting the users of that module at risk by >> establishing implied readability to an automatic module. >> >> - Encourage Java run-time systems to make it easy to identify via, >> e.g., command-line options, which observable modules are automatic, >> which observable modules would establish implied readability to >> automatic modules if resolved, and which automatic observable modules >> are actually resolved. Implement these capabilities in the RI. > > yes, one simple way to do that is to considere all automatic modules > as deprecated (forRemoval=false), so the tool already exist, it's > jdeprscan. That's asking people to use a different tool. The intent of the above suggestions is to highlight the presence of automatic modules in their full context, at run time, via the launcher itself. >> - Strongly advise developers never to publish, for broad use, explicit >> modules that require automatic modules. That's risky: An automatic >> module is unreliable, since it can depend on types on the class path, >> and its name and exported packages could change if and when it's >> converted into an explicit module. It's fine to declare and use >> explicit modules that require automatic modules in limited settings, >> but they should never be published to Maven Central or any similar >> public repository. > > yes, > you can formally ask Sonatype to never allow to publish an automatic module ? An automatic module is just a plain old JAR file, so I don't think we can ask them to stop publishing those. I do think it'd be a fine thing for all the major artifact repositories to refuse to publish explicit modules that require modules that are only available in automatic form, i.e., as plain old JAR files. >> - Do not implement the previous suggestion to define a `Module-Name` >> manifest attribute [4][5]. It would be confusing, because it would >> establish two ways to name a module, one of which is rooted in the >> past. It would also make it easy for people other than the author >> of a module, e.g., tool maintainers [6], to try to establish the >> module's name via the default effect, a move to which many module >> authors would understandably object. >> >> ... > > Yes, the idea of using the "Module-Name" manifest attribute is an idea > of the past, but it's a quick fix that library author can use to > reserve the name of their module before anyone choose a name. There's another quick fix: If you don't like the automatic-module name of your library then rename its artifact to have an automatic-module name that you do like. It's fairly straightforward to rename artifacts in Maven [b]. > Creating a module-info only works if all your dependencies are modular > too, that's why we have automatic module. > > ... > > Perhaps a middle ground is to allow 'Module-Name' in 9 and emit a > warning like --permit-illegal-access if an automatic module with a > "Module-Name" is used in 10. People really hate warning messages today, as they should. I can see using them for things that are dangerous from the very start, such as breaking encapsulation globally. If, however, we use them for things that are acceptable now but become unacceptable in some later release, such as the `Module-name` attribute, then we risk teaching people over time to ignore all warning messages. I'd rather not reduce the value of warning messages in general just to support this attribute for a couple of releases. - Mark [b] https://maven.apache.org/guides/mini/guide-relocation.html From mark.reinhold at oracle.com Wed Apr 5 00:08:31 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Tue, 04 Apr 2017 17:08:31 -0700 Subject: Proposal: #AutomaticModuleNames In-Reply-To: References: <20170403163015.90385A1987@eggemoggin.niobe.net> <1623657670.701980.1491302008159.JavaMail.zimbra@u-pem.fr> Message-ID: <20170404170831.166729982@eggemoggin.niobe.net> 2017/4/4 5:43:42 -0700, Brian Fox : > The Module-Name was enough of a hook to allow us to have Maven "fix" this > problem automatically by inserting a name based on the coordinates. Taking > that away I think again ties our hands. Yes. That's precisely the point. This isn't a "problem" for Maven to "fix". The final, explicit name of a module should be chosen by its author. It's not appropriate for a build tool or an artifact repository to enforce any particular convention for naming modules. - Mark From david.lloyd at redhat.com Wed Apr 5 00:32:03 2017 From: david.lloyd at redhat.com (David M. Lloyd) Date: Tue, 4 Apr 2017 19:32:03 -0500 Subject: Proposal: #AutomaticModuleNames In-Reply-To: <20170404170831.166729982@eggemoggin.niobe.net> References: <20170403163015.90385A1987@eggemoggin.niobe.net> <1623657670.701980.1491302008159.JavaMail.zimbra@u-pem.fr> <20170404170831.166729982@eggemoggin.niobe.net> Message-ID: <77329de3-0c5e-c7af-a325-fb5a6cd93bd7@redhat.com> On 04/04/2017 07:08 PM, mark.reinhold at oracle.com wrote: > 2017/4/4 5:43:42 -0700, Brian Fox : >> The Module-Name was enough of a hook to allow us to have Maven "fix" this >> problem automatically by inserting a name based on the coordinates. Taking >> that away I think again ties our hands. > > Yes. That's precisely the point. > > This isn't a "problem" for Maven to "fix". > > The final, explicit name of a module should be chosen by its author. > It's not appropriate for a build tool or an artifact repository to > enforce any particular convention for naming modules. This is deceptive. Yes, build tools or repositories shouldn't enforce conventions for naming. But at the same time, it's unrealistic to expect that module authors are going to bundle module descriptors that are universally compatible, and while naming is a part of this, it also extends to things like dependency names and export types. All you need is two modules which provide the same specification API in different ways to expose this as truth, but the mere existence of Maven's dependencyManagement and exclusions also are strong indicators. OSGi can get away with library developers shipping descriptors, sometimes, but this is generally a consequence of dependencies being expressed by package instead of by bundle name, as well as other key differences between the descriptor types. I think rather what we'll end up with in the end is that non-trivial application builders who want to use Jigsaw will rely almost exclusively on build tooling to generate or regenerate "real" descriptors "safely" for their application distributions, which in turn will be based on meta-information from the artifacts being assembled and their build descriptions. The build system has much more capability (and leisure) than the run time to analyze the artifact graph to produce a cohesive module structure and useful diagnostics. There are a lot of little reasons why this kind of tooling is really the only way users will be able be sure that their assembly of modules is internally cohesive and consistent. The module descriptor is fundamentally a different concern from the ABI and behavioral contracts of software. I think that in this light, the entire concept of automatic modules is questionable because with such tooling, automatic modules give you nothing of value over adding a couple lines of XML to your POM. -- - DML From brianf at infinity.nu Wed Apr 5 17:22:09 2017 From: brianf at infinity.nu (Brian Fox) Date: Wed, 5 Apr 2017 18:22:09 +0100 Subject: Proposal: #AutomaticModuleNames In-Reply-To: <77329de3-0c5e-c7af-a325-fb5a6cd93bd7@redhat.com> References: <20170403163015.90385A1987@eggemoggin.niobe.net> <1623657670.701980.1491302008159.JavaMail.zimbra@u-pem.fr> <20170404170831.166729982@eggemoggin.niobe.net> <77329de3-0c5e-c7af-a325-fb5a6cd93bd7@redhat.com> Message-ID: On Wed, Apr 5, 2017 at 1:32 AM, David M. Lloyd wrote: > I think rather what we'll end up with in the end is that non-trivial > application builders who want to use Jigsaw will rely almost exclusively on > build tooling to generate or regenerate "real" descriptors "safely" for > their application distributions, I totally agree. The next big popular maven plugin will be the maven-fix-automodule-hell-plugin due to user demand that without any other hooks in place will have to resort to modifying 3rd party dependencies to unwind otherwise messed up default names. It blows my mind that we keep coming back to the filename as the best default despite all the alternatives that have been proposed. From forax at univ-mlv.fr Thu Apr 6 08:01:35 2017 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Thu, 6 Apr 2017 10:01:35 +0200 (CEST) Subject: Proposal: #AutomaticModuleNames In-Reply-To: <20170404165330.587372716@eggemoggin.niobe.net> References: <20170403163015.90385A1987@eggemoggin.niobe.net> <1623657670.701980.1491302008159.JavaMail.zimbra@u-pem.fr> <20170404165330.587372716@eggemoggin.niobe.net> Message-ID: <326484726.1629758.1491465695348.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "mark reinhold" > ?: "Remi Forax" > Cc: jpms-spec-experts at openjdk.java.net > Envoy?: Mercredi 5 Avril 2017 01:53:30 > Objet: Re: Proposal: #AutomaticModuleNames > 2017/4/4 3:33:28 -0700, Remi Forax : >> 2017/4/3 9:30:15 -0700, mark.reinhold at oracle.com: >>> ... >>> >>> Proposal >>> -------- >>> >>> - Do not revise the algorithm that computes the names of automatic >>> modules. The suggestion to use Maven coordinates when available [2] >>> would help less than half of the most popular projects in use today, >>> as determined by three different surveys. It would result in module >>> names that are either annoyingly verbose or not so obviously related >>> to their artifacts' original coordinates, and in either case known to >>> be objectionable to some module authors. It would, finally, raise >>> non-trivial issues with regard to standardization and convention. >>> No algorithm better than the current one has been proposed. [3] >> >> I agree with the conclusion but not the text above. >> We should not read any property file generated by Maven just because >> Maven sit on top of the JDK and not the opposite, the dependency (in >> general meaning) goes in the wrong direction. > > I agree with that line of reasoning too, but I think it's essentially > equivalent to the points I made about standardization and convention. > >>> - Do not drop the automatic-modules feature. It's a critical part of >>> the overall JPMS migration story, and it's been well-received by a >>> wide variety of developers despite the fact that some expert users >>> are not comfortable with it. >> >> i fully agree. > > Glad to hear it. > >>> - To increase awareness of when automatic modules are used, and of the >>> consequences of their use, encourage Java language compilers to issue >>> two new types of warnings, and implement these warnings in the RI: >>> >>> - An opt-in (i.e., off-by-default) warning on a reference to >>> an automatic module in a `requires` directive in a module >>> declaration, and >> >> it should be an opt-out one, automatic module are cool in short term >> and evil in long term, so you need a remainder. > > I'm not convinced. We want developers just getting started to feel free > to experiment with automatic modules without shame. Confronting them > with warnings right away could be counterproductive. You maybe right, at the same time, we have just introduced --permit-illegal-access ... > >>> - An opt-out (i.e., on-by-default) warning on a reference to >>> an automatic module in a `requires transitive` directive in >>> a module declaration. >>> >>> The first warning allows developers to be maximally hygienic if they >>> so choose. The second helps make the author of an explicit module >>> aware that they're putting the users of that module at risk by >>> establishing implied readability to an automatic module. >>> >>> - Encourage Java run-time systems to make it easy to identify via, >>> e.g., command-line options, which observable modules are automatic, >>> which observable modules would establish implied readability to >>> automatic modules if resolved, and which automatic observable modules >>> are actually resolved. Implement these capabilities in the RI. >> >> yes, one simple way to do that is to considere all automatic modules >> as deprecated (forRemoval=false), so the tool already exist, it's >> jdeprscan. > > That's asking people to use a different tool. The intent of the above > suggestions is to highlight the presence of automatic modules in their > full context, at run time, via the launcher itself. ok, did not get that, i agree, it's better if the warning comes from the launcher itself. > >>> - Strongly advise developers never to publish, for broad use, explicit >>> modules that require automatic modules. That's risky: An automatic >>> module is unreliable, since it can depend on types on the class path, >>> and its name and exported packages could change if and when it's >>> converted into an explicit module. It's fine to declare and use >>> explicit modules that require automatic modules in limited settings, >>> but they should never be published to Maven Central or any similar >>> public repository. >> >> yes, >> you can formally ask Sonatype to never allow to publish an automatic module ? > > An automatic module is just a plain old JAR file, so I don't think we > can ask them to stop publishing those. > > I do think it'd be a fine thing for all the major artifact repositories > to refuse to publish explicit modules that require modules that are only > available in automatic form, i.e., as plain old JAR files. yes, i was thinking exactly that, Sonatype (and others) should not allow in public repos, artifacts that have automatic modules as dependencies. I think we should add something like that in the spec as a recommendation. > >>> - Do not implement the previous suggestion to define a `Module-Name` >>> manifest attribute [4][5]. It would be confusing, because it would >>> establish two ways to name a module, one of which is rooted in the >>> past. It would also make it easy for people other than the author >>> of a module, e.g., tool maintainers [6], to try to establish the >>> module's name via the default effect, a move to which many module >>> authors would understandably object. >>> >>> ... >> >> Yes, the idea of using the "Module-Name" manifest attribute is an idea >> of the past, but it's a quick fix that library author can use to >> reserve the name of their module before anyone choose a name. > > There's another quick fix: If you don't like the automatic-module name > of your library then rename its artifact to have an automatic-module > name that you do like. It's fairly straightforward to rename artifacts > in Maven [b]. yes, but that not the point, the point is to have a stable name. > >> Creating a module-info only works if all your dependencies are modular >> too, that's why we have automatic module. >> >> ... >> >> Perhaps a middle ground is to allow 'Module-Name' in 9 and emit a >> warning like --permit-illegal-access if an automatic module with a >> "Module-Name" is used in 10. > > People really hate warning messages today, as they should. I can see > using them for things that are dangerous from the very start, such as > breaking encapsulation globally. If, however, we use them for things > that are acceptable now but become unacceptable in some later release, > such as the `Module-name` attribute, then we risk teaching people over > time to ignore all warning messages. I'd rather not reduce the value > of warning messages in general just to support this attribute for a > couple of releases. ok, so let add Module-Name in the spec, having stable name is really something important, and as i said, it rewards people that migrate soon to the modular world. > > - Mark > > > [b] https://maven.apache.org/guides/mini/guide-relocation.html R?mi From forax at univ-mlv.fr Thu Apr 6 08:09:19 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 6 Apr 2017 10:09:19 +0200 (CEST) Subject: Proposal: #AutomaticModuleNames In-Reply-To: References: <20170403163015.90385A1987@eggemoggin.niobe.net> <1623657670.701980.1491302008159.JavaMail.zimbra@u-pem.fr> <20170404170831.166729982@eggemoggin.niobe.net> <77329de3-0c5e-c7af-a325-fb5a6cd93bd7@redhat.com> Message-ID: <1484971960.1648716.1491466159415.JavaMail.zimbra@u-pem.fr> Here is the code (for my build tool, not Maven): https://github.com/forax/pro/blob/master/src/main/java/com.github.forax.pro.plugin.modulefixer/com/github/forax/pro/plugin/modulefixer/ModuleFixerPlugin.java It does static analysis to find the requires and export, open the META-INF/services to add provides and looks for ServiceLoader.load to add uses (doing a flow analysis of the class constants). You still need to add few additional requires because of Class.forName, but most of the grunt works is done automatically. R?mi ----- Mail original ----- > De: "Brian Fox" > ?: "jpms-spec-observers" > Cc: jpms-spec-experts at openjdk.java.net > Envoy?: Mercredi 5 Avril 2017 19:22:09 > Objet: Re: Proposal: #AutomaticModuleNames > On Wed, Apr 5, 2017 at 1:32 AM, David M. Lloyd > wrote: > >> I think rather what we'll end up with in the end is that non-trivial >> application builders who want to use Jigsaw will rely almost exclusively on >> build tooling to generate or regenerate "real" descriptors "safely" for >> their application distributions, > > > I totally agree. The next big popular maven plugin will be the > maven-fix-automodule-hell-plugin due to user demand that without any other > hooks in place will have to resort to modifying 3rd party dependencies to > unwind otherwise messed up default names. It blows my mind that we keep > coming back to the filename as the best default despite all the > alternatives that have been proposed. From mark.reinhold at oracle.com Thu Apr 6 15:21:21 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 06 Apr 2017 08:21:21 -0700 Subject: Proposal: #AutomaticModuleNames In-Reply-To: <326484726.1629758.1491465695348.JavaMail.zimbra@u-pem.fr> References: <20170403163015.90385A1987@eggemoggin.niobe.net> <1623657670.701980.1491302008159.JavaMail.zimbra@u-pem.fr> <20170404165330.587372716@eggemoggin.niobe.net> <326484726.1629758.1491465695348.JavaMail.zimbra@u-pem.fr> Message-ID: <20170406082121.691622335@eggemoggin.niobe.net> 2017/4/6 1:01:35 -0700, Remi Forax : > 2017/4/4 16:53:30 -0700, mark.reinhold at oracle.com: >> 2017/4/4 3:33:28 -0700, Remi Forax : >>> 2017/4/3 9:30:15 -0700, mark.reinhold at oracle.com: >>>> ... >>>> >>>> - Do not implement the previous suggestion to define a `Module-Name` >>>> manifest attribute [4][5]. It would be confusing, because it would >>>> establish two ways to name a module, one of which is rooted in the >>>> past. It would also make it easy for people other than the author >>>> of a module, e.g., tool maintainers [6], to try to establish the >>>> module's name via the default effect, a move to which many module >>>> authors would understandably object. >>>> >>>> ... >>> >>> Yes, the idea of using the "Module-Name" manifest attribute is an idea >>> of the past, but it's a quick fix that library author can use to >>> reserve the name of their module before anyone choose a name. >> >> There's another quick fix: If you don't like the automatic-module name >> of your library then rename its artifact to have an automatic-module >> name that you do like. It's fairly straightforward to rename artifacts >> in Maven [b]. > > yes, but that not the point, the point is to have a stable name. Agreed -- an automatic module name computed from the name of your artifact, even if it's a name that you like, is not quite as stable as the value of a `Module-Name` attribute would be. Still, it is a way to help prepare your users for the future. >>> Creating a module-info only works if all your dependencies are modular >>> too, that's why we have automatic module. >>> >>> ... >>> >>> Perhaps a middle ground is to allow 'Module-Name' in 9 and emit a >>> warning like --permit-illegal-access if an automatic module with a >>> "Module-Name" is used in 10. >> >> People really hate warning messages today, as they should. I can see >> using them for things that are dangerous from the very start, such as >> breaking encapsulation globally. If, however, we use them for things >> that are acceptable now but become unacceptable in some later release, >> such as the `Module-name` attribute, then we risk teaching people over >> time to ignore all warning messages. I'd rather not reduce the value >> of warning messages in general just to support this attribute for a >> couple of releases. > > ok, so let add Module-Name in the spec, having stable name is really > something important, and as i said, it rewards people that migrate > soon to the modular world. I completely agree that having stable names is important, but I'm also firmly convinced that they should be chosen consciously and with due care by module authors and maintainers -- and no one else. The trouble with the `Module-Name` attribute is that it makes it too easy for people other than module authors and maintainers to establish the stable names of modules. That's inappropriate, and we should not enable it. As to encouraging people to migrate to the modular world sooner rather than later, just adding a `Module-Name` attribute doesn't really count as migration. Actual migration requires you to write a proper module declaration, and if the only way to claim a stable name is to do that then this seems likely to encourage more developers to do so. - Mark From brianf at infinity.nu Thu Apr 6 18:24:57 2017 From: brianf at infinity.nu (Brian Fox) Date: Thu, 6 Apr 2017 20:24:57 +0200 Subject: Proposal: #AutomaticModuleNames In-Reply-To: <20170406082121.691622335@eggemoggin.niobe.net> References: <20170403163015.90385A1987@eggemoggin.niobe.net> <1623657670.701980.1491302008159.JavaMail.zimbra@u-pem.fr> <20170404165330.587372716@eggemoggin.niobe.net> <326484726.1629758.1491465695348.JavaMail.zimbra@u-pem.fr> <20170406082121.691622335@eggemoggin.niobe.net> Message-ID: On Thu, Apr 6, 2017 at 5:21 PM, wrote: > 2017/4/6 1:01:35 -0700, Remi Forax : > > 2017/4/4 16:53:30 -0700, mark.reinhold at oracle.com: > >> 2017/4/4 3:33:28 -0700, Remi Forax : > >>> 2017/4/3 9:30:15 -0700, mark.reinhold at oracle.com: > >>>> ... > >>>> > >>>> - Do not implement the previous suggestion to define a `Module-Name` > >>>> manifest attribute [4][5]. It would be confusing, because it would > >>>> establish two ways to name a module, one of which is rooted in the > >>>> past. It would also make it easy for people other than the author > >>>> of a module, e.g., tool maintainers [6], to try to establish the > >>>> module's name via the default effect, a move to which many module > >>>> authors would understandably object. > >>>> > >>>> ... > >>> > >>> Yes, the idea of using the "Module-Name" manifest attribute is an idea > >>> of the past, but it's a quick fix that library author can use to > >>> reserve the name of their module before anyone choose a name. > >> > >> There's another quick fix: If you don't like the automatic-module name > >> of your library then rename its artifact to have an automatic-module > >> name that you do like. It's fairly straightforward to rename artifacts > >> in Maven [b]. > > > > yes, but that not the point, the point is to have a stable name. > > Agreed -- an automatic module name computed from the name of your > artifact, even if it's a name that you like, is not quite as stable > as the value of a `Module-Name` attribute would be. Still, it is > a way to help prepare your users for the future. > > >>> Creating a module-info only works if all your dependencies are modular > >>> too, that's why we have automatic module. > >>> > >>> ... > >>> > >>> Perhaps a middle ground is to allow 'Module-Name' in 9 and emit a > >>> warning like --permit-illegal-access if an automatic module with a > >>> "Module-Name" is used in 10. > >> > >> People really hate warning messages today, as they should. I can see > >> using them for things that are dangerous from the very start, such as > >> breaking encapsulation globally. If, however, we use them for things > >> that are acceptable now but become unacceptable in some later release, > >> such as the `Module-name` attribute, then we risk teaching people over > >> time to ignore all warning messages. I'd rather not reduce the value > >> of warning messages in general just to support this attribute for a > >> couple of releases. > > > > ok, so let add Module-Name in the spec, having stable name is really > > something important, and as i said, it rewards people that migrate > > soon to the modular world. > > I completely agree that having stable names is important, but I'm also > firmly convinced that they should be chosen consciously and with due > care by module authors and maintainers -- and no one else. > > The trouble with the `Module-Name` attribute is that it makes it too > easy for people other than module authors and maintainers to establish > the stable names of modules. That's inappropriate, and we should not > enable it. > You're clearly referring to build tools here. Our users are your users and therefore our interests should be absolutely aligned. Our users won't tolerate the build tool doing something completely stupid. The ability for build tools to adequately support the jigsaw efforts will be key to driving adoption, so lets take a step back and analyze the situation. It's hopefully not controversial and we should be able to agree that in an ideal world, filename is not the best default module name...however for a legacy jar without any other mechanism, you feel there's there's no other choice. In other words, there's no metadata that can be convincingly used to indicate the author's intended module name. We disagree on the priority of a "beautiful" name vs a properly name spaced and unique one. You prefer the former, I prefer the later (although generally I am a neat-nick on naming, it's just a case of priority for me in this case). Fine, set that aside for now. The guidance in your last proposal was to block publishing of libraries that depend upon automodules. This is largely impractical in the real world, but lets just assume there's a way to dis-incent library creators from depending upon an automodule. The problem with this is that no library creator can ever fully modularize until ALL OF his/her dependencies have also done so. With an ecosystem that has transitive dependencies sometimes dozens to hundreds of layers deep and with some of those very deep dependencies quite stable and infrequently updated, this will take an eternity before the ecosystem can get over this hump. *The build tools can help this migration from the old world to the new one, but you must let us help.* The vision that I had behind the Module Name was simply that we could make it easy for module authors starting nearly immediately to choose their module name. (There is disagreement within Maven committers over defaulting this, so I seem to be in the minority of this and I think you don't have to be afraid because of that.) We can make choosing and declaring a name easy, maybe even required very soon for library authors. That means that we can start to build up the very metadata that is missing and forcing you to lean only on filename as the default. We can do so starting now, and by the time jigsaw starts to hit critical mass, there will hopefully be very few important libraries that aren't already properly named _by their authors_ as you intend. The bar to picking a good name is clearly much lower than fully modularizing...especially if you are barred or shamed into doing so before all your dependencies have. If you both require that someone is fully ready to modularize their own library (and their users are equally ready and willing to upgrade to Java9) and after their dependencies have gone first before you let someone declare their chosen name in a stable way... well you're going to be waiting a long time. Maybe forever. On the contrary, if people start declaring their Module-Name now, and the rule against automodule dependencies is redefined such that it's ok to lean on something with a Module-Name, it becomes very easy and very quick for the ecosystem to get to a sane building point for full modularization. Without the Module-Name metadata or some equivalent, you are effectively barring all the build systems from helping with the conversion to achieve the very goal of this entire process. > > As to encouraging people to migrate to the modular world sooner rather > than later, just adding a `Module-Name` attribute doesn't really count > as migration. Actual migration requires you to write a proper module > declaration, and if the only way to claim a stable name is to do that > then this seems likely to encourage more developers to do so. > > - Mark > From mark.reinhold at oracle.com Thu Apr 6 20:54:34 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 6 Apr 2017 13:54:34 -0700 (PDT) Subject: Closing out still more open issues Message-ID: <20170406205434.54354A36B8@eggemoggin.niobe.net> In early March I posted proposals to resolve the following issues without implementing the changes that they suggest, either because they can be done in a future release or because they're not worth doing in the first place [1]: #AvoidConcealedPackageConflicts * #ExportAnnotation #MultiModuleExecutableJARs #MultiModuleJARs #MultipleModuleVersions * #ReifiedModuleGraphs #ResourceExistenceAndSize #StaticLayerConfiguration * #VersionsInModuleNames One EG member raised a concern about the three proposals marked with an asterisk above; otherwise there were no responses or only positive responses, so I've marked all these issues as resolved. No one objected to my more recent #MoveModuleAndLayerClasses proposal, and one member supported it, so I've marked that as resolved as well. The #AutomaticModuleNames and #ModuleNameInManifest issues are still under discussion. I have simple proposals in mind for the remaining three, #ModuleNameSyntax, #StandardModuleAttributes, and #VersionSyntax, which I will post shortly. - Mark [1] http://openjdk.java.net/projects/jigsaw/spec/issues/ From david.lloyd at redhat.com Thu Apr 6 20:56:03 2017 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 6 Apr 2017 15:56:03 -0500 Subject: Closing out still more open issues In-Reply-To: <20170406205434.54354A36B8@eggemoggin.niobe.net> References: <20170406205434.54354A36B8@eggemoggin.niobe.net> Message-ID: <7fd4af8e-bbd8-0279-99a8-d4a483e13ff2@redhat.com> On 04/06/2017 03:54 PM, mark.reinhold at oracle.com wrote: > In early March I posted proposals to resolve the following issues without > implementing the changes that they suggest, either because they can be > done in a future release or because they're not worth doing in the first > place [1]: > > #AvoidConcealedPackageConflicts * > #MultipleModuleVersions * > #StaticLayerConfiguration * We object, obviously. -- - DML From forax at univ-mlv.fr Fri Apr 7 09:27:22 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 7 Apr 2017 11:27:22 +0200 (CEST) Subject: Closing out still more open issues In-Reply-To: <20170406205434.54354A36B8@eggemoggin.niobe.net> References: <20170406205434.54354A36B8@eggemoggin.niobe.net> Message-ID: <1695355476.2316056.1491557242222.JavaMail.zimbra@u-pem.fr> I agree that the issues marked with an '*' should be deferred. As i have already said, i think that allowing to have different versions of the same module inside the same ModuleLayer should be possible but this should be solved at the VM level and not at the JDK level. The current class file format allows to specify version of the dependencies, so there are enough pieces in place to let people to experiment and try to come up with a solution. regards, R?mi ----- Mail original ----- > De: "mark reinhold" > ?: jpms-spec-experts at openjdk.java.net > Envoy?: Jeudi 6 Avril 2017 22:54:34 > Objet: Closing out still more open issues > In early March I posted proposals to resolve the following issues without > implementing the changes that they suggest, either because they can be > done in a future release or because they're not worth doing in the first > place [1]: > > #AvoidConcealedPackageConflicts * > #ExportAnnotation > #MultiModuleExecutableJARs > #MultiModuleJARs > #MultipleModuleVersions * > #ReifiedModuleGraphs > #ResourceExistenceAndSize > #StaticLayerConfiguration * > #VersionsInModuleNames > > One EG member raised a concern about the three proposals marked with > an asterisk above; otherwise there were no responses or only positive > responses, so I've marked all these issues as resolved. > > No one objected to my more recent #MoveModuleAndLayerClasses proposal, > and one member supported it, so I've marked that as resolved as well. > > The #AutomaticModuleNames and #ModuleNameInManifest issues are still > under discussion. I have simple proposals in mind for the remaining > three, #ModuleNameSyntax, #StandardModuleAttributes, and #VersionSyntax, > which I will post shortly. > > - Mark > > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/ From mark.reinhold at oracle.com Sun Apr 9 20:50:03 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Sun, 9 Apr 2017 13:50:03 -0700 (PDT) Subject: Updated JPMS Public Review Specification Message-ID: <20170409205003.409D6A4186@eggemoggin.niobe.net> I've updated the API section of the Public Review Specification [1] to make two agreed changes: - The names of automatic modules, as specified in the `ModuleFinder` [2] interface, can include trailing digits, per the accepted proposal [3]. - The `Module`, `Layer`, and `LayerInstantiationException` classes are moved from the `java.lang.reflect` package up into the `java.lang` package [4] and the names of the latter two are prefixed with `Module`, per the accepted proposal [5]. The JCP PMO will post this update shortly. - Mark [1] http://cr.openjdk.java.net/~mr/jigsaw/spec/ [2] http://cr.openjdk.java.net/~mr/jigsaw/spec/api/java/lang/module/ModuleFinder.html#of-java.nio.file.Path...- [3] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000659.html [4] http://cr.openjdk.java.net/~mr/jigsaw/spec/api/java/lang/package-summary.html [5] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000663.html From mark.reinhold at oracle.com Mon Apr 24 15:46:52 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Mon, 24 Apr 2017 08:46:52 -0700 (PDT) Subject: Proposal: #AutomaticModuleNames In-Reply-To: References: <20170403163015.90385A1987@eggemoggin.niobe.net> <1623657670.701980.1491302008159.JavaMail.zimbra@u-pem.fr> <20170404165330.587372716@eggemoggin.niobe.net> <326484726.1629758.1491465695348.JavaMail.zimbra@u-pem.fr> <20170406082121.691622335@eggemoggin.niobe.net> Message-ID: <20170424154652.709C1B5B74@eggemoggin.niobe.net> 2017/4/6 11:24:57 -0700, Brian Fox : > ... > > The vision that I had behind the Module Name was simply that we could make > it easy for module authors starting nearly immediately to choose their > module name. (There is disagreement within Maven committers over defaulting > this, so I seem to be in the minority of this and I think you don't have to > be afraid because of that.) We can make choosing and declaring a name easy, > maybe even required very soon for library authors. That means that we can > start to build up the very metadata that is missing and forcing you to lean > only on filename as the default. We can do so starting now, and by the time > jigsaw starts to hit critical mass, there will hopefully be very few > important libraries that aren't already properly named _by their authors_ > as you intend. > > The bar to picking a good name is clearly much lower than fully > modularizing...especially if you are barred or shamed into doing so before > all your dependencies have. If you both require that someone is fully ready > to modularize their own library (and their users are equally ready and > willing to upgrade to Java9) and after their dependencies have gone first > before you let someone declare their chosen name in a stable way... well > you're going to be waiting a long time. Maybe forever. > > On the contrary, if people start declaring their Module-Name now, and the > rule against automodule dependencies is redefined such that it's ok to lean > on something with a Module-Name, it becomes very easy and very quick for > the ecosystem to get to a sane building point for full modularization. > > Without the Module-Name metadata or some equivalent, you are effectively > barring all the build systems from helping with the conversion to achieve > the very goal of this entire process. Build systems and artifact repositories can help, but in a different way than you propose. Even if I did trust every tool maintainer not to abuse the `Module-Name` attribute then I'd still think that giving stable names to automatic modules is a fundamentally bad idea. The APIs of automatic modules are inherently unstable, so it's risky to publish an explicit module, i.e., one with a `module-info.class` file, that depends upon automatic modules. Giving stable names to automatic modules would just encourage that. Automatic modules are best viewed as a transitional tool that allows you to modularize an isolated, unpublished set of components in a top-down fashion over time. You can use existing published JAR files, from Maven Central or elsewhere, as automatic modules. The names and APIs of those automatic modules are unstable and will change as they are modularized explicitly, but in an isolated set of components you can easily adjust your own `module-info.java` files as needed to cope with those changes. Once all of your dependencies have been modularized then it's safe to publish your explicit modules, if desired. The modularization of a large repository of actively-used existing libraries is, by contrast, better done from the bottom up. If a library is modularized explicitly, and it depends upon some other libraries that have not yet been modularized and thus can only be used as automatic modules, then it's likely to break when those libraries are modularized. If this library is widely used in modular form then its users will be stuck with a broken module until its maintainer repairs the problem, and that repair may in turn require non-trivial changes to user modules. Bottom-up modularization will take time, and in the case of repositories as large as Maven Central it will never be complete since, inevitably, some libraries are abandoned over time. Available data suggest, however, that library popularity is a long-tail distribution, so many projects across the ecosystem should be able to modularize once the most popular libraries, and their dependencies, have modularized. Artifact repositories and build tools can be a big help in this process. If I maintain a library and want to modularize it and publish it as an explicit module then I need to wait until all of its dependencies have been modularized, as you correctly pointed out. If Maven Central and other repositories could track and publish the modularization status of every project then a build-tool plugin could tell me when it's safe to modularize my library. That plugin could even have a default setting that reports a fatal error if I try to publish a library that has non-modular dependencies. Repositories such as Central could enforce this constraint as part of their submission requirements. You wrote that it's impractical to block the publication of libraries that depend upon automatic modules, but it's not obvious to me why that's so. A library is either modularized or not, and all of its dependencies are either modularized or not. Version ranges in dependencies complicate the problem, but if we assume that once a library is modularized then it will remain modularized forever then it should suffice to check just the earliest version of each of a library's dependencies. Is there some other complexity that I'm missing here? - Mark From brianf at infinity.nu Mon Apr 24 16:39:16 2017 From: brianf at infinity.nu (Brian Fox) Date: Mon, 24 Apr 2017 12:39:16 -0400 Subject: Proposal: #AutomaticModuleNames In-Reply-To: <20170424154652.709C1B5B74@eggemoggin.niobe.net> References: <20170403163015.90385A1987@eggemoggin.niobe.net> <1623657670.701980.1491302008159.JavaMail.zimbra@u-pem.fr> <20170404165330.587372716@eggemoggin.niobe.net> <326484726.1629758.1491465695348.JavaMail.zimbra@u-pem.fr> <20170406082121.691622335@eggemoggin.niobe.net> <20170424154652.709C1B5B74@eggemoggin.niobe.net> Message-ID: On Mon, Apr 24, 2017 at 11:46 AM, wrote: > You wrote that it's impractical to block the publication of libraries > that depend upon automatic modules, but it's not obvious to me why that's > so. A library is either modularized or not, and all of its dependencies > are either modularized or not. Version ranges in dependencies complicate > the problem, but if we assume that once a library is modularized then it > will remain modularized forever then it should suffice to check just the > earliest version of each of a library's dependencies. Is there some > other complexity that I'm missing here? > It's more that central has many official, distributed entry points. Getting all of those updated to enforce a new rule is not always practical. If Central rejects it on ingest then this is problematic for the author as they've half released something (released in their forge, nak'd in central). People also tend to complain about the rules so this makes us the bad guys and gives fodder for less stringent repos to advertise "easy". In general this approach is a bit like putting the horses back in the barn instead of trying to keep the door shut to start with. From volker.berlin at goebel-clan.de Wed Apr 26 19:18:03 2017 From: volker.berlin at goebel-clan.de (Volker Berlin) Date: Wed, 26 Apr 2017 21:18:03 +0200 Subject: Compatibility of Java 8 libraries with Java 9 Message-ID: <3bdb7e69-46e9-5266-fc03-f6b46ded5ce1@goebel-clan.de> Hi, After reading many of migrate to Java 9 it look like that you expect that the people switch 100% to Java 9. I think this is wrong. Like with older Java versions it must be possible to run a library/application with Java 9, 8, ... One command line in a script must be run with any Java VM in the supported version range. If this will not be valid then Java 9 will be dead birth. All the new command line parameter of Java 9 does not work with Java 8. That there is no way to use any of this parameters until Java 9 is installed on a wide range of systems as runtime. That also with Java 9 a typical application will be start with: java -jar foo.jar If I start an application with this classic command line then any module info in the jar file is ignored. I have test it today. Is this correct? A hello world application will work. But a more complex application will fail. For example if the application use JavaMail API. The application need the extra modul java.activation. Now I need to add some of the new command line parameters. But then it work not with Java 8. I think we need a compatible way that can work with Java 7 to Java 9. Extra information, for example in the manifest.mf, that will be ignored in older Java versions. Of course the extra information can also be added via module-info.class. But then the call via -jar syntax should be read it. The same is valid for JNLP. Volker From Alan.Bateman at oracle.com Thu Apr 27 12:47:40 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 27 Apr 2017 13:47:40 +0100 Subject: Compatibility of Java 8 libraries with Java 9 In-Reply-To: <3bdb7e69-46e9-5266-fc03-f6b46ded5ce1@goebel-clan.de> References: <3bdb7e69-46e9-5266-fc03-f6b46ded5ce1@goebel-clan.de> Message-ID: On 26/04/2017 20:18, Volker Berlin wrote: > Hi, > > After reading many of migrate to Java 9 it look like that you expect > that the people switch 100% to Java 9. I think this is wrong. Like > with older Java versions it must be possible to run a > library/application with Java 9, 8, ... I suspect that the issues that you are concerned are not JSR-376 issues so it might be better to bring specific questions to jigsaw-dev for discussion. You mentioned the main manifest of the executable JARs and also JNLP and you will find some discussions in the archives on those topics already. -Alan From volker.berlin at goebel-clan.de Thu Apr 27 19:02:52 2017 From: volker.berlin at goebel-clan.de (Volker Berlin) Date: Thu, 27 Apr 2017 21:02:52 +0200 Subject: Compatibility of Java 8 libraries with Java 9 In-Reply-To: References: <3bdb7e69-46e9-5266-fc03-f6b46ded5ce1@goebel-clan.de> Message-ID: <85d56854-be89-d1b6-4047-00f877f784b3@goebel-clan.de> Am 27.04.2017 um 14:47 schrieb Alan Bateman: > On 26/04/2017 20:18, Volker Berlin wrote: > >> Hi, >> >> After reading many of migrate to Java 9 it look like that you expect >> that the people switch 100% to Java 9. I think this is wrong. Like >> with older Java versions it must be possible to run a >> library/application with Java 9, 8, ... > I suspect that the issues that you are concerned are not JSR-376 > issues so it might be better to bring specific questions to jigsaw-dev > for discussion. The cause of the compatibility problems are the module system of JSR-376 and its command line parameters. I understand your answer that it is not a target that one compiled application can run with Java 8 and Java 9. You prefer a migration and two versions of a application for different Java versions. > You mentioned the main manifest of the executable JARs and also JNLP > and you will find some discussions in the archives on those topics > already. > > -Alan > I have search the mail archive back to February 2015. I can not find any on this problem. Can you give a hint where I can find this in the large archive? Volker From Tim_Ellison at uk.ibm.com Fri Apr 28 15:48:43 2017 From: Tim_Ellison at uk.ibm.com (Tim Ellison) Date: Fri, 28 Apr 2017 16:48:43 +0100 Subject: Concerns with JPMS spec and Jigsaw implementation In-Reply-To: <20170427200209.55053B6A18@eggemoggin.niobe.net> References: <1622960697.16469710.1492226246622.JavaMail.zimbra@redhat.com> <1031408094.16469711.1492226283945.JavaMail.zimbra@redhat.com> <20170427200209.55053B6A18@eggemoggin.niobe.net> Message-ID: mark.reinhold at oracle.com wrote on 27/04/2017 21:02:09: > 2017/4/14 20:18:03 -0700, Scott Stark : > > Via our participation in JSR 376/JPMS we have raised a number of concerns > > regarding the implementation decision in Jigsaw as well as the scope and > > consensus of the expert group efforts. We, along with other EC members, and > > EG members have compiled a document that details these concerns. These > > concerns are outlined in the following blog: > > https://developer.jboss.org/blogs/scott.stark/2017/04/14/critical- > deficiencies-in-jigsawjsr-376-java-platform-module-system-ec-member-concerns > > This document does not raise any substantive technical issues that have not > already been discussed in the JSR 376 EG. I believe that document demonstrates there is still work required to bring the community closer to an agreement on the proposed standard. Even where the technical issues have already been discussed in the EG, it is incumbent upon this group to make best efforts to understand and resolve any differences. If people feel that their argument has not been considered, then it is not surprising that frustration ensues. I am personally disappointed that the move to call the Public Review ballot was made despite explicit objection from a number of EG members. > > As it stands, Red Hat will not vote for the approval of this public review > > draft of JPMS as it is not in the best interest of the Java community. > > Acknowledged. IBM is also voting "no" which reflects our position that the JSR is not ready at this time to move beyond the Public Review stage and proceed to Proposed Final Draft. The JSR 376 Expert Group and the public have raised a number of reasonable issues and concerns with the current public review draft of the specification that warrant further discussion and resolution. We advocate work continuing amongst all members of the Expert Group to address the issues documented on the mailing lists. IBM would like to see closer consensus amongst the entire Expert Group before this specification proceeds to the next step. Regards, Tim Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU