From plugins at lingocoder.com Tue Nov 12 02:30:50 2019 From: plugins at lingocoder.com (Plugins) Date: Mon, 11 Nov 2019 19:30:50 -0700 Subject: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow Message-ID: <20191111193050.b6ff8ab09a6c540215cae158a90d7e03.dee7ed3045.wbe@email17.godaddy.com> Hi all, I might have spoken too soon in my last October email [1]. Ever since, I've been having a hell of a time trying to get Gradle's so-called ?shaded? API jar to play nicely with the other modules in my JPMS-build-enabling Gradle plugin [2] For some reason, Gradle's generated artifact redundantly duplicates all the packages of two modules from the JDK: javax.xml and jdk.xml.dom. Any other modules in the module path that read packages in either of those two system modules breaks the JPMS with your classic split package error: ---- $ java -ea src\eg\UnPatchable.java Exception in thread "main" java.lang.module.ResolutionException: Modules gradle.api and java.xml export package javax.xml.transform to module com.github.javaparser.symbolsolver.core at java.base/java.lang.module.Resolver.resolveFail(Resolver.java:885) at java.base/java.lang.module.Resolver.failTwoSuppliers(Resolver.java:797) at java.base/java.lang.module.Resolver.checkExportSuppliers(Resolver.java:718) at java.base/java.lang.module.Resolver.finish(Resolver.java:362) at java.base/java.lang.module.Configuration.(Configuration.java:141) at java.base/java.lang.module.Configuration.resolve(Configuration.java:424) at java.base/java.lang.module.Configuration.resolve(Configuration.java:256) at eg.UnPatchable.main(UnPatchable.java:21) ---- Or the same failure but with a different message: ---- $ java -ea --module-path lib src\eg\UnPatchable.java Error occurred during initialization of boot layer java.lang.module.ResolutionException: Module gradle.api contains package org.xml.sax, module java.xml exports package org.xml.sax to gradle.api ---- What the subject of this email is about, is that even attempting to patch the Gradle API jar does not fix the split package problem: ---- $ java -ea --module-path lib --add-modules ALL-MODULE-PATH \ --patch-module java.xml=lib\gradle-api-6.0-rc-2.jar \ --patch-module jdk.xml.dom=lib\gradle-api-6.0-rc-2.jar; \ --patch-module gradle.api=com.github.javaparser.symbolsolver.core src\eg\UnPatchable.java Error occurred during initialization of boot layer java.lang.module.ResolutionException: Modules gradle.api and java.xml export package javax.xml.validation to module com.github.javaparser.symbolsolver.core ---- Those errors are reproducible with these steps: 1. Download, unzip and cd into the attached example project [3] 2. From a terminal, enter the commands I entered in the snippets above I'm hoping that it's not as bad as I think it is. Hopefully, it's nothing worse than I just haven't learned what the right command is to fix this. Please, can jigsaw-dev shed some light on this so far intractable problem? TIA. ---- [1] http://bit.ly/014298 [2] http://bit.ly/mrJar [3] http://bit.ly/014309zip From Alan.Bateman at oracle.com Tue Nov 12 08:04:59 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 12 Nov 2019 08:04:59 +0000 Subject: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow In-Reply-To: <20191111193050.b6ff8ab09a6c540215cae158a90d7e03.dee7ed3045.wbe@email17.godaddy.com> References: <20191111193050.b6ff8ab09a6c540215cae158a90d7e03.dee7ed3045.wbe@email17.godaddy.com> Message-ID: On 12/11/2019 02:30, Plugins wrote: > Hi all, > > I might have spoken too soon in my last October email [1]. Ever since, > I've > been having a hell of a time trying to get Gradle's so-called > ?shaded? API jar > to play nicely with the other modules in my JPMS-build-enabling Gradle > plugin [2] > > For some reason, Gradle's generated artifact redundantly duplicates all > the > packages of two modules from the JDK: javax.xml and jdk.xml.dom. The Gradle devs will know but I assume these are coming from the xml-apis-${version}.jar (Apache Xerces project). It's not clear, to me anyway, why these API classes are still being distributed in 2019. The XML processing API was added to Java SE 1.4, the XML stream API came a bit later. The XML APIs started out in Java SE as "standalone technologies" where they were in the JDK but could be upgraded by deploying a newer version with the endorsed standards override mechanism. It's possible, and probable, that developers have been deploying xml-apis on the class path and got lucky that it didn't cause problems. Anyway, none of this makes sense since Java SE 9 as both JSR-206 and JSR-173 ceased to be standalone technologies (see the final MR for both JSRs), and mechanism to upgrade them (the endorsed standards override mechanism) has been removed. -Alan. From plugins at lingocoder.com Tue Nov 12 09:19:56 2019 From: plugins at lingocoder.com (Plugins) Date: Tue, 12 Nov 2019 02:19:56 -0700 Subject: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow Message-ID: <20191112021956.b6ff8ab09a6c540215cae158a90d7e03.c22ffbb954.wbe@email17.godaddy.com> Hi Alan, > ...why these API classes are still being distributed in 2019... > ...none of this makes sense since Java SE 9 as both > JSR-206 and JSR-173 ceased to be standalone technologies... You've put what I was thinking all along, into the perfect words there. Thanks, Alan. If you can answer a few questions for me please Alan, you could help salvage the little sanity I have left after wrestling with this: 1. What's the correct command to patch javax.xml and jdk.xml.dom with Gradle's API JAR? 2. If it is actually impossible to patch either of them with Gradle, what's your best guess for why that might be? 3. How can I _exclude_ javax.xml and jdk.xml.dom from the module graph? ? I have already tried --limit-modules, but haven't had any luck In the meantime, I will try to muster up the will to report the issue to the Gradle devs. TIA -------- Original Message -------- Subject: Re: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow From: Alan Bateman Date: Tue, November 12, 2019 1:04 am To: Plugins , jigsaw-dev at openjdk.java.net On 12/11/2019 02:30, Plugins wrote: > Hi all, > > I might have spoken too soon in my last October email [1]. Ever since, > I've > been having a hell of a time trying to get Gradle's so-called > ?shaded? API jar > to play nicely with the other modules in my JPMS-build-enabling Gradle > plugin [2] > > For some reason, Gradle's generated artifact redundantly duplicates all > the > packages of two modules from the JDK: javax.xml and jdk.xml.dom. The Gradle devs will know but I assume these are coming from the xml-apis-${version}.jar (Apache Xerces project). It's not clear, to me anyway, why these API classes are still being distributed in 2019. The XML processing API was added to Java SE 1.4, the XML stream API came a bit later. The XML APIs started out in Java SE as "standalone technologies" where they were in the JDK but could be upgraded by deploying a newer version with the endorsed standards override mechanism. It's possible, and probable, that developers have been deploying xml-apis on the class path and got lucky that it didn't cause problems. Anyway, none of this makes sense since Java SE 9 as both JSR-206 and JSR-173 ceased to be standalone technologies (see the final MR for both JSRs), and mechanism to upgrade them (the endorsed standards override mechanism) has been removed. -Alan. From Alan.Bateman at oracle.com Tue Nov 12 11:50:51 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 12 Nov 2019 11:50:51 +0000 Subject: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow In-Reply-To: <20191112021956.b6ff8ab09a6c540215cae158a90d7e03.c22ffbb954.wbe@email17.godaddy.com> References: <20191112021956.b6ff8ab09a6c540215cae158a90d7e03.c22ffbb954.wbe@email17.godaddy.com> Message-ID: On 12/11/2019 09:19, Plugins wrote: > : > > > If you can answer a few questions for me please Alan, you could help > salvage the little sanity I have left after wrestling with this: > > 1. What's the correct command to patch javax.xml and jdk.xml.dom with > Gradle's API JAR? > 2. If it is actually impossible to patch either of them with Gradle, > what's your best guess for why that might be? > 3. How can I _exclude_ javax.xml and jdk.xml.dom from the module graph? > ? I have already tried --limit-modules, but haven't had any luck > > In the meantime, I will try to muster up the will to report the issue > to the Gradle devs. > You can't use --patch-module to augment both java.xml and jdk.xml.dom with the classes/resources from gradle-api-6.0-rc-2.jar as that will create a split package issue. You might get away with `--limit-modules java.se` and patching java.xml but it would be really odd to have classes for Gradle, SLF4J, JodaTime, and more in the java.xml module. I think the only sane approach is to deconstruct gradle-api-6.0-rc-2.jar so that the components it contains can be used as automatic modules. I'm not sure what to say about xml-apis. I assume it ends up in gradle-api-6.0-rc-2.jar because something depends on it. I can't think why anything might be depending on this relic in 2019. It might be useful track down why it is included in the first place. -Alan [1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=536928 From plugins at lingocoder.com Tue Nov 12 13:30:33 2019 From: plugins at lingocoder.com (Plugins) Date: Tue, 12 Nov 2019 06:30:33 -0700 Subject: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow Message-ID: <20191112063033.b6ff8ab09a6c540215cae158a90d7e03.112dc58ae1.wbe@email17.godaddy.com> > ...You might get away with `--limit-modules > java.se` and patching java.xml... Awesome! I will give that a shot. Many thanks! > ...it would be really odd to have classes for Gradle, > SLF4J, JodaTime, and more in the java.xml module... I take your advice from JEP 261's ?Patching module content? section to heart: ?The --patch-module option is intended only for testing and debugging. Its use in production settings is strongly discouraged? [1] My reasons for patching is only to allow temporary access to the classes under test by unit tests. Since those are conventionally in the same package but in two different Gradle source sets. > ...It might be useful track down why it is included in the first place... I agree. I will see what I can find out about that. Thanks for the Eclipse bug link too, Alan. Super informative! [1] http://bit.ly/JEP261Patch -------- Original Message -------- Subject: Re: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow From: Alan Bateman Date: Tue, November 12, 2019 4:50 am To: Plugins , jigsaw-dev at openjdk.java.net On 12/11/2019 09:19, Plugins wrote: > : > > > If you can answer a few questions for me please Alan, you could help > salvage the little sanity I have left after wrestling with this: > > 1. What's the correct command to patch javax.xml and jdk.xml.dom with > Gradle's API JAR? > 2. If it is actually impossible to patch either of them with Gradle, > what's your best guess for why that might be? > 3. How can I _exclude_ javax.xml and jdk.xml.dom from the module graph? > ? I have already tried --limit-modules, but haven't had any luck > > In the meantime, I will try to muster up the will to report the issue > to the Gradle devs. > You can't use --patch-module to augment both java.xml and jdk.xml.dom with the classes/resources from gradle-api-6.0-rc-2.jar as that will create a split package issue. You might get away with `--limit-modules java.se` and patching java.xml but it would be really odd to have classes for Gradle, SLF4J, JodaTime, and more in the java.xml module. I think the only sane approach is to deconstruct gradle-api-6.0-rc-2.jar so that the components it contains can be used as automatic modules. I'm not sure what to say about xml-apis. I assume it ends up in gradle-api-6.0-rc-2.jar because something depends on it. I can't think why anything might be depending on this relic in 2019. It might be useful track down why it is included in the first place. -Alan [1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=536928 From tjwats at gmail.com Tue Nov 12 16:48:21 2019 From: tjwats at gmail.com (Thomas Watson) Date: Tue, 12 Nov 2019 10:48:21 -0600 Subject: Doubts on automatic modules and what is exported at runtime In-Reply-To: <5bc266b0-2fb3-17b4-77d6-5eae34395241@oracle.com> References: <5bc266b0-2fb3-17b4-77d6-5eae34395241@oracle.com> Message-ID: Sorry to revisit this old topic. I was testing this again with the latest Java 11 on adopt open jdk and it still appears to be an issue. Did this bug ever get opened and addressed? Thanks Tom On Fri, Oct 5, 2018 at 5:04 PM Alan Bateman wrote: > On 05/10/2018 22:23, Thomas Watson wrote: > > : > > > > When launching Java I can use a module path that includes automatic > modules > > along with normal modules that require the automatic module names. > > Everything works as expected, my normal modules are able to access the > > packages included in the automatic module as if they are exported. Now I > > try to use the ModuleLayer API to create a new layer which includes the > > same set of automatic and normal modules. If I use one of > > the defineModulesWithOneLoader method it all works just like it would > when > > loading them from the module path. But if I instead use one of > > the defineModulesWithManyLoaders my normal modules that requires one of > the > > automatic modules can no longer load classes from the automatic module. > You are correct. For module M1 and M2, mapped to class loaders L1 and > L2: if M1 reads M2 then L1 needs to delegate to L2 for each package that > M2 exports to M1. If M2 is an automatic module then it doesn't have any > exports to this is why the delegation isn't setup. I'll create a bug for > this. > > -Alan > From Alan.Bateman at oracle.com Tue Nov 12 16:58:14 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 12 Nov 2019 16:58:14 +0000 Subject: Doubts on automatic modules and what is exported at runtime In-Reply-To: References: <5bc266b0-2fb3-17b4-77d6-5eae34395241@oracle.com> Message-ID: <38467443-100b-291a-4bda-1be194f0e8dd@oracle.com> On 12/11/2019 16:48, Thomas Watson wrote: > Sorry to revisit this old topic.? I was testing this again with the > latest Java 11 on adopt open jdk and it still appears to be an issue. > Did this bug ever get opened and addressed? Yes, it was fixed in JDK 12 via JDK-8211825 [1][2]. -Alan [1] https://bugs.openjdk.java.net/browse/JDK-8211825 [2] https://mail.openjdk.java.net/pipermail/jigsaw-dev/2018-October/013977.html From plugins at lingocoder.com Tue Nov 12 23:45:41 2019 From: plugins at lingocoder.com (Plugins) Date: Tue, 12 Nov 2019 16:45:41 -0700 Subject: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow Message-ID: <20191112164541.b6ff8ab09a6c540215cae158a90d7e03.11c980969f.wbe@email17.godaddy.com> > ...You might get away with `--limit-modules > java.se` and patching java.xml... ?--limit-modules java.se? wasn't enough. To get the java launcher past some ?error: ...? messages, I needed to add a few more modules: ?--limit-modules java.se,jdk.compiler,jdk.zipfs?. But then I was back to the same ResolutionException blocker that had originally caused me to lose the will to live: $ java -ea --limit-modules java.se,jdk.compiler,jdk.zipfs --patch-module java.xml=lib\gradle-api-6.0-rc-2.jar src\eg\UnPatchable.java Exception in thread "main" java.lang.module.ResolutionException: Module gradle.api contains package org.w3c.dom.views, module java.xml exports package org.w3c.dom.views to gradle.api at java.base/java.lang.module.Resolver.resolveFail(Resolver.java:885) at java.base/java.lang.module.Resolver.failTwoSuppliers(Resolver.java:789) at java.base/java.lang.module.Resolver.checkExportSuppliers(Resolver.java:737) at java.base/java.lang.module.Resolver.finish(Resolver.java:362) at java.base/java.lang.module.Configuration.(Configuration.java:141) at java.base/java.lang.module.Configuration.resolve(Configuration.java:424) at java.base/java.lang.module.Configuration.resolve(Configuration.java:256) at eg.UnPatchable.main(UnPatchable.java:24) It really is impossible to patch Gradle's API JAR, it looks like :( Please tell me I'm wrong? > ...I'm not sure what to say about xml-apis. I assume it ends up in > gradle-api-6.0-rc-2.jar because something depends on it. I can't think > why anything might be depending on this relic in 2019. It might be > useful track down why it is included in the first place. Gradle makes heavy use a lot of stuff in the javax.xml.* and org.w3c.* packages internally [1] And, again, this shaded jar we're talking about is generated on demand ? IFF a Gradle plugin author happens to have applied a specific internal Gradle plugin that is only ever applied to assist in the development of custom Gradle plugins. So this api jar is not actually ?distributed? in the same sense that other artifacts are normally ?distributed? ? through a public repository, say. I can't say with 100% certainty, but I _think_ everything contained in this shaded JAR is expected to never show up on the classpath(s) of end users of my plugin for example. So I think you're right, Alan. Those xml-apis dependencies in that shaded JAR are probably just one of a ton of other rebundled internal dependencies that Gradle itself needs to support the plugin development use case. ---- [1] http://bit.ly/xmlapisHits -------- Original Message -------- Subject: RE: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow From: "Plugins" Date: Tue, November 12, 2019 6:30 am To: "Alan Bateman" , jigsaw-dev at openjdk.java.net > ...You might get away with `--limit-modules > java.se` and patching java.xml... Awesome! I will give that a shot. Many thanks! > ...it would be really odd to have classes for Gradle, > SLF4J, JodaTime, and more in the java.xml module... I take your advice from JEP 261's ?Patching module content? section to heart: ?The --patch-module option is intended only for testing and debugging. Its use in production settings is strongly discouraged? [1] My reasons for patching is only to allow temporary access to the classes under test by unit tests. Since those are conventionally in the same package but in two different Gradle source sets. > ...It might be useful track down why it is included in the first place... I agree. I will see what I can find out about that. Thanks for the Eclipse bug link too, Alan. Super informative! [1] http://bit.ly/JEP261Patch -------- Original Message -------- Subject: Re: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow From: Alan Bateman Date: Tue, November 12, 2019 4:50 am To: Plugins , jigsaw-dev at openjdk.java.net On 12/11/2019 09:19, Plugins wrote: > : > > > If you can answer a few questions for me please Alan, you could help > salvage the little sanity I have left after wrestling with this: > > 1. What's the correct command to patch javax.xml and jdk.xml.dom with > Gradle's API JAR? > 2. If it is actually impossible to patch either of them with Gradle, > what's your best guess for why that might be? > 3. How can I _exclude_ javax.xml and jdk.xml.dom from the module graph? > ? I have already tried --limit-modules, but haven't had any luck > > In the meantime, I will try to muster up the will to report the issue > to the Gradle devs. > You can't use --patch-module to augment both java.xml and jdk.xml.dom with the classes/resources from gradle-api-6.0-rc-2.jar as that will create a split package issue. You might get away with `--limit-modules java.se` and patching java.xml but it would be really odd to have classes for Gradle, SLF4J, JodaTime, and more in the java.xml module. I think the only sane approach is to deconstruct gradle-api-6.0-rc-2.jar so that the components it contains can be used as automatic modules. I'm not sure what to say about xml-apis. I assume it ends up in gradle-api-6.0-rc-2.jar because something depends on it. I can't think why anything might be depending on this relic in 2019. It might be useful track down why it is included in the first place. -Alan [1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=536928 From Alan.Bateman at oracle.com Wed Nov 13 07:55:08 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 13 Nov 2019 07:55:08 +0000 Subject: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow In-Reply-To: <20191112164541.b6ff8ab09a6c540215cae158a90d7e03.11c980969f.wbe@email17.godaddy.com> References: <20191112164541.b6ff8ab09a6c540215cae158a90d7e03.11c980969f.wbe@email17.godaddy.com> Message-ID: On 12/11/2019 23:45, Plugins wrote: > Gradle makes heavy use a lot of stuff in the javax.xml.* and org.w3c.* > packages internally [1] And, again, this shaded jar we're talking about > is generated on demand ? IFF a Gradle plugin author happens to have > applied a specific internal Gradle plugin that is only ever applied to > assist in the development of custom Gradle plugins. So this api jar is > not actually ?distributed? in the same sense that other artifacts > are normally ?distributed? ? through a public repository, say. > > I can't say with 100% certainty, but I _think_ everything contained in > this shaded JAR is expected to never show up on the classpath(s) of end > users of my plugin for example. > > So I think you're right, Alan. Those xml-apis dependencies in that > shaded JAR are probably just one of a ton of other rebundled internal > dependencies that Gradle itself needs to support the plugin development > use case. > I've no doubt that Gradle is making use of the XML processing and W3C DOM APIs but they are provided by the JDK so I don't know why Gradle 6.0 includes xml-apis-1.4.01.jar in its distribution. If you run with `-Xlog:class+load` then I assume none of the javax.xml.* and org.w3c.* classes are loaded from this JAR file, this goes for JDK 8 and older too. I took a quite look at xml-apis-1.4.01.jar and it looks like a copy of the XML API classes (not the implementation) from a very old JDK or JAXP release. The class files v45.3 = JDK 1.0.2 :-) The class files seems to have been compiled in Dec 2009 and I suspect were compiled from the JAXP API classes that were in Java SE 6 (one of the classes defines a method that was added in Java SE 6). So my guess is that this JAR file has been obsolete for a long time. It may have been used for deployments on JDK 5 or older that could upgrade the JDK but want to use newer versions of the XML APIs. Alternatively, maybe it was useful for the period when the JDK didn't include the all of the W3C API. Hopefully someone from Gradle can comment on this issue, my guess is that xml-apis is not needed. -Alan From dawid.weiss at gmail.com Thu Nov 14 10:07:14 2019 From: dawid.weiss at gmail.com (Dawid Weiss) Date: Thu, 14 Nov 2019 11:07:14 +0100 Subject: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow In-Reply-To: References: <20191112164541.b6ff8ab09a6c540215cae158a90d7e03.11c980969f.wbe@email17.godaddy.com> Message-ID: Hi Alan, > I took a quite look at xml-apis-1.4.01.jar and it looks like a copy of > the XML API classes (not the implementation) from a very old JDK or JAXP > release. The class files v45.3 = JDK 1.0.2 :-) Just FYI: the latest Apache Xerces release (that's June, 2018) still depends on this jar... http://repo1.maven.org/maven2/xerces/xercesImpl/2.12.0/xercesImpl-2.12.0.pom Dawid From Alan.Bateman at oracle.com Thu Nov 14 11:32:33 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Nov 2019 11:32:33 +0000 Subject: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow In-Reply-To: References: <20191112164541.b6ff8ab09a6c540215cae158a90d7e03.11c980969f.wbe@email17.godaddy.com> Message-ID: <0aee87ad-f3aa-94b3-45ff-114466afa588@oracle.com> On 14/11/2019 10:07, Dawid Weiss wrote: > : > Just FYI: the latest Apache Xerces release (that's June, 2018) still > depends on this jar... > > http://repo1.maven.org/maven2/xerces/xercesImpl/2.12.0/xercesImpl-2.12.0.pom > A release in 2018 suggests it may still be maintained.? Do you, or anyone else, have cycles to reach out to this project to help them? It's not clear to me why they would want developers to downgrade the XML API to where it was in Java SE 6. -Alan From dawid.weiss at gmail.com Thu Nov 14 17:52:35 2019 From: dawid.weiss at gmail.com (Dawid Weiss) Date: Thu, 14 Nov 2019 18:52:35 +0100 Subject: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow In-Reply-To: <0aee87ad-f3aa-94b3-45ff-114466afa588@oracle.com> References: <20191112164541.b6ff8ab09a6c540215cae158a90d7e03.11c980969f.wbe@email17.godaddy.com> <0aee87ad-f3aa-94b3-45ff-114466afa588@oracle.com> Message-ID: They seem to be aware about it -- see this issue, for example: https://issues.apache.org/jira/browse/XERCESJ-1689 Dawid On Thu, Nov 14, 2019 at 12:33 PM Alan Bateman wrote: > > On 14/11/2019 10:07, Dawid Weiss wrote: > > : > > Just FYI: the latest Apache Xerces release (that's June, 2018) still > > depends on this jar... > > > > http://repo1.maven.org/maven2/xerces/xercesImpl/2.12.0/xercesImpl-2.12.0.pom > > > A release in 2018 suggests it may still be maintained. Do you, or > anyone else, have cycles to reach out to this project to help them? It's > not clear to me why they would want developers to downgrade the XML API > to where it was in Java SE 6. > > -Alan From Alan.Bateman at oracle.com Thu Nov 14 18:50:46 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Nov 2019 18:50:46 +0000 Subject: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow In-Reply-To: References: <20191112164541.b6ff8ab09a6c540215cae158a90d7e03.11c980969f.wbe@email17.godaddy.com> <0aee87ad-f3aa-94b3-45ff-114466afa588@oracle.com> Message-ID: <935a2088-ab4a-25a4-4cf1-dc45d7dbcbc5@oracle.com> On 14/11/2019 17:52, Dawid Weiss wrote: > They seem to be aware about it -- see this issue, for example: > https://issues.apache.org/jira/browse/XERCESJ-1689 > Thanks but that looks to be xercesImpl.jar. The main troublemaker seems to be xml-apis.jar as it has a copy of the JAXP APIs from JDK 6 or older. -Alan From dawid.weiss at gmail.com Fri Nov 15 07:07:47 2019 From: dawid.weiss at gmail.com (Dawid Weiss) Date: Fri, 15 Nov 2019 08:07:47 +0100 Subject: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow In-Reply-To: <935a2088-ab4a-25a4-4cf1-dc45d7dbcbc5@oracle.com> References: <20191112164541.b6ff8ab09a6c540215cae158a90d7e03.11c980969f.wbe@email17.godaddy.com> <0aee87ad-f3aa-94b3-45ff-114466afa588@oracle.com> <935a2088-ab4a-25a4-4cf1-dc45d7dbcbc5@oracle.com> Message-ID: I realize this. However xercesImpl has a POM dependency on those xml-apis -- this dependency brings this JAR in for other projects (like mine). I have no idea who published xml-apis [1] -- this is indeed ancient code. Sigh. Dawid [1] http://repo1.maven.org/maven2/xml-apis/xml-apis/ On Thu, Nov 14, 2019 at 7:51 PM Alan Bateman wrote: > > On 14/11/2019 17:52, Dawid Weiss wrote: > > They seem to be aware about it -- see this issue, for example: > > https://issues.apache.org/jira/browse/XERCESJ-1689 > > > Thanks but that looks to be xercesImpl.jar. The main troublemaker seems > to be xml-apis.jar as it has a copy of the JAXP APIs from JDK 6 or older. > > -Alan From Alan.Bateman at oracle.com Mon Nov 18 12:34:15 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 18 Nov 2019 12:34:15 +0000 Subject: 8233922: Service binding augments module graph with observable incubator modules Message-ID: This issue concerns the interaction of service binding with incubator modules, a left over from JDK 9 that resurfaces with the upcoming JEP 343 Packaging Tool (Incubator). The summary on this issue is that service binding augments the module graph with the modules induced by the service-use relation. This happens irrespective of the incubating status of the observable modules because the incubating designation is a JDK rather than Java SE/specified concept. It shows up with module `jdk.incubator.jpackage` because this module wants to implement the `java.util.spi.ToolProvider` service interface, a service interface used by `java.base`. If service binding were to resolve `jdk.incubator.jpackage` then a warning would be emitted in all phases that java.base is resolved (so everywhere). It's on my radar to more precisely specify service binding and also work out the specification changes needed to allow filtering of the observing modules but this is not needed at this time. Instead, we can limit the changes to the configuration for the boot layer as the set of observable modules when creating the configuration for the boot layer is not specified (it's up to the implementation). The change proposed here means that incubator modules are not candidates to add to the module graph during service binding for the boot layer's configuration. It has no impact on the general case where incubating modules may be observable when creating the configuration for custom layers. While in the area, I've also fixed the existing DefaultModules test to allow for incubator modules as this test has been failing the jpackage branch. We didn't have any incubator modules in JDK 11, when the test was added, to notice this oversight in the test. The webrev with the proposed changes is here. The reason that the jlink system modules plugin is updated is because it creates the configuration for the boot layer at link-time, for cases where we are skip resolution at run-time. ? http://cr.openjdk.java.net/~alanb/8233922/webrev/ -Alan. From alex.buckley at oracle.com Mon Nov 18 17:54:25 2019 From: alex.buckley at oracle.com (Alex Buckley) Date: Mon, 18 Nov 2019 09:54:25 -0800 Subject: 8233922: Service binding augments module graph with observable incubator modules In-Reply-To: References: Message-ID: <38162093-f47c-787d-fd20-ac06f301086a@oracle.com> On 11/18/2019 4:34 AM, Alan Bateman wrote: > The summary on this issue is that service binding augments the > module graph with the modules induced by the service-use relation. > ... If service binding were to resolve `jdk.incubator.jpackage` > then a warning would be emitted in all phases that java.base > is resolved (so everywhere). > > ... The change proposed here means that incubator modules are not > candidates to add to the module graph during service binding for the > boot layer's configuration. It has no impact on the general case > where incubating modules may be observable when creating the > configuration for custom layers. An incubator module's service providers will now be unavailable by default even if a module on the module path says `uses`. I believe that JEP 11 should say the following: ----- Incubator modules are part of the JDK run-time image produced by the standard JDK build. *****However, by default, incubator modules are not resolved for applications on the class path. Furthermore, by default, incubator modules do not participate in service binding for applications on the class path or the module path.***** Applications on the class path must use the --add-modules command-line option to request that an incubator module be resolved. Applications developed as modules can specify `requires` or `requires transitive` dependences upon an incubator module directly. *****(Some incubator modules do not export packages but rather provide implementations of services. It is usually not recommended to require a module that exports no packages, but it is necessary in this case in order to resolve the incubator module and thus have it participate in service binding.)***** During the JDK build, incubator modules must be packaged ... ----- Alex From Alan.Bateman at oracle.com Mon Nov 18 19:10:27 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 18 Nov 2019 19:10:27 +0000 Subject: 8233922: Service binding augments module graph with observable incubator modules In-Reply-To: <38162093-f47c-787d-fd20-ac06f301086a@oracle.com> References: <38162093-f47c-787d-fd20-ac06f301086a@oracle.com> Message-ID: On 18/11/2019 17:54, Alex Buckley wrote: > An incubator module's service providers will now be unavailable by > default even if a module on the module path says `uses`. I believe > that JEP 11 should say the following: > > ----- > Incubator modules are part of the JDK run-time image produced by the > standard JDK build. *****However, by default, incubator modules are > not resolved for applications on the class path. Furthermore, by > default, incubator modules do not participate in service binding for > applications on the class path or the module path.***** That looks good. > > Applications on the class path must use the --add-modules command-line > option to request that an incubator module be resolved. Applications > developed as modules can specify `requires` or `requires transitive` > dependences upon an incubator module directly. *****(Some incubator > modules do not export packages but rather provide implementations of > services. It is usually not recommended to require a module that > exports no packages, but it is necessary in this case in order to > resolve the incubator module and thus have it participate in service > binding.)***** We could potentially say something about tools that are incubator modules here. The launcher for the tool, "jpackage" in the case of JEP 343, will use the name of the incubator module as the root module to resolve. > > During the JDK build, incubator modules must be packaged ... There's a typo in this paragraph in that the jmod (and jar) tools use "--warn-if-resolved=incubating" rather than "--warn-if-resolved=incubator" to package an incubator module. -Alan From alex.buckley at oracle.com Mon Nov 18 19:34:50 2019 From: alex.buckley at oracle.com (Alex Buckley) Date: Mon, 18 Nov 2019 11:34:50 -0800 Subject: 8233922: Service binding augments module graph with observable incubator modules In-Reply-To: References: <38162093-f47c-787d-fd20-ac06f301086a@oracle.com> Message-ID: On 11/18/2019 11:10 AM, Alan Bateman wrote: > We could potentially say something about tools that are incubator > modules here. The launcher for the tool, "jpackage" in the case of JEP > 343, will use the name of the incubator module as the root module to > resolve. I updated JEP 11 to allude to incubator modules containing tools, hence no exported API but rather a service provider. >> During the JDK build, incubator modules must be packaged ... > There's a typo in this paragraph in that the jmod (and jar) tools use > "--warn-if-resolved=incubating" rather than > "--warn-if-resolved=incubator" to package an incubator module. JEP corrected to say `incubating`. Alex From mandy.chung at oracle.com Mon Nov 18 20:13:35 2019 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 18 Nov 2019 12:13:35 -0800 Subject: 8233922: Service binding augments module graph with observable incubator modules In-Reply-To: References: Message-ID: <66de0200-fafc-1946-e169-e5561b73997e@oracle.com> On 11/18/19 4:34 AM, Alan Bateman wrote: > > This issue concerns the interaction of service binding with incubator > modules, a left over from JDK 9 that resurfaces with the upcoming JEP > 343 Packaging Tool (Incubator). > > The summary on this issue is that service binding augments the module > graph with the modules induced by the service-use relation. This > happens irrespective of the incubating status of the observable > modules because the incubating designation is a JDK rather than Java > SE/specified concept. It shows up with module `jdk.incubator.jpackage` > because this module wants to implement the > `java.util.spi.ToolProvider` service interface, a service interface > used by `java.base`. If service binding were to resolve > `jdk.incubator.jpackage` then a warning would be emitted in all phases > that java.base is resolved (so everywhere). > > It's on my radar to more precisely specify service binding and also > work out the specification changes needed to allow filtering of the > observing modules but this is not needed at this time. Instead, we can > limit the changes to the configuration for the boot layer as the set > of observable modules when creating the configuration for the boot > layer is not specified (it's up to the implementation). The change > proposed here means that incubator modules are not candidates to add > to the module graph during service binding for the boot layer's > configuration. It has no impact on the general case where incubating > modules may be observable when creating the configuration for custom > layers. > Limiting this change to the boot layer's configuration is a good solution that allows time to work out the specification change. > : > ? http://cr.openjdk.java.net/~alanb/8233922/webrev/ The patch looks good to me. Mandy From chris.hegarty at oracle.com Tue Nov 19 12:43:33 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 19 Nov 2019 12:43:33 +0000 Subject: 8233922: Service binding augments module graph with observable incubator modules In-Reply-To: References: Message-ID: <31B01D52-7321-48C9-9491-72D8B19007E3@oracle.com> > On 18 Nov 2019, at 12:34, Alan Bateman wrote: > > ... > http://cr.openjdk.java.net/~alanb/8233922/webrev/ I think this looks ok. The original patch that added incubator module support had similar changes ( to exclude incubator modules from service binding ), but it impacted on the public Configuration::resolveAndBind ( so backed away from doing this ). Your changes to limit this to the boot layer are good ( since it is unlikely to be a more general issue ). I?m surprise to see the resolver use `mres.hasIncubatingWarning() == false`, rather than ModuleResolution.doNotResolveByDefault(mres). -Chris. From kevin.rushforth at oracle.com Tue Nov 19 13:13:00 2019 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Tue, 19 Nov 2019 05:13:00 -0800 Subject: 8233922: Service binding augments module graph with observable incubator modules In-Reply-To: <66de0200-fafc-1946-e169-e5561b73997e@oracle.com> References: <66de0200-fafc-1946-e169-e5561b73997e@oracle.com> Message-ID: <7a3692c4-0177-1d72-0e01-4f1cce1b5d42@oracle.com> Alan, Thanks for getting this fixed. I verified that the jpacakge ToolProvider implementation now works as expected. -- Kevin On 11/18/2019 12:13 PM, Mandy Chung wrote: > > On 11/18/19 4:34 AM, Alan Bateman wrote: >> >> This issue concerns the interaction of service binding with incubator >> modules, a left over from JDK 9 that resurfaces with the upcoming JEP >> 343 Packaging Tool (Incubator). >> >> The summary on this issue is that service binding augments the module >> graph with the modules induced by the service-use relation. This >> happens irrespective of the incubating status of the observable >> modules because the incubating designation is a JDK rather than Java >> SE/specified concept. It shows up with module >> `jdk.incubator.jpackage` because this module wants to implement the >> `java.util.spi.ToolProvider` service interface, a service interface >> used by `java.base`. If service binding were to resolve >> `jdk.incubator.jpackage` then a warning would be emitted in all >> phases that java.base is resolved (so everywhere). >> >> It's on my radar to more precisely specify service binding and also >> work out the specification changes needed to allow filtering of the >> observing modules but this is not needed at this time. Instead, we >> can limit the changes to the configuration for the boot layer as the >> set of observable modules when creating the configuration for the >> boot layer is not specified (it's up to the implementation). The >> change proposed here means that incubator modules are not candidates >> to add to the module graph during service binding for the boot >> layer's configuration. It has no impact on the general case where >> incubating modules may be observable when creating the configuration >> for custom layers. >> > > Limiting this change to the boot layer's configuration is a good > solution that allows time to work out the specification change. > >> : >> ? http://cr.openjdk.java.net/~alanb/8233922/webrev/ > > The patch looks good to me. > > Mandy > From Alan.Bateman at oracle.com Tue Nov 19 13:51:03 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 19 Nov 2019 13:51:03 +0000 Subject: 8233922: Service binding augments module graph with observable incubator modules In-Reply-To: <31B01D52-7321-48C9-9491-72D8B19007E3@oracle.com> References: <31B01D52-7321-48C9-9491-72D8B19007E3@oracle.com> Message-ID: On 19/11/2019 12:43, Chris Hegarty wrote: > : > > I?m surprise to see the resolver use `mres.hasIncubatingWarning() == false`, rather than ModuleResolution.doNotResolveByDefault(mres). > I think you are asking about the semantics of the two flags in the JDK-specific ModuleResolution class file attribute. Some day we should probably document it although I'm not sure if it's the j.l.module package description, JEP 11, or somewhere else. The warn-incubating flag is used in all phases to indicate that a module is an incubator module and to warn when the module is resolved. Technically these two concerns could be separated but this need hasn't arisen to date. The do-not-resolve-by-default flag excludes modules exporting APIs from the default set of named modules to resolve when the initial module is the unnamed module, i.e. class path applications. Technically this could be used to exclude modules that aren't incubator modules (and I think we've got close once or two to doing that). -Alan. From chris.hegarty at oracle.com Tue Nov 19 14:27:15 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 19 Nov 2019 14:27:15 +0000 Subject: 8233922: Service binding augments module graph with observable incubator modules In-Reply-To: References: <31B01D52-7321-48C9-9491-72D8B19007E3@oracle.com> Message-ID: <0E61B76F-BB40-4BD6-8393-67CDA1827B03@oracle.com> > On 19 Nov 2019, at 13:51, Alan Bateman wrote: > > ... > The warn-incubating flag is used in all phases to indicate that a module is an incubator module and to warn when the module is resolved. Technically these two concerns could be separated but this need hasn't arisen to date. > > The do-not-resolve-by-default flag excludes modules exporting APIs from the default set of named modules to resolve when the initial module is the unnamed module, i.e. class path applications. Technically this could be used to exclude modules that aren't incubator modules (and I think we've got close once or two to doing that). The original design of ModuleResolution intended do-not-resolve-by-default to be consulted by the resolver when making decisions about whether or not to include the module in the resolution process by default. The warn-if-resolved value was intended to be used by tools and the launcher, to provide a warning reason if the module in question was resolved. -Chris. From chris.hegarty at oracle.com Tue Nov 19 14:29:30 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 19 Nov 2019 14:29:30 +0000 Subject: 8233922: Service binding augments module graph with observable incubator modules In-Reply-To: <38162093-f47c-787d-fd20-ac06f301086a@oracle.com> References: <38162093-f47c-787d-fd20-ac06f301086a@oracle.com> Message-ID: <8B1C5EBD-07D5-4D7D-9B9C-8103C1B52A8B@oracle.com> > On 18 Nov 2019, at 17:54, Alex Buckley wrote: >> .. > An incubator module's service providers will now be unavailable by default even if a module on the module path says `uses`. I believe that JEP 11 should say the following: > > ----- > Incubator modules are part of the JDK run-time image produced by the standard JDK build. *****However, by default, incubator modules are not resolved for applications on the class path. Furthermore, by default, incubator modules do not participate in service binding for applications on the class path or the module path.***** > > Applications on the class path must use the --add-modules command-line option to request that an incubator module be resolved. Applications developed as modules can specify `requires` or `requires transitive` dependences upon an incubator module directly. *****(Some incubator modules do not export packages but rather provide implementations of services. It is usually not recommended to require a module that exports no packages, but it is necessary in this case in order to resolve the incubator module and thus have it participate in service binding.)***** > > During the JDK build, incubator modules must be packaged ... > ----- Thanks for the updates to JEP 11 Alex, they look good. -Chris. From Alan.Bateman at oracle.com Tue Nov 19 15:33:37 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 19 Nov 2019 15:33:37 +0000 Subject: 8233922: Service binding augments module graph with observable incubator modules In-Reply-To: <0E61B76F-BB40-4BD6-8393-67CDA1827B03@oracle.com> References: <31B01D52-7321-48C9-9491-72D8B19007E3@oracle.com> <0E61B76F-BB40-4BD6-8393-67CDA1827B03@oracle.com> Message-ID: <9f9d41fe-d361-477f-1aea-97b73531754e@oracle.com> On 19/11/2019 14:27, Chris Hegarty wrote: > : > The original design of ModuleResolution intended do-not-resolve-by-default to be consulted by the resolver when making decisions about whether or not to include the module in the resolution process by default. > > The warn-if-resolved value was intended to be used by tools and the launcher, to provide a warning reason if the module in question was resolved. > The resolver has never used the ModuleResolution attribute during resolution or service binding before now. To date, we've used the do-no-resolve-by-default to exclude modules that export an API from the default set of root modules. The semantics that we want for service binding is to say whether the module participates or not, maybe you are thinking we should expand the meaning of this flag to beyond the default modules case? I remember the warning reasons although I don't think it's possible to create an incubator module without the warn-incubating flag. The warn-incubating is the only way to know if a module is an incubator module. -Alan