Multi-Release JAR file patch as applied to build 108 of Java 9 breaks almost every project out there (Apache Ant, Gradle, partly Apache Maven)
Hi OpenJDK Core Developers, you may know the Apache Lucene team is testing early access releases of Java 9. We reported many bugs already, but most of them only applied to Hotspot and Lucene itsself. But this problem since build 108 is now really severe, because it breaks the build system already! To allow further testing of Open Source Projects, I'd suggest to revert the Multi-Release-JAR runtime support patch and provide a new preview build ASAP, because we found out after a night of debugging a build system from which we don't know all internals what is causing the problems and there is no workaround. I am very sorry that I have to say this, but it unfortunately build 108 breaks *ALL* versions of Apache Ant, the grandfather of all Java build systems :-) I know also OpenJDK is using it, too! So with Multi-Release JAR file patch applied (see http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c), any Ant-based build - including the JDK build itsself - would no longer bootstrap. It is impossible to also build Gradle projects, because Gradle uses Ant internally for many tasks). Maven projects may be affected, too. Now you might have the question: What happened? We tried to build Lucene on our Jenkins server, but the build itsself failed with a stupid error message: BUILD FAILED /home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:21: The following error occurred while executing this line: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:56: not doesn't support the nested "matches" element. The first idea was: Ah, there were changes in XML parsing (JDK-8149915). So we debugged the build. But it was quite clear that XML parsing was not the issue. It got quite clear when we enabled "-debug" on the build. What happened was that Ant was not loading its internal conditions/tasks/type definitions anymore, so the build system does not know almost any type anymore. The debug log showed that Ant was no longer able to load the resource "/org/apache/tools/ant/antlib.xml" from its own JAR file anymore. Instead it printed some strange debugging output (which looked totally broken). I spend the whole night digging through their code and found the issue: The commit of Multi-Release-Jar files (see http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c) broke resource handling in Apache Ant. In short: If you call ClassLoader.getResources() / or getResource() you get back an URL from where you can load the Resource - this is all fine and still works. But, with the Multi-Release JAR files patch this now has an URL fragment appended to the URL: '#release' (see http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c); this also applies to non-multi-release JAR files like Apache Ant's "ant.jar". In Java 7, Java 8,... and Java 9pre-b108, ClassLoader.getResource()/getResources() returned stuff like: "jar:file:/C:/Program%20Files/Java/apache-ant-1.9.6/lib/ant.jar!/org/apache/tools/ant/antlib.xml" Now in Java 9b108 the following is returned: "jar:file:/C:/Program%20Files/Java/apache-ant-1.9.6/lib/ant.jar!/org/apache/tools/ant/antlib.xml#release" And here Ant breaks (and I assume many other projects like Maven, too). Ant checks for the file extension of the string (because it may load definitions from both XML and properties files). So it does endsWith(".xml") and of course this now returns false. The effect is that Ant tries to load its own task definitions as a java properties file instead of XML. Of course this fails, because the data behind this URL is XML. The effect is that Ant cannot bootstrap as everything to build is missing. One might say: Ant's code is broken (I agree, it is not nice because it relies on the string representation of the resource URL - which is a no-go anyways), but it is impossible to fix, because Ant is bundled on most developer computers and those will suddenly break with Java 9! There is also no version out there that works around this, so we cannot test anything anymore! The problematic line in Ant's code is here: http://grepcode.com/file/repo1.maven.org/maven2/org.apache.ant/ant/1.9.6/org... I'd suggest to please ASAP revert the Multi-Release JAR file patch and provide a new preview build as soon as possible. I think there is more work needed to fix this. If this does not revert to the original state, it will be impossible to build and test Lucene, Elasticsearch,.... (and almost every Java project out there!). So short: We cannot test anymore and it is likely that we cannot support Java 9 anymore because the build system used by most Java projects behind the scenes does not bootstrap itself anymore. My suggestion would be to investigate other versions for this patch that does *not* modify the resource URLs by appending a fragment to them (at least not for the "standard" case without an actual Multi-Release Jar). For new multi-release JAR files I am fine with appending fragments, but please not for default ones. Maybe change code to handle the URLs from the non-versioned part differently (without fragment). Leaving the fragment inide may break many othe rprojects, because many programmers are very sloppy with handling URLs (well-known issue is calling URL#getFile() of a file:-URL that breaks on Windows systems and spaces in path name). Many people just call toString() on URL and do some test on it (startsWith, endsWith). So appending fragments is a no-go for backwards compatibility with JAR resources! I posted this to the mailing list and did not open a bug report on http://bugs.java.com/, because this is a more general issue - feel free to open bug reports around this!!! I would be very happy if we could find a quick solution for this problem. Until there is a solution we have to stop testing Java 9 with Apache Lucene/Solr/..., and this is not a good sign, especially as Jigsaw will be merged soon. Thanks for listening, Uwe P.S.: I also CCed the Apache Ant team. They should fix the broken code anyways, but this won't help for many projects already out there (e.g. Apache Lucene still has a minimum requirement of Ant 1.8.2 because MacOSX computers ship with that version since years). ----- Uwe Schindler uschindler@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/
Hi, similar issues were discovered too late to stop b108, e.g., https://bugs.openjdk.java.net/browse/JDK-8150920. Fix is already in jdk9/dev, so I think the next build should be more well-behaved and hope we can provide it more promptly than normal. If you can build OpenJDK from jdk9/dev and report any remaining issues due to the multi-release feature that would be quite helpful! Thanks! /Claes Uwe Schindler <uschindler@apache.org> skrev: (5 mars 2016 14:24:37 CET)
Hi OpenJDK Core Developers,
you may know the Apache Lucene team is testing early access releases of Java 9. We reported many bugs already, but most of them only applied to Hotspot and Lucene itsself. But this problem since build 108 is now really severe, because it breaks the build system already!
To allow further testing of Open Source Projects, I'd suggest to revert the Multi-Release-JAR runtime support patch and provide a new preview build ASAP, because we found out after a night of debugging a build system from which we don't know all internals what is causing the problems and there is no workaround. I am very sorry that I have to say this, but it unfortunately build 108 breaks *ALL* versions of Apache Ant, the grandfather of all Java build systems :-) I know also OpenJDK is using it, too! So with Multi-Release JAR file patch applied (see http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c), any Ant-based build - including the JDK build itsself - would no longer bootstrap. It is impossible to also build Gradle projects, because Gradle uses Ant internally for many tasks). Maven projects may be affected, too.
Now you might have the question: What happened?
We tried to build Lucene on our Jenkins server, but the build itsself failed with a stupid error message:
BUILD FAILED /home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:21: The following error occurred while executing this line: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:56: not doesn't support the nested "matches" element.
The first idea was: Ah, there were changes in XML parsing (JDK-8149915). So we debugged the build. But it was quite clear that XML parsing was not the issue. It got quite clear when we enabled "-debug" on the build. What happened was that Ant was not loading its internal conditions/tasks/type definitions anymore, so the build system does not know almost any type anymore. The debug log showed that Ant was no longer able to load the resource "/org/apache/tools/ant/antlib.xml" from its own JAR file anymore. Instead it printed some strange debugging output (which looked totally broken).
I spend the whole night digging through their code and found the issue: The commit of Multi-Release-Jar files (see http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c) broke resource handling in Apache Ant. In short: If you call ClassLoader.getResources() / or getResource() you get back an URL from where you can load the Resource - this is all fine and still works. But, with the Multi-Release JAR files patch this now has an URL fragment appended to the URL: '#release' (see http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c); this also applies to non-multi-release JAR files like Apache Ant's "ant.jar".
In Java 7, Java 8,... and Java 9pre-b108, ClassLoader.getResource()/getResources() returned stuff like:
"jar:file:/C:/Program%20Files/Java/apache-ant-1.9.6/lib/ant.jar!/org/apache/tools/ant/antlib.xml"
Now in Java 9b108 the following is returned:
"jar:file:/C:/Program%20Files/Java/apache-ant-1.9.6/lib/ant.jar!/org/apache/tools/ant/antlib.xml#release"
And here Ant breaks (and I assume many other projects like Maven, too). Ant checks for the file extension of the string (because it may load definitions from both XML and properties files). So it does endsWith(".xml") and of course this now returns false. The effect is that Ant tries to load its own task definitions as a java properties file instead of XML. Of course this fails, because the data behind this URL is XML. The effect is that Ant cannot bootstrap as everything to build is missing.
One might say: Ant's code is broken (I agree, it is not nice because it relies on the string representation of the resource URL - which is a no-go anyways), but it is impossible to fix, because Ant is bundled on most developer computers and those will suddenly break with Java 9! There is also no version out there that works around this, so we cannot test anything anymore!
The problematic line in Ant's code is here: http://grepcode.com/file/repo1.maven.org/maven2/org.apache.ant/ant/1.9.6/org...
I'd suggest to please ASAP revert the Multi-Release JAR file patch and provide a new preview build as soon as possible. I think there is more work needed to fix this. If this does not revert to the original state, it will be impossible to build and test Lucene, Elasticsearch,.... (and almost every Java project out there!). So short: We cannot test anymore and it is likely that we cannot support Java 9 anymore because the build system used by most Java projects behind the scenes does not bootstrap itself anymore.
My suggestion would be to investigate other versions for this patch that does *not* modify the resource URLs by appending a fragment to them (at least not for the "standard" case without an actual Multi-Release Jar). For new multi-release JAR files I am fine with appending fragments, but please not for default ones. Maybe change code to handle the URLs from the non-versioned part differently (without fragment). Leaving the fragment inide may break many othe rprojects, because many programmers are very sloppy with handling URLs (well-known issue is calling URL#getFile() of a file:-URL that breaks on Windows systems and spaces in path name). Many people just call toString() on URL and do some test on it (startsWith, endsWith). So appending fragments is a no-go for backwards compatibility with JAR resources!
I posted this to the mailing list and did not open a bug report on http://bugs.java.com/, because this is a more general issue - feel free to open bug reports around this!!! I would be very happy if we could find a quick solution for this problem. Until there is a solution we have to stop testing Java 9 with Apache Lucene/Solr/..., and this is not a good sign, especially as Jigsaw will be merged soon.
Thanks for listening, Uwe
P.S.: I also CCed the Apache Ant team. They should fix the broken code anyways, but this won't help for many projects already out there (e.g. Apache Lucene still has a minimum requirement of Ant 1.8.2 because MacOSX computers ship with that version since years).
----- Uwe Schindler uschindler@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/
-- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Hi Claes, is there a way to just build a new runtime library without compiling a full JDK (including Hotspot). So just replacing the jimage files locally? Uwe ----- Uwe Schindler uschindler@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ From: Claes Redestad [mailto:claes.redestad@oracle.com] Sent: Saturday, March 05, 2016 2:50 PM To: Uwe Schindler <uschindler@apache.org>; core-libs-dev@openjdk.java.net Cc: rory.odonnell@oracle.com; dev@ant.apache.org; bodewig@apache.org Subject: Re: Multi-Release JAR file patch as applied to build 108 of Java 9 breaks almost every project out there (Apache Ant, Gradle, partly Apache Maven) Hi, similar issues were discovered too late to stop b108, e.g., https://bugs.openjdk.java.net/browse/JDK-8150920. Fix is already in jdk9/dev, so I think the next build should be more well-behaved and hope we can provide it more promptly than normal. If you can build OpenJDK from jdk9/dev and report any remaining issues due to the multi-release feature that would be quite helpful! Thanks! /Claes Uwe Schindler <uschindler@apache.org <mailto:uschindler@apache.org> > skrev: (5 mars 2016 14:24:37 CET) Hi OpenJDK Core Developers, you may know the Apache Lucene team is testing early access releases of Java 9. We reported many bugs already, but most of them only applied to Hotspot and Lucene itsself. But this problem since build 108 is now really severe, because it breaks the build system already! To allow further testing of Open Source Projects, I'd suggest to revert the Multi-Release-JAR runtime support patch and provide a new preview build ASAP, because we found out after a night of debugging a build system from which we don't know all internals what is causing the problems and there is no workaround. I am very sorry that I have to say this, but it unfortunately build 108 breaks *ALL* versions of Apache Ant, the grandfather of all Java build systems :-) I know also OpenJDK is using it, too! So with Multi-Release JAR file patch applied (see http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c), any Ant-based build - including the JDK build itsself - would no longer bootstrap. It is impossible to also build Gradle projects, because Gradle uses Ant internally for many tasks). Maven projects may be affected, too. Now you might have the question: What happened? We tried to build Lucene on our Jenkins server, but the build itsself failed with a stupid error message: BUILD FAILED /home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:21: The following error occurred while executing this line: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:56: not doesn't support the nested "matches" element. The first idea was: Ah, there were changes in XML parsing (JDK-8149915). So we debugged the build. But it was quite clear that XML parsing was not the issue. It got quite clear when we enabled "-debug" on the build. What happened was that Ant was not loading its internal conditions/tasks/type definitions anymore, so the build system does not know almost any type anymore. The debug log showed that Ant was no longer able to load the resource "/org/apache/tools/ant/antlib.xml" from its own JAR file anymore. Instead it printed some strange debugging output (which looked totally broken). I spend the whole night digging through their code and found the issue: The commit of Multi-Release-Jar files (see http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c) broke resource handling in Apache Ant. In short: If you call ClassLoader.getResources() / or getResource() you get back an URL from where you can load the Resource - this is all fine and still works. But, with the Multi-Release JAR files patch this now has an URL fragment appended to the URL: '#release' (see http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c); this also applies to non-multi-release JAR files like Apache Ant's "ant.jar". In Java 7, Java 8,... and Java 9pre-b108, ClassLoader.getResource()/getResources() returned stuff like: "jar:file:/C:/Program%20Files/Java/apache-ant-1.9.6/lib/ant.jar!/org/apache/tools/ant/antlib.xml" Now in Java 9b108 the following is returned: "jar:file:/C:/Program%20Files/Java/apache-ant-1.9.6/lib/ant.jar!/org/apache/tools/ant/antlib.xml#release" And here Ant breaks (and I assume many other projects like Maven, too). Ant checks for the file extension of the string (because it may load definitions from both XML and properties files). So it does endsWith(".xml") and of course this now returns false. The effect is that Ant tries to load its own task definitions as a java properties file instead of XML. Of course this fails, because the data behind this URL is XML. The effect is that Ant cannot bootstrap as everything to build is missing. One might say: Ant's code is broken (I agree, it is not nice because it relies on the string representation of the resource URL - which is a no-go anyways), but it is impossible to fix, because Ant is bundled on most developer computers and those will suddenly break with Java 9! There is also no version out there that works around this, so we cannot test anything anymore! The problematic line in Ant's code is here: http://grepcode.com/file/repo1.maven.org/maven2/org.apache.ant/ant/1.9.6/org... I'd suggest to please ASAP revert the Multi-Release JAR file patch and provide a new preview build as soon as possible. I think there is more work needed to fix this. If this does not revert to the original state, it will be impossible to build and test Lucene, Elasticsearch,.... (and almost every Java project out there!). So short: We cannot test anymore and it is likely that we cannot support Java 9 anymore because the build system used by most Java projects behind the scenes does not bootstrap itself anymore. My suggestion would be to investigate other versions for this patch that does *not* modify the resource URLs by appending a fragment to them (at least not for the "standard" case without an actual Multi-Release Jar). For new multi-release JAR files I am fine with appending fragments, but please not for default ones. Maybe change code to handle the URLs from the non-versioned part differently (without fragment). Leaving the fragment inide may break many othe rprojects, because many programmers are very sloppy with handling URLs (well-known issue is calling URL#getFile() of a file:-URL that breaks on Windows systems and spaces in path name). Many people just call toString() on URL and do some test on it (startsWith, endsWith). So appending fragments is a no-go for backwards compatibility with JAR resources! I posted this to the mailing list and did not open a bug report on http://bugs.java.com/, because this is a more general issue - feel free to open bug reports around this!!! I would be very happy if we could find a quick solution for this problem. Until there is a solution we have to stop testing Java 9 with Apache Lucene/Solr/..., and this is not a good sign, especially as Jigsaw will be merged soon. Thanks for listening, Uwe P.S.: I also CCed the Apache Ant team. They should fix the broken code anyways, but this won't help for many projects already out there (e.g. Apache Lucene still has a minimum requirement of Ant 1.8.2 because MacOSX computers ship with that version since years). ----- Uwe Schindler <mailto:uschindler@apache.org> uschindler@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany <http://lucene.apache.org> http://lucene.apache.org/ -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Hi, Given the issues we are seeing, and I suspect this is not the only code with these assumptions, is there any way this functionality can be limited to "multi-release aware" code, either via a constructor parameter or a new method? What is the most elegant approach? Andre
On 5 Mar, 2016, at 08:50, Claes Redestad <claes.redestad@oracle.com> wrote:
Hi,
similar issues were discovered too late to stop b108, e.g., https://bugs.openjdk.java.net/browse/JDK-8150920. Fix is already in jdk9/dev, so I think the next build should be more well-behaved and hope we can provide it more promptly than normal.
If you can build OpenJDK from jdk9/dev and report any remaining issues due to the multi-release feature that would be quite helpful!
Thanks!
/Claes
Uwe Schindler <uschindler@apache.org> skrev: (5 mars 2016 14:24:37 CET)
Hi OpenJDK Core Developers,
you may know the Apache Lucene team is testing early access releases of Java 9. We reported many bugs already, but most of them only applied to Hotspot and Lucene itsself. But this problem since build 108 is now really severe, because it breaks the build system already!
To allow further testing of Open Source Projects, I'd suggest to revert the Multi-Release-JAR runtime support patch and provide a new preview build ASAP, because we found out after a night of debugging a build system from which we don't know all internals what is causing the problems and there is no workaround. I am very sorry that I have to say this, but it unfortunately build 108 breaks *ALL* versions of Apache Ant, the grandfather of all Java build systems :-) I know also OpenJDK is using it, too! So with Multi-Release JAR file patch applied (see http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c), any Ant-based build - including the JDK build itsself - would no longer bootstrap. It is impossible to also build Gradle projects, because Gradle uses Ant internally for many tasks). Maven projects may be affected, too.
Now you might have the question: What happened?
We tried to build Lucene on our Jenkins server, but the build itsself failed with a stupid error message:
BUILD FAILED /home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:21: The following error occurred while executing this line: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:56: not doesn't support the nested "matches" element.
The first idea was: Ah, there were changes in XML parsing (JDK-8149915). So we debugged the build. But it was quite clear that XML parsing was not the issue. It got quite clear when we enabled "-debug" on the build. What happened was that Ant was not loading its internal conditions/tasks/type definitions anymore, so the build system does not know almost any type anymore. The debug log showed that Ant was no longer able to load the resource "/org/apache/tools/ant/antlib.xml" from its own JAR file anymore. Instead it printed some strange debugging output (which looked totally broken).
I spend the whole night digging through their code and found the issue: The commit of Multi-Release-Jar files (see http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c) broke resource handling in Apache Ant. In short: If you call ClassLoader.getResources() / or getResource() you get back an URL from where you can load the Resource - this is all fine and still works. But, with the Multi-Release JAR files patch this now has an URL fragment appended to the URL: '#release' (see http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c); this also applies to non-multi-release JAR files like Apache Ant's "ant.jar".
In Java 7, Java 8,... and Java 9pre-b108, ClassLoader.getResource()/getResources() returned stuff like:
"jar:file:/C:/Program%20Files/Java/apache-ant-1.9.6/lib/ant.jar!/org/apache/tools/ant/antlib.xml"
Now in Java 9b108 the following is returned:
"jar:file:/C:/Program%20Files/Java/apache-ant-1.9.6/lib/ant.jar!/org/apache/tools/ant/antlib.xml#release"
And here Ant breaks (and I assume many other projects like Maven, too). Ant checks for the file extension of the string (because it may load definitions from both XML and properties files). So it does endsWith(".xml") and of course this now returns false. The effect is that Ant tries to load its own task definitions as a java properties file instead of XML. Of course this fails, because the data behind this URL is XML. The effect is that Ant cannot bootstrap as everything to build is missing.
One might say: Ant's code is broken (I agree, it is not nice because it relies on the string representation of the resource URL - which is a no-go anyways), but it is impossible to fix, because Ant is bundled on most developer computers and those will suddenly break with Java 9! There is also no version out there that works around this, so we cannot test anything anymore!
The problematic line in Ant's code is here: http://grepcode.com/file/repo1.maven.org/maven2/org.apache.ant/ant/1.9.6/org...
I'd suggest to please ASAP revert the Multi-Release JAR file patch and provide a new preview build as soon as possible. I think there is more work needed to fix this. If this does not revert to the original state, it will be impossible to build and test Lucene, Elasticsearch,.... (and almost every Java project out there!). So short: We cannot test anymore and it is likely that we cannot support Java 9 anymore because the build system used by most Java projects behind the scenes does not bootstrap itself anymore.
My suggestion would be to investigate other versions for this patch that does *not* modify the resource URLs by appending a fragment to them (at least not for the "standard" case without an actual Multi-Release Jar). For new multi-release JAR files I am fine with appending fragments, but please not for default ones. Maybe change code to handle the URLs from the non-versioned part differently (without fragment). Leaving the fragment inide may break many othe rprojects, because many programmers are very sloppy with handling URLs (well-known issue is calling URL#getFile() of a file:-URL that breaks on Windows systems and spaces in path name). Many people just call toString() on URL and do some test on it (startsWith, endsWith). So appending fragments is a no-go for backwards compatibility with JAR resources!
I posted this to the mailing list and did not open a bug report on http://bugs.java.com/, because this is a more general issue - feel free to open bug reports around this!!! I would be very happy if we could find a quick solution for this problem. Until there is a solution we have to stop testing Java 9 with Apache Lucene/Solr/..., and this is not a good sign, especially as Jigsaw will be merged soon.
Thanks for listening, Uwe
P.S.: I also CCed the Apache Ant team. They should fix the broken code anyways, but this won't help for many projects already out there (e.g. Apache Lucene still has a minimum requirement of Ant 1.8.2 because MacOSX computers ship with that version since years).
----- Uwe Schindler uschindler@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/
-- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Hi Andre,
On 6 Mar 2016, at 00:15, André-John Mas <andrejohn.mas@gmail.com> wrote:
Hi,
Given the issues we are seeing, and I suspect this is not the only code with these assumptions, is there any way this functionality can be limited to "multi-release aware" code, either via a constructor parameter or a new method? What is the most elegant approach?
For resource URLs, associated with an MR-JAR, and obtained from a class loader, here are three possible routes we could take: 1) Modify the resources URLs, cognisent of the known issues processing such URLs; 2) Resources URLs are reified; or 3) Resources URLs are not modified (meaning they are not runtime versioned). By 2) i mean that: URL u = loader.getResource(“foo/Bar.class”) may return u that is say: “jar:file:/….!/META-INF/versions/9/foo/Bar.class” rather than: “jar:file:/….!/foo/Bar.class” But we need to work through the implications of that approach. Paul.
On 5/03/2016 11:50 PM, Claes Redestad wrote:
Hi,
similar issues were discovered too late to stop b108, e.g., https://bugs.openjdk.java.net/browse/JDK-8150920. Fix is already in jdk9/dev, so I think the next build should be more well-behaved and hope we can provide it more promptly than normal.
As that bug leads to a non-open bug here's the changeset URL: http://hg.openjdk.java.net/jdk9/dev/jdk/rev/721288127c82 David
If you can build OpenJDK from jdk9/dev and report any remaining issues due to the multi-release feature that would be quite helpful!
Thanks!
/Claes
Uwe Schindler <uschindler@apache.org> skrev: (5 mars 2016 14:24:37 CET)
Hi OpenJDK Core Developers,
you may know the Apache Lucene team is testing early access releases of Java 9. We reported many bugs already, but most of them only applied to Hotspot and Lucene itsself. But this problem since build 108 is now really severe, because it breaks the build system already!
To allow further testing of Open Source Projects, I'd suggest to revert the Multi-Release-JAR runtime support patch and provide a new preview build ASAP, because we found out after a night of debugging a build system from which we don't know all internals what is causing the problems and there is no workaround. I am very sorry that I have to say this, but it unfortunately build 108 breaks *ALL* versions of Apache Ant, the grandfather of all Java build systems :-) I know also OpenJDK is using it, too! So with Multi-Release JAR file patch applied (see http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c), any Ant-based build - including the JDK build itsself - would no longer bootstrap. It is impossible to also build Gradle projects, because Gradle uses Ant internally for many tasks). Maven projects may be affected, too.
Now you might have the question: What happened?
We tried to build Lucene on our Jenkins server, but the build itsself failed with a stupid error message:
BUILD FAILED /home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:21: The following error occurred while executing this line: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:56: not doesn't support the nested "matches" element.
The first idea was: Ah, there were changes in XML parsing (JDK-8149915). So we debugged the build. But it was quite clear that XML parsing was not the issue. It got quite clear when we enabled "-debug" on the build. What happened was that Ant was not loading its internal conditions/tasks/type definitions anymore, so the build system does not know almost any type anymore. The debug log showed that Ant was no longer able to load the resource "/org/apache/tools/ant/antlib.xml" from its own JAR file anymore. Instead it printed some strange debugging output (which looked totally broken).
I spend the whole night digging through their code and found the issue: The commit of Multi-Release-Jar files (see http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c) broke resource handling in Apache Ant. In short: If you call ClassLoader.getResources() / or getResource() you get back an URL from where you can load the Resource - this is all fine and still works. But, with the Multi-Release JAR files patch this now has an URL fragment appended to the URL: '#release' (see http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c); this also applies to non-multi-release JAR files like Apache Ant's "ant.jar".
In Java 7, Java 8,... and Java 9pre-b108, ClassLoader.getResource()/getResources() returned stuff like:
"jar:file:/C:/Program%20Files/Java/apache-ant-1.9.6/lib/ant.jar!/org/apache/tools/ant/antlib.xml"
Now in Java 9b108 the following is returned:
"jar:file:/C:/Program%20Files/Java/apache-ant-1.9.6/lib/ant.jar!/org/apache/tools/ant/antlib.xml#release"
And here Ant breaks (and I assume many other projects like Maven, too). Ant checks for the file extension of the string (because it may load definitions from both XML and properties files). So it does endsWith(".xml") and of course this now returns false. The effect is that Ant tries to load its own task definitions as a java properties file instead of XML. Of course this fails, because the data behind this URL is XML. The effect is that Ant cannot bootstrap as everything to build is missing.
One might say: Ant's code is broken (I agree, it is not nice because it relies on the string representation of the resource URL - which is a no-go anyways), but it is impossible to fix, because Ant is bundled on most developer computers and those will suddenly break with Java 9! There is also no version out there that works around this, so we cannot test anything anymore!
The problematic line in Ant's code is here: http://grepcode.com/file/repo1.maven.org/maven2/org.apache.ant/ant/1.9.6/org...
I'd suggest to please ASAP revert the Multi-Release JAR file patch and provide a new preview build as soon as possible. I think there is more work needed to fix this. If this does not revert to the original state, it will be impossible to build and test Lucene, Elasticsearch,.... (and almost every Java project out there!). So short: We cannot test anymore and it is likely that we cannot support Java 9 anymore because the build system used by most Java projects behind the scenes does not bootstrap itself anymore.
My suggestion would be to investigate other versions for this patch that does *not* modify the resource URLs by appending a fragment to them (at least not for the "standard" case without an actual Multi-Release Jar). For new multi-release JAR files I am fine with appending fragments, but please not for default ones. Maybe change code to handle the URLs from the non-versioned part differently (without fragment). Leaving the fragment inide may break many othe rprojects, because many programmers are very sloppy with handling URLs (well-known issue is calling URL#getFile() of a file:-URL that breaks on Windows systems and spaces in path name). Many people just call toString() on URL and do some test on it (startsWith, endsWith). So appending fragments is a no-go for backwards compatibility with JAR resources!
I posted this to the mailing list and did not open a bug report on http://bugs.java.com/, because this is a more general issue - feel free to open bug reports around this!!! I would be very happy if we could find a quick solution for this problem. Until there is a solution we have to stop testing Java 9 with Apache Lucene/Solr/..., and this is not a good sign, especially as Jigsaw will be merged soon.
Thanks for listening, Uwe
P.S.: I also CCed the Apache Ant team. They should fix the broken code anyways, but this won't help for many projects already out there (e.g. Apache Lucene still has a minimum requirement of Ant 1.8.2 because MacOSX computers ship with that version since years).
----- Uwe Schindler uschindler@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/
On 05/03/2016 13:24, Uwe Schindler wrote:
:
I'd suggest to please ASAP revert the Multi-Release JAR file patch and provide a new preview build as soon as possible. I think there is more work needed to fix this. If this does not revert to the original state, it will be impossible to build and test Lucene, Elasticsearch,.... (and almost every Java project out there!). So short: We cannot test anymore and it is likely that we cannot support Java 9 anymore because the build system used by most Java projects behind the scenes does not bootstrap itself anymore.
Sigh, I think those of us that reviewed this missed the point that the fragment is appended by default. This will of course break code that parses URL strings in naive ways (anything looking for ".xml" should be looking at the path component of course). I'll create a bug for this now, assuming you haven't created one already. One general point is that the purpose of EA builds and timely testing by Lucene and other projects is invaluable for shaking out issues. There will be issues periodically and much better to find these within a few days of pushing a change rather than months later. -Alan
Thanks Alan, I am glad that the appending of "#resource" is indeed a bug.
I'd suggest to please ASAP revert the Multi-Release JAR file patch and provide a new preview build as soon as possible. I think there is more work needed to fix this. If this does not revert to the original state, it will be impossible to build and test Lucene, Elasticsearch,.... (and almost every Java project out there!). So short: We cannot test anymore and it is likely that we cannot support Java 9 anymore because the build system used by most Java projects behind the scenes does not bootstrap itself anymore.
Sigh, I think those of us that reviewed this missed the point that the fragment is appended by default. This will of course break code that parses URL strings in naive ways (anything looking for ".xml" should be looking at the path component of course).
This is why I put the Ant developers in CC. The correct way would be to look at the *decoded* path (not just getPath() because this is also one of the "famous" traps in the URL class - one reason why it should be avoided in favor of URI). URL.toURI().getPath() is most safe to fix the issue in Apache Ant (Stefan Bodewig: Should I open an issue in Ant?). Maybe Ant developers can fix this code in later versions to handle URLs more correct. In general there is lots of code outside that incorrectly uses URLs, because developers are lazy...
I'll create a bug for this now, assuming you haven't created one already.
No, I haven't. Thanks for doing this.
One general point is that the purpose of EA builds and timely testing by Lucene and other projects is invaluable for shaking out issues. There will be issues periodically and much better to find these within a few days of pushing a change rather than months later.
This is why we do this! The problem with the EA builds is still the large delay until pushes are appearing in builds. In most cases it takes > 2 weeks until an EA build contains pushed fixes. We are still waiting for fixes of JDK-8150280 and JDK-8150436 (duplicate of JDK-8148786). Both issues were resolved long time ago. The problem is if we have fatal issues like this, because it prevents testing the above bugs (once they are fixed). Thanks, Uwe
On 2016-03-05, Uwe Schindler wrote:
This is why I put the Ant developers in CC. The correct way would be to look at the *decoded* path (not just getPath() because this is also one of the "famous" traps in the URL class - one reason why it should be avoided in favor of URI). URL.toURI().getPath() is most safe to fix the issue in Apache Ant
Part of the reason for this certainly is that the code has been written before the URI class even existed.
(Stefan Bodewig: Should I open an issue in Ant?).
Yes, please do. Thanks Uwe.
Maybe Ant developers can fix this code in later versions to handle URLs more correct.
+1 Stefan
Hi Stefan,
-----Original Message----- From: Stefan Bodewig [mailto:bodewig@apache.org] Sent: Saturday, March 05, 2016 7:56 PM To: dev@ant.apache.org; Uwe Schindler <uschindler@apache.org> Cc: 'Alan Bateman' <Alan.Bateman@oracle.com>; core-libs- dev@openjdk.java.net; rory.odonnell@oracle.com; dev@ant.apache.org; 'Steve Drach' <steve.drach@oracle.com> Subject: Re: Multi-Release JAR file patch as applied to build 108 of Java 9 breaks almost every project out there (Apache Ant, Gradle, partly Apache Maven)
On 2016-03-05, Uwe Schindler wrote:
This is why I put the Ant developers in CC. The correct way would be to look at the *decoded* path (not just getPath() because this is also one of the "famous" traps in the URL class - one reason why it should be avoided in favor of URI). URL.toURI().getPath() is most safe to fix the issue in Apache Ant
Part of the reason for this certainly is that the code has been written before the URI class even existed.
(Stefan Bodewig: Should I open an issue in Ant?).
Yes, please do. Thanks Uwe.
I opened: https://bz.apache.org/bugzilla/show_bug.cgi?id=59130
Maybe Ant developers can fix this code in later versions to handle URLs more correct.
+1
Unfortunately this is not the only issue caused by this. After I tried to build Lucene with the patch applied, the next candidate for the issue broke: Apache Ivy. It was no longer able to load the ivy-settings.xml file from its JAR file. The reason here is another one: It constructs the JAR file URL on its own (it looks like this), but does not add the #release fragment. And because of this, JarURLConnection does not find the file...: [...] multiple parent causes [...] Caused by: java.io.FileNotFoundException: JAR entry org/apache/ivy/core/settings/ivysett/ivysettings-public.xml not found in C:\Users\Uwe Schindler\.ant\lib\ivy-2.3.0.jar at sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:142) at sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:150) at org.apache.ivy.util.url.BasicURLHandler.openStream(BasicURLHandler.java:171) at org.apache.ivy.util.url.URLHandlerDispatcher.openStream(URLHandlerDispatcher.java:74) at org.apache.ivy.core.settings.XmlSettingsParser.doParse(XmlSettingsParser.java:157) at org.apache.ivy.core.settings.XmlSettingsParser.parse(XmlSettingsParser.java:183) at org.apache.ivy.core.settings.XmlSettingsParser.includeStarted(XmlSettingsParser.java:435) at org.apache.ivy.core.settings.XmlSettingsParser.startElement(XmlSettingsParser.java:211) ... 35 more So it looks like the Multi-release JAR file patch also breaks the other way round: Code constructing JAR URLs according to the standard no longer work. In my opinion, the JAR URLs should not change at all and the code should transparently choose the right release version. Maybe add a fragment only to explicitly state a specific version (so one would be able to load the Java 7 version). But this could also be done using the META-INF/... path. The default handling should be that "old" and (I think they are standardized) JAR URLs still works as they should - not requiring the fragment! Uwe
This is why I put the Ant developers in CC. The correct way would be to look at the *decoded* path (not just getPath() because this is also one of the "famous" traps in the URL class - one reason why it should be avoided in favor of URI). URL.toURI().getPath() is most safe to fix the issue in Apache Ant
Part of the reason for this certainly is that the code has been written before the URI class even existed.
(Stefan Bodewig: Should I open an issue in Ant?).
Yes, please do. Thanks Uwe.
I opened: https://bz.apache.org/bugzilla/show_bug.cgi?id=59130
Maybe Ant developers can fix this code in later versions to handle URLs more correct.
+1
Unfortunately this is not the only issue caused by this. After I tried to build Lucene with the patch applied, the next candidate for the issue broke: Apache Ivy. It was no longer able to load the ivy-settings.xml file from its JAR file.
The reason here is another one: It constructs the JAR file URL on its own (it looks like this), but does not add the #release fragment. And because of this, JarURLConnection does not find the file...:
[...] multiple parent causes [...] Caused by: java.io.FileNotFoundException: JAR entry org/apache/ivy/core/settings/ivysett/ivysettings-public.xml not found in C:\Users\Uwe Schindler\.ant\lib\ivy-2.3.0.jar at sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.jav a:142) at sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnec tion.java:150) at org.apache.ivy.util.url.BasicURLHandler.openStream(BasicURLHandler.java:1 71) at org.apache.ivy.util.url.URLHandlerDispatcher.openStream(URLHandlerDispat cher.java:74) at org.apache.ivy.core.settings.XmlSettingsParser.doParse(XmlSettingsParser.j ava:157) at org.apache.ivy.core.settings.XmlSettingsParser.parse(XmlSettingsParser.java :183) at org.apache.ivy.core.settings.XmlSettingsParser.includeStarted(XmlSettingsP arser.java:435) at org.apache.ivy.core.settings.XmlSettingsParser.startElement(XmlSettingsPar ser.java:211) ... 35 more
So it looks like the Multi-release JAR file patch also breaks the other way round: Code constructing JAR URLs according to the standard no longer work. In my opinion, the JAR URLs should not change at all and the code should transparently choose the right release version. Maybe add a fragment only to explicitly state a specific version (so one would be able to load the Java 7 version). But this could also be done using the META-INF/... path. The default handling should be that "old" and (I think they are standardized) JAR URLs still works as they should - not requiring the fragment!
I tried another project (a private one) and it failed in similar ways while loading XSL templates. This project produced no self-crafted jar:-URLs; instead it relied on relative URL resolving (the same applies to Apache Ivy). A common pattern (especially in the "XML world") is to have relative links in your files, e.g. an XSLT file that includes another ones. If you place those XSL or XML files containing relative links in a JAR file, with previous Java versions everything worked as it should. You started the XML parser with the URL returned by the classloader and it was able to also resolve relative links between the files (because the jar: URL protocol correctly supports relative resolving of paths). So xml/xsl file containing a reference to another file in same package using a filename like <include src="./otherfile.xsl"/> works perfectly with the JAR URL protocol. If the original file had a URL like "jar:file:....!/package/master.xsl" and this was passed to XML parser [e.g, like TranformerFactory#newTransformer(new StreamSource(classloader.getResource("package/master.xsl ").toString())], the XML parser would load "jar:file:....!/package/otherfile.xsl" But because the fragment is lost during resolving relative URLs, this no longer works with Multi-Release JAR files. It looks like JARURLConnection throws FileNotFoundException without the #release fragment. I hope this helps to see why using fragments as part of the identifier is not quite correct in the URL world. I'd use some other way to refer to specific versions. At least let the no-fragment case always load the version-based file. Only use a fragment to refer to another version. Uwe
Hi Uwe, Alan, Uwe, thanks so much for testing and investigating, that is very helpful and really appreciated. The EA process is working as intended, although i wish the result was not so debilitating in this case. Sorry about that.
On 5 Mar 2016, at 15:03, Alan Bateman <Alan.Bateman@oracle.com> wrote:
On 05/03/2016 13:24, Uwe Schindler wrote:
:
I'd suggest to please ASAP revert the Multi-Release JAR file patch and provide a new preview build as soon as possible. I think there is more work needed to fix this. If this does not revert to the original state, it will be impossible to build and test Lucene, Elasticsearch,.... (and almost every Java project out there!). So short: We cannot test anymore and it is likely that we cannot support Java 9 anymore because the build system used by most Java projects behind the scenes does not bootstrap itself anymore.
Sigh, I think those of us that reviewed this missed the point that the fragment is appended by default.
Yes :-( i missed that in review <blush/> Here is a possible fix: URLClassPath.java: — /** * This class is used to maintain a search path of URLs for loading classes * and resources from both JAR files and directories. @@ -760,7 +759,11 @@ try { // add #runtime fragment to tell JarURLConnection to use // runtime versioning if the underlying jar file is multi-release - url = new URL(getBaseURL(), ParseUtil.encodePath(name, false) + "#runtime"); + if (jar.isMultiRelease()) { + url = new URL(getBaseURL(), ParseUtil.encodePath(name, false) + "#runtime"); + } else { + url = new URL(getBaseURL(), ParseUtil.encodePath(name, false)); + } if (check) { URLClassPath.check(url); } With that fix i can successfully build Lucene (i think the problem with Ivy is the same underlying cause as with Ant. We have also noticed problems with Jetty). My intention was the #runtime fragment should only be used for MR-JARs. We may need to reconsider that given the fragility of processing URLs that have been reported, although MR-JARs are new and it will take time for this to work through the eco-system allowing time to weed out the bugs. Ideally the best solution is to change the URL scheme, say “mrjar:file:/…!/…class” only for MR-JARs of course, but i considered this might be even more invasive for class scanners etc, (assuming URLs are processed correctly). However, the Jigsaw image is already adjusting the scheme for classes in an image: l.getResource("java/net/URL.class”) -> jrt:/java.base/java/net/URL.class and that will also impact other stuff folded into the image. So perhaps we should revisit? Tricky tradeoffs here.
This will of course break code that parses URL strings in naive ways (anything looking for ".xml" should be looking at the path component of course). I'll create a bug for this now, assuming you haven't created one already.
Alan created: https://bugs.openjdk.java.net/browse/JDK-8151339 Thanks, Paul.
One general point is that the purpose of EA builds and timely testing by Lucene and other projects is invaluable for shaking out issues. There will be issues periodically and much better to find these within a few days of pushing a change rather than months later.
-Alan
On 03/07/2016 03:43 AM, Paul Sandoz wrote:
Hi Uwe, Alan,
Uwe, thanks so much for testing and investigating, that is very helpful and really appreciated. The EA process is working as intended, although i wish the result was not so debilitating in this case. Sorry about that. [...] Here is a possible fix:
URLClassPath.java: —
/** * This class is used to maintain a search path of URLs for loading classes * and resources from both JAR files and directories. @@ -760,7 +759,11 @@ try { // add #runtime fragment to tell JarURLConnection to use // runtime versioning if the underlying jar file is multi-release - url = new URL(getBaseURL(), ParseUtil.encodePath(name, false) + "#runtime"); + if (jar.isMultiRelease()) { + url = new URL(getBaseURL(), ParseUtil.encodePath(name, false) + "#runtime"); + } else { + url = new URL(getBaseURL(), ParseUtil.encodePath(name, false)); + } if (check) { URLClassPath.check(url); }
With that fix i can successfully build Lucene (i think the problem with Ivy is the same underlying cause as with Ant. We have also noticed problems with Jetty).
My intention was the #runtime fragment should only be used for MR-JARs.
Does that go far enough though? I think there is a substantial amount of code which assumes (rightly) that you can build an exact path to a class in JAR URL and until today that'd work fine. It makes more sense to me that you'd only want to have to add the fragment if you want to tell it "hey I want Java 8's view of this path" or something - basically only change API when you're doing something that the API could not previously do, rather than changing JAR URLs for everyone.
We may need to reconsider that given the fragility of processing URLs that have been reported, although MR-JARs are new and it will take time for this to work through the eco-system allowing time to weed out the bugs.
Ideally the best solution is to change the URL scheme, say “mrjar:file:/…!/…class” only for MR-JARs of course, but i considered this might be even more invasive for class scanners etc, (assuming URLs are processed correctly). However, the Jigsaw image is already adjusting the scheme for classes in an image:
l.getResource("java/net/URL.class”) -> jrt:/java.base/java/net/URL.class
and that will also impact other stuff folded into the image.
Yeah but that is isolated to JDK cases. In my experience, very very few tools or containers normally construct URLs for system class path items. I think that a substantially larger pool of software is likely to try accessing JARs by URL (at least going off of a highly unscientific bit of poking around on grepcode), and I don't think that this behavior should change from an API perspective. -- - DML
On 7 Mar 2016, at 14:46, David M. Lloyd <david.lloyd@redhat.com> wrote:
My intention was the #runtime fragment should only be used for MR-JARs.
Does that go far enough though? I think there is a substantial amount of code which assumes (rightly) that you can build an exact path to a class in JAR URL and until today that'd work fine.
I would question the “rightly” part if directly parsing the characters of a URL without taking into account the encoding. I don’t know how much of this is just a short-cut or because URL being buggy has forced this approach.
It makes more sense to me that you'd only want to have to add the fragment if you want to tell it "hey I want Java 8's view of this path" or something - basically only change API when you're doing something that the API could not previously do, rather than changing JAR URLs for everyone.
So, a class loader “covering” an MR-JAR would: 1) return resource URLs as they do today; and 2) any consumer can opt in by modifying that URL to get a versioned view (scheme or fragment, preferable the former in that case) FWIW that is how the jar-based URL connection works today, you have to opt in. It’s only for MR-JAR contained resources from a class loader where that is not the case. We would need to carefully check other JDK areas, especially security/validation to see what the knock on effect it. This area is extremely fragile.
We may need to reconsider that given the fragility of processing URLs that have been reported, although MR-JARs are new and it will take time for this to work through the eco-system allowing time to weed out the bugs.
Ideally the best solution is to change the URL scheme, say “mrjar:file:/…!/…class” only for MR-JARs of course, but i considered this might be even more invasive for class scanners etc, (assuming URLs are processed correctly). However, the Jigsaw image is already adjusting the scheme for classes in an image:
l.getResource("java/net/URL.class”) -> jrt:/java.base/java/net/URL.class
and that will also impact other stuff folded into the image.
Yeah but that is isolated to JDK cases.
Not necessarily in the future, where it will be possible to fold libraries or applications into an image. FWIW certain application servers also have different URL schemes for their class loaders. Thanks, Paul.
In my experience, very very few tools or containers normally construct URLs for system class path items. I think that a substantially larger pool of software is likely to try accessing JARs by URL (at least going off of a highly unscientific bit of poking around on grepcode), and I don't think that this behavior should change from an API perspective.
-- - DML
participants (8)
-
Alan Bateman
-
André-John Mas
-
Claes Redestad
-
David Holmes
-
David M. Lloyd
-
Paul Sandoz
-
Stefan Bodewig
-
Uwe Schindler