Multi-Release JAR file patch as applied to build 108 of Java 9 breaks almost every project out there (Apache Ant, Gradle, partly Apache Maven)

Sat Mar 5 13:50:12 UTC 2016

Hi,

similar issues were discovered too late to stop b108, e.g., https://bugs.openjdk.java.net/browse/JDK-8150920. Fix is already in jdk9/dev, so I think the next build should be more well-behaved and hope we can provide it more promptly than normal.

If you can build OpenJDK from jdk9/dev and report any remaining issues due to the multi-release feature that would be quite helpful!

Thanks!

/Claes

Uwe Schindler <uschindler at apache.org> skrev: (5 mars 2016 14:24:37 CET)
>Hi OpenJDK Core Developers,
>
>you may know the Apache Lucene team is testing early access releases of
>Java 9. We reported many bugs already, but most of them only applied to
>Hotspot and Lucene itsself. But this problem since build 108 is now
>really severe, because it breaks the build system already!
>
>To allow further testing of Open Source Projects, I'd suggest to revert
>the Multi-Release-JAR runtime support patch and provide a new preview
>build ASAP, because we found out after a night of debugging a build
>system from which we don't know all internals what is causing the
>problems and there is no workaround. I am very sorry that I have to say
>this, but it unfortunately build 108 breaks *ALL* versions of Apache
>Ant, the grandfather of all Java build systems :-) I know also OpenJDK
>is using it, too! So with Multi-Release JAR file patch applied (see
>http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c), any
>Ant-based build - including the JDK build itsself - would no longer
>bootstrap. It is impossible to also build Gradle projects, because
>Gradle uses Ant internally for many tasks). Maven projects may be
>affected, too.
>
>Now you might have the question: What happened?
>
>We tried to build Lucene on our Jenkins server, but the build itsself
>failed with a stupid error message:
>
>BUILD FAILED
>/home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:21: The
>following error occurred while executing this line:
>/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:56:
>not doesn't support the nested "matches" element.
>
>The first idea was: Ah, there were changes in XML parsing
>(JDK-8149915). So we debugged the build. But it was quite clear that
>XML parsing was not the issue. It got quite clear when we enabled
>"-debug" on the build. What happened was that Ant was not loading its
>internal conditions/tasks/type definitions anymore, so the build system
>does not know almost any type anymore. The debug log showed that Ant
>was no longer able to load the resource
>"/org/apache/tools/ant/antlib.xml" from its own JAR file anymore.
>Instead it printed some strange debugging output (which looked totally
>broken).
>
>I spend the whole night digging through their code and found the issue:
>The commit of Multi-Release-Jar files (see
>http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c) broke
>resource handling in Apache Ant. In short: If you call
>ClassLoader.getResources() / or getResource() you get back an URL from
>where you can load the Resource - this is all fine and still works.
>But, with the Multi-Release JAR files patch this now has an URL
>fragment appended to the URL: '#release' (see
>http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c); this also
>applies to non-multi-release JAR files like Apache Ant's "ant.jar".
>
>In Java 7, Java 8,... and Java 9pre-b108,
>ClassLoader.getResource()/getResources() returned stuff like: 
>
>"jar:file:/C:/Program%20Files/Java/apache-ant-1.9.6/lib/ant.jar!/org/apache/tools/ant/antlib.xml"
>
>Now in Java 9b108 the following is returned:
>
>"jar:file:/C:/Program%20Files/Java/apache-ant-1.9.6/lib/ant.jar!/org/apache/tools/ant/antlib.xml#release"
>
>And here Ant breaks (and I assume many other projects like Maven, too).
>Ant checks for the file extension of the string (because it may load
>definitions from both XML and properties files). So it does
>endsWith(".xml") and of course this now returns false. The effect is
>that Ant tries to load its own task definitions as a java properties
>file instead of XML. Of course this fails, because the data behind this
>URL is XML. The effect is that Ant cannot bootstrap as everything to
>build is missing.
>
>One might say: Ant's code is broken (I agree, it is not nice because it
>relies on the string representation of the resource URL - which is a
>no-go anyways), but it is impossible to fix, because Ant is bundled on
>most developer computers and those will suddenly break with Java 9!
>There is also no version out there that works around this, so we cannot
>test anything anymore!
>
>The problematic line in Ant's code is here:
>http://grepcode.com/file/repo1.maven.org/maven2/org.apache.ant/ant/1.9.6/org/apache/tools/ant/taskdefs/Definer.java?av=f#259
>
>I'd suggest to please ASAP revert the Multi-Release JAR file patch and
>provide a new preview build as soon as possible. I think there is more
>work needed to fix this. If this does not revert to the original state,
>it will be impossible to build and test Lucene, Elasticsearch,.... (and
>almost every Java project out there!). So short: We cannot test anymore
>and it is likely that we cannot support Java 9 anymore because the
>build system used by most Java projects behind the scenes does not
>bootstrap itself anymore.
>
>My suggestion would be to investigate other versions for this patch
>that does *not* modify the resource URLs by appending a fragment to
>them (at least not for the "standard" case without an actual
>Multi-Release Jar). For new multi-release JAR files I am fine with
>appending fragments, but please not for default ones. Maybe change code
>to handle the URLs from the non-versioned part differently (without
>fragment). Leaving the fragment inide may break many othe rprojects,
>because many programmers are very sloppy with handling URLs (well-known
>issue is calling URL#getFile() of a file:-URL that breaks on Windows
>systems and spaces in path name). Many people just call toString() on
>URL and do some test on it (startsWith, endsWith). So appending
>fragments is a no-go for backwards compatibility with JAR resources!
>
>I posted this to the mailing list and did not open a bug report on
>http://bugs.java.com/, because this is a more general issue - feel free
>to open bug reports around this!!! I would be very happy if we could
>find a quick solution for this problem. Until there is a solution we
>have to stop testing Java 9 with Apache Lucene/Solr/..., and this is
>not a good sign, especially as Jigsaw will be merged soon.
>
>Thanks for listening,
>Uwe
>
>P.S.: I also CCed the Apache Ant team. They should fix the broken code
>anyways, but this won't help for many projects already out there (e.g.
>Apache Lucene still has a minimum requirement of Ant 1.8.2 because
>MacOSX computers ship with that version since years).
>
>-----
>Uwe Schindler
>uschindler at apache.org 
>ASF Member, Apache Lucene PMC / Committer
>Bremen, Germany
>http://lucene.apache.org/

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.