Invoke MR-JAR file verifier on command line without actually packaging

[Paul Sandoz]

Hi Uwe,

Unfortunately i don’t recall there being a publicly supported way to validate an existing Mr.Jar (except by recreating it), either by the jar tool itself nor by a programatic API.

In a more general sense the problem is one of given two jars validate that the class file bytes in each expose a compatible public API, with a valid Mr.Jar being more restrictive that no new public artifacts should be introduced. I don’t know of any tooling outside of the JDK that might do such checks.


> On 27 Sep 2017, at 03:31, Uwe Schindler <uschindler at apache.org> wrote:
> 
> Hi,
> 
> I am building a multi-release JAR file using Apache Ant (for Lucene, see https://issues.apache.org/jira/browse/LUCENE-7966).
> 
> As build tools like Ant/Gradle/Maven are packaging JAR files on their own, not using the official JAR tool, I wanted to do a quick check if the resulting JAR file is valid. https://bugs.openjdk.java.net/browse/JDK-8158295 added some checks to compare method signatures and class file between the base and the META-INF/versions occurrences.
> 
> Is there a way to invoke those checks on an already packaged JAR file?
> 
> Uwe
> 
> P.S.: Just a bit of background for those who are interested! This is a totally new way to create a MR JAR by just pacthing the already compiled class files before packaging as JAR file. The use case here is:
> - We want to use java.util.Arrays / java.util.Objects static methods throughout Lucene, especially to improve bounds checks by having Java 9 use the intrinsics when accessing arrays/ByteBuffers. We have already seen a huge speed improvement for our implementation of the LZ4 compression!

Good know!


> - We need to be compatible with Java 8, but improve Java 9.
> - So we compile all our Lucene classes against a "custom" variant of the interesting methods, located in a fallback classes we have implemented in our own org.apache.lucene.future.FutureArrays/FutureObjects.
> - We do not want to duplicate our code by creating clonesof all .java files that use these methods and we still want to still compile against Java 8 - as this is the release JDK version! We want it automatic!
> 
> So our trick is new and also way cool: During packaging the JAR file we patch all class files using ASM's ClassRemapper and replace the Type org.apache.lucene.future.FutureArrays by java.util.Arrays, only for those that actually contain a reference to our own replacement methods. This is a 74-liner in Groovy! After patching we also place the patched files in the MR part of JAR file.
> 
> Patch can be seen here:
> https://github.com/apache/lucene-solr/compare/master...uschindler:jira/LUCENE-7966-v2
> 
> Especially this is how we patch our class files, way cool:
> https://github.com/uschindler/lucene-solr/blob/d67eb6274d4c0b8315848db3f09809fbc0797f6e/lucene/core/src/tools/groovy/patch-mrjar-classes.groovy
> 

Very neat.

Paul.