Please stop incrementing the classfile version number when there are no format changes

[Luke Hutchison]

On Sat, Oct 12, 2019 at 6:09 AM Mike Hearn <mike at plan99.net> wrote:

> Take this example. When Java 10 came out I wanted to try JPMS with Kotlin
> in a hobby project. But that didn't work: Kotlin's stdlib wasn't
> modularised. I investigated why not and it turned out JetBrains had tried
> but hit a critical problem - Android had a tool that barfed when trying to
> parse module-info.class, because of course that file isn't a class, it's a
> module. So modularising anything used by Android devs was impossible.
> Google eventually fixed it but it was a low priority issue, Android devs
> have upgrade latency, and nobody wants to release a modular JAR until most
> Android devs have upgraded.
>
> In turn that caused JetBrains to punt modularisation of the Kotlin
> ecosystem until Kotlin 1.4, their next major release, so that added even
> more latency, and because modules can only depend on other modules that
> means the entire Kotlin ecosystem can't easily adopt JPMS and thus jlink,
> ensuring there was mostly no reason to upgrade to Java 9+. Kotlin 1.4 still
> hasn't happened nearly two years later:
>

Yes, this is an important point, since module-info.class was the most
common compatibility issue that came up with the constant pool changes in
JDK 9. I'll give more context for how it affected ClassGraph, since it is
illustrative:

ClassGraph builds on JDK 8 in JDK 7 compatibility mode, which is a nice way
to support JDK 7 while still allowing the use of @FunctionalInterface
(defined in JDK 8). However, ClassGraph also wanted to be forwards
compatible with the module system, so I added module-info.class, which was
the only class built for JDK 9. Immediately after release, I started
getting bug reports that all sorts of build and runtime environments were
breaking for people, because projects were using multiple classfile
scanners somewhere in the classpath, often unknown to the user, due to the
transitive dependency problem.

I saw a clever suggestion that the safest place to put module-info.class
was in META-INF/versions/9 , since classfiles should not be contained in
META-INF in JDK 8 or earlier. So I moved module-info.class there. This
resolved about half the problems, as far as I could tell. However, there
were still classpath scanners (including the Android one that Mike referred
to) that scanned the entire classpath recursively for classes, and would
throw an exception and bomb on the new module descriptor no matter where it
was.

There were two classes of problems caused by module-info.class: (1)
classpath scanners that would throw an exception based on the classfile
format number alone (the "once every 6 months problem"); and (2) classpath
scanners that would throw an exception due to assuming the module
descriptor was a traditional class, trying to parse it, and failing (the
"once every ten years or so problem").

In both cases, the robustness principle that I linked earlier in this
thread would have saved the day for most users:
https://en.wikipedia.org/wiki/Robustness_principle  In this context, this
translates to: (1) the classfile format and the compiler should support
parsing of a valid subset of the classfile through a self-descriptive
format, to allow graceful failover when new features are introduced ("be
conservative in what you send"); and (2) the scanners involved should
possibly have ignored "corrupt" classfiles, rather than dying with an
exception ("be liberal in what you accept").

More than 2 years after the release of JDK 9, I am still getting bug
reports where simply including ClassGraph on the classpath on an old
version of an app server (JBoss or Spring) causes the application to crash
with an exception (because another scanner in the runtime platform finds
ClassGraph's module-info.class). People blame ClassGraph for this, since it
is supposed to be (and is) compatible back to JDK 7. They are usually not
able to upgrade their runtime environment to fix the issue. (This is not
just limited to ClassGraph of course -- even newer log4j releases cause
this issue, so yes, it will get fixed in time as more people hit this:
https://issues.jboss.org/browse/JBEAP-15716?_sscc=t ).

I am considering switching to automatic module naming to solve this for
once and for all, but that is just sidestepping the problem -- and as I
have pointed out, the faster JDK release cadence will cause these sorts of
problems much more often in future if scanners just bomb on classfile
versions they don't recognize, even if they could actually handle the
classfile just fine.