Please stop incrementing the classfile version number when there are no format changes

[Luke Hutchison]

At the very least, when the classfile format is updated, old tools should
not break when encountering classfiles containing new record types -- the
tools should still be able to handle the subset of classfile functionality
that they have always understood, ignoring anything new that they don't
understand.

However, this is not possible due to a problem with the design of the
classfile format: record lengths are not always specifically encoded in the
format itself, so when new record types are added to the classfile format,
old parsers can't continue to parse the classfile as soon as the first
record of an unknown type is encountered. This is an issue with at least
one part of the classfile format specification, the constant pool.

I ran into this problem with the ClassGraph library. When constant pool tag
types 19 (for module) and 20 (for package) were encountered in a classfile
generated by JDK 9+, ClassGraph simply had to stop parsing the classfile
and throw an exception, until the library was fixed to handle these entry
types. There was no way to correctly guess how large those new constant
pool entries would be, based on the definition of the cp_info record type
(which does not include a length field):

https://docs.oracle.com/javase/specs/jvms/se13/html/jvms-4.html#jvms-4.4

https://github.com/classgraph/classgraph/blob/master/src/main/java/io/github/classgraph/Classfile.java#L1084

By contrast, method and field attributes always have a size field, so even
if you don't know how to parse a new attribute type, you can just skip over
the number of bytes stated in the length field to move on to the next
attribute.

https://docs.oracle.com/javase/specs/jvms/se13/html/jvms-4.html#jvms-4.7

If the entire classfile format followed this design principle, then a lot
of compatibility issues could be solved by having tools simply parse the
subset of the format that they already knew how to handle. This would allow
for a much stronger backwards compatibility claim to be made.

To fix this specific issue, the classfile spec should be updated to say
that if any new constant pool attributes that are added, with a tag index
greater than 20, they will instead use a new cp_info_with_len entry type
that includes a length field.

The rest of the classfile format spec should similarly be carefully
considered to make sure a historically valid subset of functionality can
always be parsed by older tools. Then the exact classfile format number
won't matter as much -- a tool can simply parse what it can understand.



On Fri, Oct 11, 2019 at 1:32 PM Christian Stein <sormuras at gmail.com> wrote:

> On Fri, Oct 11, 2019 at 3:51 PM Brian Goetz <brian.goetz at oracle.com>
> wrote:
>
> > [...]
> > > The way the class format is versioned is proving
> > > disastrous for the community.
> >
> > Again, you’re acting like this is a new thing.  Given that we’ve been
> > doing it this way for nearly 25 years, it can’t be all that disastrous.
> >
>
> An extreme thought experiment:
> Update the class file version every 6 weeks, days, ... and see what happens
> to "all" build tools.
> A lot of build tools, ranging from foundation tools like `javac` to Maven
> will just continue to work.
> This also goes for Gradle using its Kotlin-based DSL, if I'm not mistaken.
>
> "If it hurts, do it more often."
>
> I guess, "all" build tools would soon be agnostic to the class file version
> -- or updated their
> delivery process to match the 6 ... hours cadence.
>
> Cheers,
> Christian
>