Please stop incrementing the classfile version number when there are no format changes

[Remi Forax]

----- Mail original -----
> De: "Luke Hutchison" <luke.hutch at gmail.com>
> À: "Christian Stein" <sormuras at gmail.com>
> Cc: "jdk-dev" <jdk-dev at openjdk.java.net>
> Envoyé: Vendredi 11 Octobre 2019 22:01:36
> Objet: Re: Please stop incrementing the classfile version number when there are no format changes

Hi Luke,

> At the very least, when the classfile format is updated, old tools should
> not break when encountering classfiles containing new record types -- the
> tools should still be able to handle the subset of classfile functionality
> that they have always understood, ignoring anything new that they don't
> understand.
> 
> However, this is not possible due to a problem with the design of the
> classfile format: record lengths are not always specifically encoded in the
> format itself, so when new record types are added to the classfile format,
> old parsers can't continue to parse the classfile as soon as the first
> record of an unknown type is encountered. This is an issue with at least
> one part of the classfile format specification, the constant pool.
> 
> I ran into this problem with the ClassGraph library. When constant pool tag
> types 19 (for module) and 20 (for package) were encountered in a classfile
> generated by JDK 9+, ClassGraph simply had to stop parsing the classfile
> and throw an exception, until the library was fixed to handle these entry
> types. There was no way to correctly guess how large those new constant
> pool entries would be, based on the definition of the cp_info record type
> (which does not include a length field):
> 
> https://docs.oracle.com/javase/specs/jvms/se13/html/jvms-4.html#jvms-4.4
> 
> https://github.com/classgraph/classgraph/blob/master/src/main/java/io/github/classgraph/Classfile.java#L1084
> 
> By contrast, method and field attributes always have a size field, so even
> if you don't know how to parse a new attribute type, you can just skip over
> the number of bytes stated in the length field to move on to the next
> attribute.
> 
> https://docs.oracle.com/javase/specs/jvms/se13/html/jvms-4.html#jvms-4.7

You can only skip if you read the classfile, if read/transform/write it like by example the Maven shade plugin does,
you have to be able to interpret the content of the attribute to know if you have to rewrite it or not.
You mention the record, it you change the name of the component of a record, you have to update at least the corresponding accessor.

Now a bit of history, the first versions of ASM did not have that version check and a lot of reported issues were just people using the wrong version of ASM but it was hard to pinpoint because either you will get an exception that was deep in the code of ASM or worst the generated bytecode was rejected by the VM.
At least now, it's easy for a end-user like Mike to see the problem.

> 
> If the entire classfile format followed this design principle, then a lot
> of compatibility issues could be solved by having tools simply parse the
> subset of the format that they already knew how to handle. This would allow
> for a much stronger backwards compatibility claim to be made.
> 
> To fix this specific issue, the classfile spec should be updated to say
> that if any new constant pool attributes that are added, with a tag index
> greater than 20, they will instead use a new cp_info_with_len entry type
> that includes a length field.
> 
> The rest of the classfile format spec should similarly be carefully
> considered to make sure a historically valid subset of functionality can
> always be parsed by older tools. Then the exact classfile format number
> won't matter as much -- a tool can simply parse what it can understand.

Rémi