Please stop incrementing the classfile version number when there are no format changes

[Brian Goetz]

Not true at all.  The classfile version is in a well-known place in the 
classfile format, and tools should not attempt to interpret classfiles 
for whose version they don't understand.  Your argument below is a stark 
demonstration of the moral hazard of our working hard to keep the 
classfile format stable -- users assume (with absolutely no basis) that 
it should be safe to just blindly parse a classfile whose version they 
do not recognize, and treat it as a bug that somehow this isn't 
possible.  The JVMS has been specified such that it is allowable to 
entirely redo the bytecode set in every version.  The protection against 
classfile misinterpretation is not trying to reason about classfile 
versions you don't recognize.

I understand you are thinking "well, it's possible to define the 
classfile format in a way that is more self-describing, and then it 
would be easier, so isn't it obvious that we should just do that?" But, 
you're making the classic mistake of considering only the benefit, and 
not the cost.  For example, fatter classfile structures mean longer 
startup times for 100% of Java developers.  Should we really punish the 
100% with worse startup for the sake of the .00001% of those who are 
maintaining bytecode-parsing tools, but aren't committed to keeping up?  
That seems like putting the burden in the wrong place.

ASM works hard to have a release ready for JDK N by the time JDK is 
ready to ship.  If other bytecode libraries want to play in this space, 
that's the bar, and it's a reasonable one.  It's not like changes to the 
classfile format aren't broadly discussed in public forums for a long 
time prior to making a change.  The ASM guys have figured out what good 
citizenship looks like here (thanks Remi!); other libraries should take 
a page from their book.




On 10/11/2019 4:01 PM, Luke Hutchison wrote:
> At the very least, when the classfile format is updated, old tools should
> not break when encountering classfiles containing new record types -- the
> tools should still be able to handle the subset of classfile functionality
> that they have always understood, ignoring anything new that they don't
> understand.
>
> However, this is not possible due to a problem with the design of the
> classfile format: record lengths are not always specifically encoded in the
> format itself, so when new record types are added to the classfile format,
> old parsers can't continue to parse the classfile as soon as the first
> record of an unknown type is encountered. This is an issue with at least
> one part of the classfile format specification, the constant pool.
>
> I ran into this problem with the ClassGraph library. When constant pool tag
> types 19 (for module) and 20 (for package) were encountered in a classfile
> generated by JDK 9+, ClassGraph simply had to stop parsing the classfile
> and throw an exception, until the library was fixed to handle these entry
> types. There was no way to correctly guess how large those new constant
> pool entries would be, based on the definition of the cp_info record type
> (which does not include a length field):
>
> https://docs.oracle.com/javase/specs/jvms/se13/html/jvms-4.html#jvms-4.4
>
> https://github.com/classgraph/classgraph/blob/master/src/main/java/io/github/classgraph/Classfile.java#L1084
>
> By contrast, method and field attributes always have a size field, so even
> if you don't know how to parse a new attribute type, you can just skip over
> the number of bytes stated in the length field to move on to the next
> attribute.
>
> https://docs.oracle.com/javase/specs/jvms/se13/html/jvms-4.html#jvms-4.7
>
> If the entire classfile format followed this design principle, then a lot
> of compatibility issues could be solved by having tools simply parse the
> subset of the format that they already knew how to handle. This would allow
> for a much stronger backwards compatibility claim to be made.
>
> To fix this specific issue, the classfile spec should be updated to say
> that if any new constant pool attributes that are added, with a tag index
> greater than 20, they will instead use a new cp_info_with_len entry type
> that includes a length field.
>
> The rest of the classfile format spec should similarly be carefully
> considered to make sure a historically valid subset of functionality can
> always be parsed by older tools. Then the exact classfile format number
> won't matter as much -- a tool can simply parse what it can understand.
>
>
>
> On Fri, Oct 11, 2019 at 1:32 PM Christian Stein <sormuras at gmail.com> wrote:
>
>> On Fri, Oct 11, 2019 at 3:51 PM Brian Goetz <brian.goetz at oracle.com>
>> wrote:
>>
>>> [...]
>>>> The way the class format is versioned is proving
>>>> disastrous for the community.
>>> Again, you’re acting like this is a new thing.  Given that we’ve been
>>> doing it this way for nearly 25 years, it can’t be all that disastrous.
>>>
>> An extreme thought experiment:
>> Update the class file version every 6 weeks, days, ... and see what happens
>> to "all" build tools.
>> A lot of build tools, ranging from foundation tools like `javac` to Maven
>> will just continue to work.
>> This also goes for Gradle using its Kotlin-based DSL, if I'm not mistaken.
>>
>> "If it hurts, do it more often."
>>
>> I guess, "all" build tools would soon be agnostic to the class file version
>> -- or updated their
>> delivery process to match the 6 ... hours cadence.
>>
>> Cheers,
>> Christian
>>