Please stop incrementing the classfile version number when there are no format changes

[Andrew Dinn]

I have been a casual bystander up to now. However, I'd like to add my
two cents worth of underline to Brian's advice. Indeed, I'll adapt
Mike's phrasing and characterise it as "an accurate and especially
charitable view of *how* people are arguing" and recommend that you take
it very seriously.

I do actually have a dog in this fight. I have an agent that I need to
update every time a new JDK is released -- now a 6-monthly event. I have
rarely made changes to my code in response to a bytecode version update
beyond tweaking the ASM version in my POM. However, I'm not 100% clear
that I have got that right even in my own case (e.g. does my agent now
deal with nestmate access correctly?) let alone whether a null response
would/could ever make sense for all consumers of bytecode. I certainly
don't find it anything of a stretch to accept Brian's arguments that all
changes to the bytecode format and/or associated semantic need to be
clearly signalled with a new version tag because some consumers are
bound to need to respond to that change in format/semantic.

However, the merits of the case are not really the issue that moved me
to post. I am writing to emphasize Brian's latest point which concerns
/protocol/ and /etiquette/. Brian has offered a very simple path for
negotiating a change here. Discuss, identify and clarify agreed
problems, work towards an agreed solution, implement test and revise it
until it is acceptable to all affected parties. That's what we try do
here in the OpenJDK project: work towards consensus and, where required,
compromise. Coming here on the premise that your needs are not being met
and only this solution will answer them without first checking for and
then considering other peoples' needs is not going to fly.

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill

On 14/10/2019 21:17, Brian Goetz wrote:
> At this risk of letting this thread go on any longer (really, so far
> nothing has been said that we already didn’t know, and a lot of time
> has been spent explaining it, which is time that comes out of our
> budget for everything else), let me give you some advice about how to
> effect change.
> 
> The mistake Luke made (and then repeatedly doubled down on several
> times) was that he started with the notion that he knew what the
> problem was (without building consensus around that), and that he
> knew what the solution was (without building consensus around that),
> and then lept from there to push on a specific solution.  Before he’d
> even made his argument, his subject line gave it away: “Please stop
> <doing thing I obviously think is terrible.>”.  Then, when  it was
> pointed out that the problem was far subtler and more complex than
> he’d given it credit for, _he continued to argue for the same
> non-solution, rather than trying to backtrack to the underlying
> problem._
> 
> If you want to effect change, you have to start with _building shared
> consensus that there is a problem_.  Then, you can engage in a shared
> search and evaluation over solutions (our first idea is rarely the
> best one, and even if it was the best one, if there isn’t a shared
> understanding of the problem and why it is the best solution, it is
> worthless.)
> 
> The decision on incrementing the class file version was one that was
> made a long time ago, and reaffirmed several times since, for a
> number of reasons, some of which have been stated here.  And there’s
> been no new evidence that something was missed.  So we’re not
> inclined to reconsider this policy, and I think ample evidence has
> been given why.  So, stop pushing this non-solution; these aren’t the
> droids you are looking for.
> 
> But, that doesn’t mean the problem can’t be solved; it just means
> this isn’t the solution.  So, start over, and see if you can come
> with a clear description of the actual problem (not just a solution
> written in problem-speak), and build some consensus.  Only then can
> we consider solutions.
> 
> Yes, this is hard.  It’s much easier to drive by and say “Please stop
> doing X, kthxbye”.  But nothing that affects compatibility is that
> simple.
> 
> 
>> On Oct 14, 2019, at 3:55 PM, Luke Hutchison <luke.hutch at gmail.com>
>> wrote:
>> 
>> On Mon, Oct 14, 2019 at 1:26 PM Mike Hearn <mike at plan99.net
>> <mailto:mike at plan99.net>> wrote: software reliably stop working on
>> schedule 'just in case'. ClassGraph is commonly used just to locate
>> classes that have an annotation on them after all, and as far as a
>> I know all planned format changes would still allow this task to be
>> done. Yes, people might annotate new class-like-things such as
>> values and confuse old ClassGraphs; that's an ordinary sort of bug
>> and
>> 
>> Yes, this is the crux of it. Brian's claims are all correct and
>> unarguable from a purist point of view; however, most people cannot
>> afford to operate in the realm of purism because of the realities
>> of the JVM ecosystem.
>> 
>> 99% of classfile parsing is for only one of the following two
>> usecases:
>> 
>> (1) Locating classes, fields, or methods, that have a specific
>> annotation. (2) Locating subclasses of a given class, or classes
>> that implement a given interface.
>> 
>> The JDK *does not help* people with either of these needs,
>> therefore libraries like ClassGraph have been created to provide
>> these functions. ClassGraph does a lot more than this, but few
>> users do anything more advanced than this. (The main reason
>> ClassGraph is often picked these days over some other library is
>> because it robustly supports classpath and module path scanning
>> across the extreme mess of different classpath / module path
>> specification mechanisms, something that the JVM also does not help
>> programmers come to grips with:
>> https://github.com/classgraph/classgraph/wiki/Classpath-Specification-Mechanisms
>> <https://github.com/classgraph/classgraph/wiki/Classpath-Specification-Mechanisms>
>> )
>> 
>> And here is the critical point: Programmers who need these
>> capabilities *simply need these functionalities to keep working,
>> period* -- they don't care what other exotic features the JVM might
>> choose to add ("nest mates"? How did that term even get
>> rubberstamped?) -- they just need to find their classes, fields or
>> methods using annotations, superclasses or interfaces, *and they
>> need this to never break*. They don't plan to change their whole
>> codebase to make use of entirely new paradigms that would break the
>> way the old code works.
>> 
>> This is where the numbers Brian gave are so out of touch with
>> reality:
>> 
>> And have also been three versions (5, 7, and 11) that added new
>> _constant pool forms_.  That’s almost 25%!  Not rare at all.  (If
>> your website failed on 25% of requests, you’d not get away with
>> calling that “rare failures”, you’d be out of business.)
>> 
>> We're coming up on 24 years since Java's initial 1.0 release. So
>> three breakages in 24 years that would break any scanner (like
>> ClassGraph) that ignores the classfile version number averages out
>> to one breakage (due to constant pool changes) every 8 years. (I
>> previously asserted this might happen once every 10 years.) But 24
>> years from now, assuming the 6 month cadence continues, any library
>> that simply throws an exception when it encounters a new classfile
>> version number (as Brian has suggested is the only reasonable thing
>> to do) leads to one breakage every 6 months, or (24 * 2 + 1) = 45
>> breakages over the same period. What Brian has proposed would lead
>> to breakage 45 / 8 = 5.625x as often on average. So you'd be "out
>> of business" 5.625x faster if you followed this suggestion and
>> simply balked at new version numbers.
>> 
>> I stand by my assertion that it is completely reasonable, even
>> desirable, for a library to parse only the subset of a classfile
>> that it understands, and ignore everything else, with the
>> understanding that when things do break, you get to keep the pieces
>> (and the library maintainers must take responsibility to fix their
>> library and push out a new version when that does happen). Even the
>> method and field attributes part of the classfile spec is designed
>> with exactly this in mind: you just skip attributes you're not
>> interested in. I think it is very shortsighted to outright reject
>> making a small update to the classfile format to enable this sort
>> of selective parsing for the entire classfile. But I have said all
>> I think I can say on the issue.
>> 
>> Mike's point about only bumping the version number for semantic
>> changes is orthogonal to my request for "subset-parseability", but
>> I also think this is an entirely reasonable request to make, and a
>> sensible suggestion, in order to reduce the frequency of breakage
>> for libraries like ASM that must parse 100% of a classfile in a
>> semantically correct way, so must throw an exception for unknown
>> classfile formats.
>> 
>