POC: JDK ClassModel -> ASM ClassReader

[ebruneton at free.fr]

Hi Brian,

Le 20/07/2022 15:50, Brian Goetz a écrit :
>> On Jul 20, 2022, at 2:08 AM, ebruneton at free.fr wrote:
>> About the "majority of cases" where the class file format does not
>> change between two JDK versions, why is the class file version still
>> incremented? If it was not, this would solve several cases where
>> tools such as ASM must be "updated" for no real reason. Another
>> option would be to increment the minor version only, and to reserve
>> major version increments for real format changes. If this was added
>> to the JVMS, tools such as ASM could take advantage of this, and
>> would need to be updated less often (in fact I realize that ASM is
>> already checking the major version only, so we unconsciously already
>> assumed that minor version changes are not important).
> 
>  This question comes up over and over again.  There was a long and
> drawn out thread about this a few years ago (rooted here:
> https://mail.openjdk.org/pipermail/jdk-dev/2019-October/003388.html
> [1]) as well as several times before that, but of the many times this
> has come up, no one has presented a compelling argument to change the
> policy.  I’ll summarize the conclusions briefly, but would rather
> not reopen the “debate” unless there is dramatic new evidence.
> 
> The bottom line is that it is a myth that the “majority of
> versions” do not have classfile changes.  There are actually two
> myths here:
> 
>  - The only changes that matter are “format changes”
>  - Format changes are rare.
> 
> Sure, we only introduced one new bytecode (7).  But new byte codes are
> not the only kind of format changes.  There have also been three
> versions (5, 7, and 11) that added new _constant pool forms_.
> (Keeping score: that’s three versions out of 19 versions with format
> changes significant enough to make classfile parsers fail,
> approximately 1 out of 6.  1 out of 6 is not “rare”.)
> 
> But so-called “format changes” merely scratch the surface of the
> sorts of dependencies inherent in translation from source to class
> file, for which versioning is warranted.  Additional forms of
> dependencies include:
> 
>  - New class file attributes.  The JVMS defines a number of attributes
> that carry important semantic information, including those that
> control access control decisions, such as NestMembers,
> PermittedSubtypes, Module (and friends), etc.  Yes, the class file
> format was designed so that unrecognized attributes could be
> “skipped” without losing your place in parsing — but that
> doesn’t mean that it is reasonable for a class file processor who
> claims to understand that class file version to ignore them!  New
> class file attributes are also quite common, with additions in 5, 7,
> 8, 9, 11, 16, and 17.  Not rare, and you should not be parsing (let
> alone instrumenting) Java 19 classifies if you don’t understand the
> semantics of the attributes defined for Java 19 classfiles.
> 
>  - Dependencies between translation strategy and JDK libraries.  It is
> not uncommon that language features require library support; the
> foreach loop depends on Iterable (5), try-with-resources depends on
> Autocloseable (7), lambdas depend on the LambdaMetafactory bootstrap
> (8), indified string concatenation depends on the appropriate
> bootstraps (9), and similar for many new language features such as
> records (java.lang.Record) and pattern matching.  Not bumping the
> class file version here would be setting a trap for users, because
> while they might look like they conform to an older class file spec,
> they won’t run on older JDKs.
> 
>  - Change in classfile validity.  Small changes about valid
> combinations of classfile elements are made all the time.  For
> example, prior to Java 9, it was an error for a method in an interface
> to have ACC_PRIVATE;  now that is valid.  (Soon ACC_SUPER will be
> deprecated and later the bit value repurposed.). While the
> “format” hasn’t change, the semantics has. This is also not so
> infrequent.
> 
> As you can see, both the assumptions — that “format changes”
> should be defined by the most narrow interpretation of “can I parse
> this without losing my place”, and the assumption that changes that
> require us to bump the class file version are rare — appeal to
> wishful thinking.  Even if we didn’t bump the class file version
> “unless we had to”, such bumps would happen in probably 70-80% of
> releases anyway.
> 
> (As a secondary matter, because features are late bound to releases, a
> policy of only bumping the version “when there are significant
> changes” creates a process tax because we have to wait to the end to
> see if a bump is warranted, and then do the appropriate work at the
> last minute, which introduces more friction and potential for error.
> While it may seem that that is “our problem”, such things
> eventually become everybody’s problem.  If it were truly rare (once
> a decade), we might consider this, but that’s not the case.)

Thanks for all the arguments! I was counting new constant pool items and 
new attributes in the "new format" category, as well as new semantics 
(such as the removal of jsr/ret), but not the library support. But I was 
only considering the "recent" versions, since the 6-months release 
cycle. Also I was not proposing to not bump the version at all, but to 
use the minor / major version distinction provided by the class file 
format. Your last point seems to preclude this as well...

Eric

> 
> 
> Links:
> ------
> [1] https://mail.openjdk.org/pipermail/jdk-dev/2019-October/003388.html