JEP draft: Better-defined JVM class file validation

[Dan Heidinga]

As discussed during the EG meeting, I've pulled together some examples
[1] that change the classfile version number during retransformation
without changing anything else in the file.  The results were a little
surprising to me in how different OpenJ9 and Hotspot behave in these
cases as I recall this being a common "solution" used by bytecode
agents in the Java 7 time frame to allow their classes to load despite
faulty StackMapTables by falling back to the flow verifier.

There are two examples:

* one that back-dates a Java 8 classfile to Java 6 or 7 and this fails
with a ClassFormatError on both OpenJ9 and Hotspot.

* one that back-dates a Java 16 classfile that uses NestMates to Java
8 and here the behaviour between OpenJ9 and Hotspot diverge.  Hotspot
appears to parse the attributes only if they apply to a given
classfile version and thus the redefinition fails as it sees the Nest*
attributes as being removed after the version change.  OpenJ9 on the
other hand continues to parse the attribute and allows the
redefinition to succeed.

The interesting thing - at least to me - is that both behaviours
appear to be valid in the current spec.  There's an interesting bit of
unspecified behaviour here which treats classfile attributes as both a
property of the VM version being run and of the classfile version
being parsed.  The spec isn't clear on whether the VM must take both
into account when determining if a class is valid or not.  This hole
appears to have shrunk a bit over time as the spec has spent more ink
on when attributes were added but would only close with the proposed
update.

--Dan

[1] https://github.com/DanHeidinga/version_change

On Fri, Jun 4, 2021 at 12:42 PM Dan Smith <daniel.smith at oracle.com> wrote:
>
> Posted a new JEP draft, here:
>
> http://openjdk.java.net/jeps/8267650
>
> This is some preparatory work to allow us to more cleanly introduce the variety of new JVM features coming with primitive objects (JEP 401).
>
> Most of the spec work is already done; you can browse the proposed spec changes here:
> http://cr.openjdk.java.net/~dlsmith/8267650/8267650-20210603/specs/
>