Valhalla EG minutes 6/21/17

[John Rose]

On Jul 19, 2017, at 7:31 AM, Daniel Heidinga <Daniel_Heidinga at ca.ibm.com> wrote:
> 
> There are a couple of different terms that have been used to describe early access features - incubator (jep 11), experiment, or optional.  At least for me, these different terms result in different mental models for how this should work for the VM.  Hopefully, we're all thinking about the same behaviour semantics and are just using different terms but I think it's worth clarifying.
>  
> 1) JEP 11 does a good job of describing incubator and makes it clear that it refers to non-standard modules / java-level code.  Not VM features.

And what we want is "like JDK incubator code but for VM features", but defined rigorously.

> 2) An optional feature is something you can chose to implement (or not) but must exist as part of the current JVM spec.  All of its JVM spec changes would need to be in the spec and their enablement / enforcement would depend on whether the VM chose to implement that optional feature.  Removal of the optional feature from the JVM spec would basically be impossible - it may fall into disuse through VMs not implementing it, but it would be hard to remove.

Yes.  As with incubator code we include a proviso that the experimental features will be dropped.

> 3) An experimental feature on the other hand, is something that would be allowed to have an appendix or set of experimental JVM spec changes that are only enabled by command line option + present in classes with new minor classfile versions.  These features would start life deprecated and expectation is that classfiles with this particular minor version would no longer be recognized by future VMs when the experimental features graduate to the real JVM features, providing freedom to experiment without requiring the VM to carry the support burden long term.

That sounds right.  We need features which customers can try out but which have a short shelf life.

Short means on the order of a single release and/or a year or two.  Shorter is better.

> My understanding has been that with the MVT prototype work, we've been aiming for the 3rd case.  Does this match everyone's expectations?  Anyone think we're aiming for some other point on the spectrum?

That's what I expect.


> One of my fears is that we're going to end up with the VM required to support multiple ways of recognizing ValueCapableClasses / ValueTypes, especially if there are spec changes between the different ways (think the mess that is invokespecial and the ACC_Super flag) based on attribute vs annotation or classfile version, etc.

Yes.

>  
> ...> - We could redesign this so that the VCC properties are checked during loading/verification of Foo. I am concerned that, where Foo is version 54.0 and has attribute ValueCapableClass, this sort of error checking will violate the Java SE 10 spec.
>  
> Doesn't the use of experimental features allow the new classfile versions to define new behaviour?  I would expect the ValueCapableClass attribute to be ignored in a v.54 classfile and only take affect in a v.54.1 classfile so that the semantics can be changed in the future.

The VCC is a special case, because it must be compiled from standard source with no changes to javac.
Its class file doesn't directly use the new bytecodes (vload, etc.) or modifiers (ACC_VALUE).
This feature must be triggered by an annotation, not an attribute, nor a classfile version bit.

For the actual bytecodes (vload, etc.) a classfile version bit or attribute is the sort of thing
we need to enable the experimental feature.

> > 
> > Elaborating: we're presenting values as an optional feature of Java SE 10. For a JVM that does not implement the optional feature, JVMS 10 says that a ValueCapableClass attribute on a version 54.0 class file will be ignored. JVMS 4.7.1: "any attribute not defined as part of the class file specification must not affect the semantics of the class file. Java Virtual Machine implementations are required to silently ignore attributes they do not recognize." My interpretation of our mission, in adding an optional feature, is to provide new capabilities while having no impact on existing behavior. We can do new things where JVMS 10 specifies an error; we can't generate new errors where JVMS 10 specifies none.
>  
> New classfile version == new behaviour, right?

(Or new class file version + attribute.  Or new class file version + class file minor-version bit.  But yes.)

> 
> > 
> > - We could limit usage/interpretation of "ValueCapableClass" to 54.1 class files. Then eager error checks when loading Foo would be fine. But the ability to work with 54.0 ValueCapableClass class files is an important use case.
>  
> The model here is that using a JVM 10 javac, a classfile with the annotation is v54 and on a VM with some -XX:EnableExperimentalFeatures flag, this derives a v.54.1 class for the DVT?  This basically gives new behavior to v.54 classfiles which makes it very hard to change when Java 11 comes along and fully specifies ValueTypes.

The annotation (not attribute or version bit) for a VCC (remember, it's a special case)
can be made contingent on 54.  That's fulfills the contract for an experimental VM feature.
The annotation will be treated as a no-op (like most other annotations) in 53 or 55.

And, yes, the new class file format features for the DVT (and other experimental VTs)
would be enabled only in 54.1 or (54 + SomeAttribute).

> Customers will still have these annotated v.54 classes and expect them to keep working as values, meaning the JVMs will need to support both the annotation and the new way.

They might expect this but it will fail in any version besides 54,
per the contract of the experimental feature.

> Limiting the new behaviour to v.54.1 classfiles and then making them illegal in Java 11 makes it obvious to the user that the experimental classfiles won't work the same way and likely need to be refactored to the new "real" ValueTypes way.
>  
> Would the annotation be in an incubator module so that it can be deprecated / removed easily in a future release?

Most definitely.  So the incubator module and the JVM hook would go away at the same time.

> 
> > 
> > - Actually, examining that use case more closely, we discussed how attributes are not the right tool at all. We want clients of MVT to be able to generate interesting programs using a vanilla Java SE 10 compiler. The best mechanism to communicate "I'm a value-capable class" using a vanilla Java compiler is annotations, not attributes.
>  
> Because we're dealing with experimental features, can we require a similar -XX:EnableExperimentalFeatures flag (or -D option) for javac to convert the annotation into a v.54.1 classfile with the attribute?  This makes it easy to write the code but keeps it sandboxed to the experimental classfile version.

We will likely have such a compiler, and it will probably generate VT's natively.
The VCC use case is designed for a "vanilla" tool chain.  Adding extra "are you sure?" flags is not helpful IMO.

> > 
> > Tentatively, the spec should be revised so that:
> > - The "ValueCapableClass" attribute no longer exists. Delete changes to 4.7 and delete new section 4.7.25.
> > - A "value capable class" is a class whose RuntimeVisibleAnnotations includes a ValueCapableClass annotation (package TBD; whether the referenced class must be loaded TBD)
> > - As usual, this annotation doesn't impact behavior of class Foo
> > - When loading Foo$Value, as specified in 5.3, we'll check that Foo is a value capable class and that it has the right properties

Yep.  You could mentally model this as not a JVM feature, but some sort of instrumentation-like feature
which derives the DVT by looking at the "vanilla" VCC class.  The VM features kick in which the DVT
(or a stand-alone experimental VT) is loaded with version 54.1.  (Or 54.1 plus an attribute.  But I prefer
just the version number bit.)

> > 
> > —Dan
>  
> Annotations are typically more expensive to read in the VM than attributes.  If not for the vanilla-java concerns, a feature like this would have an attribute in the JVMS.  Shouldn't we try to start with the attribute to keep us close to the path we'd pick for a future java release?

No, VCC has a different design center which mustn't be confused with the VT design center.

— John