Q-types are dead, long live non-null side attributes !

[Remi Forax]

Hi everybody, i would like to propose a slightly different semantics for Valhalla which is more backward compatible than the current one and that follows the vision of Brian that a non null zero-default value type is equivalent to what we currently what we call the .val mirror of a primitive class.

Why do we need Q-types ?
(1) to indicate that a field can be flattened,
(2) to indicate that a method parameter can be called "by value",
(3) to indicate that an array can be flattened,
(4) to indicate the type argument of a universal generics.

What is the problem with a Q-type ?
Sadly we do not have all these benefits without paying a price.
- a Q-type encodes compile time information not runtime information, so it does not support separate compilation
  (especially if Q-type is not compatible with its equivalent L-type),
- a Q-type can not be erased. Erasure helps a lot adoptions when both use-site and declaration-site need to be updated to follow a new protocol.

I propose to encode the same information as a Q-type using a side attribute instead to avoid the problems raised by the Q-type notation.

A Q-type is use site notation equivalent to a L-type (a class name) + a non-null bit. I propose to store the non-null bit into a separate attribute associated to a field or a method declaration.
For cover the use-cases (1) and (2) but not (3) and (4).

In fact (3) and (4) are the same use-case, we need to create a specialized generics or an array specialized by a non-null type. In both case, the specialised array/generics is a runtime construction so we do not need a descriptor for it but a way to inject a runtime class of a non-null type (the secondary class as this is actually called in the prototype) as a descriptor. The parametric VM design proposed by John already propose such mechanism.

Using a side attribute (on methods and fields) to record the nullability information greatly simplify the classfile verifier, because the verifier does not need to be aware of the side attribute. A worst the VM will throw a NPE at runtime.

It makes the VM implementation slightly more complex because,
- when calling a method, the VM (the interpreter) needs to checks the side attribute and emits a NPE accordingly
- when storing a value inside a field, the VM needs to check the side attribute and if the field type is a zero-default value type at runtime and emits a NPE in that case (the VM already does the same thing for arrays).

The JIT has the same information as before,
- a field can be flattened if the type is zero-default value type at runtime and the non-null bit is set,
- an argument can be passed by value if the type is a zero-default value type at runtime and the non-null bit is set,
- a local variable of a zero-default value type at runtime + a nullcheck is scalarizable.

I've implemented a prototype (John asks for it) of this semantics on top of the LW5 prototype,
  https://github.com/forax/civilizer

It uses annotations, both at declaration site (@Value and @ZeroDefault) and use site (@NonNull and @Nullable) + bytecode rewriting because it's easier than modifying the compiler and the VM.
- The nullchecks of the parameter is done by adding calls to Objects.requireNonNull() at the beginning of the methods,
- The nullcheck of the field is done by declaring it with a Q-type + rewriting the access to the field using invokedynamic, so the access are done using L-types. 

Q-types are dead, long live non-null side attributes !

regards,
Rémi