JVMS draft for L-world value types with support for nullability

[Paul Sandoz]

Hi Fred, John,

Thanks for the explanations, very informative.

Regarding enums i was not thinking of the case of implicitly retrofitting all enum classes to be value classes (that would be tricky as you point out). I was wondering if it might be possible for an enum class to also explicitly be a value class, e.g.:

__Value enum MyEnumValue { 
V1, V2, V3; 
}

(placing aside compatibility issues of an explicit transition, and even though it inherits from abstract class Enum), since enum is anyway special cased in some ways, but perhaps not sufficiently to make it easy to do.

Thanks,
Paul.
 

> On Feb 1, 2018, at 6:34 AM, John Rose <john.r.rose at oracle.com> wrote:
> 
> On Jan 31, 2018, at 9:38 PM, Frederic Parain <frederic.parain at oracle.com> wrote:
>> 
>> Hi Paul,
>> 
>>> On Jan 31, 2018, at 15:50, Paul Sandoz <paul.sandoz at oracle.com> wrote:
>>> 
>>> Hi Fred,
>>> 
>>> Some basic questions, which might be because the specification is transitioning from value-based to value classes.
>>> 
>>> 
>>>> The ACC_ENUM flag indicates that this class or its superclass is declared as an enumerated type. A class file must not have both ACC_ENUM and ACC_VALUE_TYPE flags set.
>>> 
>>> 
>>> Why can’t enum classes be values classes, where enum values are actual values? I think the answer may be below.
>> 
>> First of all, there’s a backward compatibility issue with old enums. They have been defined with full identity,
>> which is incompatible with being a value type.
>> There’s also an issue with the super-type. The super-type of all enums is the abstract class java.lang.Enum,
>> but the super-type of a value class must be java.lang.Object.
>> One advantage of values types is that the JVM can flattened them, it is possible because of two properties:
>> being identity-less and having a default value. Without a default value, the JVM cannot initialized a non-nullable
>> field, so flattened cannot be performed,
> 
> Good summary.  Although it is desirable to evolve enums to value types,
> we would have to figure out a migration strategy that would allow us to
> recompile them into value types, but also allow old code to operate
> correctly on them.  I don't think this is an easy problem, in L-world or
> any other world.
> 
> A JVM could do a heroic optimization on a field of enum type, if the
> enum type were loaded before the class containing the field.  That
> would be an instructive project, I think, but it is not an important one.
> 
>> 
>>> 
>>> 
>>>> The ACC_NON_NULLABLE flag indicates that this field must never store the null reference. The field signature must be the signature of a class. The class specified in the field’s signature is loaded during the loading phase of the class declaring this field. The class of the field must be a value class. This field must be initialized with the default value of this value class.
>>> 
>>> 
>>> I suppose in theory this attribute could be applied to a non-value class?
>> 
>> No as it is define today, because of the lack of non-null default value for non-value class, which makes the initialization
>> of such field impossible for the JVM.
>> 
>> A future project might be to enable ACC_NON_NULLABLE for non-value class, but
>> it would require a much complex initialization scheme, probably involving indy or condy.
>> This is way beyond the scope of this draft.
> 
> I agree.  The JVM would have to somehow track the initialization state
> of a non-nullable field, forcing each constructor to initialize it, and preventing
> access before initialization.  (Idea:  Use null internally, and have getfield
> throw NPE if there is accidental access before construction is complete.)
> 
>> 
>>> 
>>>> A field must not have both ACC_STATIC and ACC_NON_NULLABLE flags set. 
>>> 
>>> This would rule out the static fields of a enum class, although in this case i would presume an enum class is such that after static initializer block all static fields would be assigned since they are also marked final. 
>>> 
>>> So, if static fields can be final then why not non-nullable?
>> 
>> This is an open question.
>> 
>> The rational of the current choice is to avoid some circularity errors with some
>> common construct for static fields.
> 
> Don't we have similar circularity problems with non-static fields?  Why are
> static fields worse?  One answer:  You have to create the Class mirror
> very early (during preparation), and we use Class mirrors to store static
> field values.
> 
> value class A { static flat B BVAL; }
> value class B { static flat A AVAL; }
> 
> Thus, preparing A requires the layout of B, and vice versa.  To me this
> seems like an inconvenience, not an inherent flaw of flat statics.
> 
> Maybe a crude but effective way to break the bootstrap cycle is to
> store BVAL and AVAL with internal references, initialized to null.
> Then, have getstatic silently convert the internal nulls to default
> values.  It's a cheat, but nobody would know.
> 
> Or, more simply, convert flat statics into one-element arrays,
> again as an internal cheat:
> 
> value class A { static B[] $INTERNAL$ARRAY$BVAL = new B[1]; }
> 
> Then getstatic would secretly load the first (and only) element of the
> one-element array which stores the static.  It might also have to
> recognize a null array reference, during bootstrapping, and "load"
> a default value from the not-yet-existing array.
> 
>> 
>> The ACC_NON_NULLABLE flag is a marker for the JVM that indicates where flattening
>> is possible. The gain of flattening static fields is very small compared to the gains of
>> flattening of instance fields or array components. And the risk of circularity errors is high.
>> For instance, it is impossible to for a value class to have a static field with ACC_NON_NULLABLE
>> of its own type.
> 
> I hope we can use a different term:  It is not impossible; it is unimplemented.
> 
> The example is:
> 
> value class A { static flat A AVAL; }
> 
> 
>> Static fields are initialized during the preparation phase (JVMS 5.4.2),
>> which means for these fields  to store the default value of the class, but the class
>> has not been fully initialized yet, so it is still impossible to create such default values.
> 
> After the class is loaded, and before it is prepared, its instance layout is known.
> At that point the default value can be created.  I think what this puzzle shows
> us is that default values, like nulls, can exist before the initialization of a class.
> 
> (Or am I missing some vicious circularity?)
> 
>> The problem exists also with indirect references to the current class (cycles).
>> 
>> So allowing ACC_NON_NULLABLE for static fields would require to write a new set
>> of restrictions in the language about when the flag can be used and and it cannot.
> 
> Languages can do whatever they want; it's up to them to use a translation strategy
> that produces the designed semantics.  If the JVM refused to supply flat statics,
> the language could still get them with a translation strategy that used hidden arrays
> and null-to-default conversions.  But I think the JVM should not refuse.
> 
>> We saw theses constraints as a useless burden for the language (because the purpose
>> of the flag is to enable some implementation optimizations), so we proposed
>> to simply forbid them.
>> 
>> If the language team thinks there’s more value in allowing ACC_NON_NULLABLE, we
>> can revisit the current choice.
> 
> I don't see this as a language decision:  The question is whether the two modifier
> bits (static, non-nullable) should constrain each other or whether they should
> act orthogonally.  That's a JVM design issue, and I think we would regret linking
> them together in this way, because it's not a clean design.  (That's assuming I'm
> right about the absence of a true logical circularity here.)
> 
> The corresponding language question is whether value-statics must be nullable.
> It would be very surprising to me if the language team decided that was desirable.
> 
> — John