Moving from VVT to the L-world value types (LWVT)

[John Rose]

On Feb 20, 2018, at 10:13 PM, Srikanth <srikanth.adayapalam at oracle.com> wrote:
> 
> On Wednesday 21 February 2018 08:18 AM, John Rose wrote:
>> On Feb 12, 2018, at 11:12 AM, John Rose <john.r.rose at oracle.com> wrote:
>> ...
>> In new source code it will always be illegal to cast a null to a value type.
>> The rule for values is, "codes like a class, works like an int". If you cast
>> a null to (int) you get an NPE. That's what the language needs to do.
>> It doesn't matter what the JVM does.
> 
> Thanks very much for your comments, John.
> 
> I have made the observation a couple of times in my past mails that javac should align with what the VM does and your observation that "It doesn't matter what the JVM does." made me pause and think. I understand your point better now.
> 
> You say: "In new source code it will always be illegal to cast a null to a value type."
> 
> Let me ask expressly: In new source code is it legal to assign null to a value typed variable as long as it is not annotated @Flattenable and as long as it is not an array element that is being assigned to ?

No, it is not legal.  At the source code level a variable of a value type does
not take the value "null".  As an exception, certain violations of the static type
system ("null pollution") might cause a variable to appear to take a null value,
with the result that most operations on that variable will lead to NPE.

The "might" and "most" in the previous statement should be as close as possible
to "none" and "all".  Interlinking non-recompiled legacy code that thinks it is working
on value-based object classes with recompiled new code which knows it's working
on value type classes is a violation of static typing.  And we have to deal with it.

The best way to deal with it, IMO, is to allow the old code to work with nulls,
but have the new code be intolerant of nulls, whenever there is a reasonable
option to do so.  A reasonable option is (a) a checkcast in new code to a
value type, and also (b) a putfield in new code to a value type field.  I could
go either way on a putfield in new code to a field defined non-flat by old
code.  (It's a corner case.)  We could make the putfield throw NPE for those
guys also, if a null ever worms its way in, or we could make the putfield
silently store the offending null into the offensive field, with two wrongs
making a right.

One place I don't want to throw NPEs is at method calls and returns.
If bad old code injects a null into a new method, I think it is OK if the
new method doesn't "notice" the null until it does something with it.
If the new method returns the null without looking at it, then maybe
that's OK.  This is the kind of trade-off that binary compatibility across
these language changes forces us to pay attention to.  But I don't want
to make the trade-offs clever or involved, just decisive when possible
and never expensive for new code.

  class NewClass {
    OptionalInt identity(OptionalInt x) {
      return x;  // no NPE, might be null if called from old bad code
    }
    OptionalInt negative(OptionalInt x) {
      if (x.ifPresent())  // throw NPE
        return OptionalInt.of(-x.getAsInt());
      return x;  // never null
    }
    OptionalInt castme(Object obj) {
      OptionalInt x = (OptionalInt) obj;  // throw NPE or CCE
      return x;  // never null
    }
    __Flattenable OptionalInt f;
    OptionalInt rf;
    // the declaration of f is a type violation; static error please
    void store(OptionalInt x, OptionalInt y) {
      f = x;  // throw NPE
      rf = y;  // maybe throw NPE?? yes…
    }
  }


This leads to a question:  Should the following code be allowed if
OptionalInt is a value class?

  class NewClass {
    OptionalInt catchNull(OptionalInt x) {
      Objects.requireNonNull(x);  // can pass null?
      return x;  // never null
    }
    OptionalInt fixNull(OptionalInt x) {
      if (x == null)  // can test?? no…
        return OptionalInt.empty();  // can reach??
      return x;  // never null
    }
  }

The first method type-checks and does what the user wants, and
if a bad null sneaks in, it is promptly trapped and disposed of.

I think the second method should have a static error on the "if", which can
only be removed by casting x to Object.  It's very much like the existing
rules about casting and comparison of unrelated types:

        String x = "asdf";
//      if (x instanceof Number) // incompatible types: String cannot be converted to Number
//          System.out.println("bad code");
        if ((Object)x instanceof Number)
            System.out.println("not reached");
        Number y = 42;
//      if (x == y) // incomparable types: String and Number
//          System.out.println("bad code");
        if (x == (Object)y)
            System.out.println("not reached");

(With the difference that the above predicates are provably false
within the *dynamic* type system, while they are possibly true
in the analogous cases with polluting nulls typed as value types.
That's true as long as the JVM provides a little space for polluting
nulls to flow around in new code.)

> Or is this ACC_FLATTENABLE field flag bit only for the VM's jurisdiction - something javac is supposed to ignore other than setting it where it should be set ?

It is a better separation of concerns to make the flattenable bit be only for the JVM.
Then we don't need the complexity of nullable value types at source level.

> Likewise what about null comparison against value instances in new source code ?
> 
> void foo() {
>     ValueType v = null;
>     v = (ValueType) null;
>     if (v != null) {}
> }
> 
> In what is proposed, do the three operations shown above end up getting treated "regularly" - or is such regularity not required ?

Here's my take:

    ValueType v = null;  // error: ValueType is not nullable
    v = (ValueType) null;  // (ditto)
    if (v != null) {}  // (ditto)

Replacing the null with (Object)null defeats the static errors, pushing
the runtime into raising NPE:

    ValueType v = (ValueType) (Object) null;  // NPE: ValueType is not nullable
    v = (ValueType) (Object) null;  // (ditto)
    if (v != (Object) null) {}  // branch is always taken at runtime (assuming no polluting nulls)

Here's an important question:  Apart from binary incompatibilities, I think it is
the case that recompiled code cannot see polluting nulls, so that disguised
null checks (v == (Object) null, etc.) will always fail to see null, as long as
all code is recompiled.  Question:  Is that true?  Can anyone squeeze a
polluting null into new code *without* appealing to out-of-date code?

FTR, polluting nulls from old code can show up in the following ways:

  - as an incoming argument if called by old code
  - returning from a method implemented by old code (even if via virtual dispatch)
  - as a loaded field value from a class defined by old code (but not an array element)

It would be possible to add more null checks on those paths to throw
NPEs at earlier points, and so defend new code more completely from
polluting nulls.  Ideally, polluting nulls should *never* appear in new code,
but I think putting in null-check guards at a the above points is a little more
complicated than we want to do, for too little benefit.

OTOH, in favor of closing the doors completely on polluting nulls, we
did a similar exercise with polluting out-of-range values on booleans,
bytes, shorts, and chars, killing them off on field loads and function
entry points.  Maybe it's worth it.  It would be a VM change, not a
javac change.  And if we did it none of the rules I mentioned above
would change; it would just be the case that the null operations
that are statically illegal would, in fact, be impossible at runtime.

> I will require some time to digest all the observations - I have reopened JDK-8197791 so that suitable adjustments in behavior can be made.

Thanks!

— John