[lworld] Handling of missing ValueTypes attributes

[Karen Kinnear]

John,

We are trying to track multiple phases at once :-)
Agree with your descriptions.
Also looking forward to additional discussions on how to handle nullability
at JVMLS.

thanks,
Karen


> On Jul 10, 2018, at 3:55 PM, John Rose <john.r.rose at oracle.com> wrote:
> 
> Thanks, Karen; I blundered into talking about post-LW1 intentions
> when the questions were about LW1.  So, in LW1, our intention is
> to cut off, rather than adapt, mismatched accesses.  This means
> lying in wait at v-table preparation time and field and method linkage
> time, throwing an ICCE if anyone tries to set up a mismatched
> access.
> 
> Here are a few more comments, mainly to make sure I am saying
> the right things, and perhaps to throw some light in dark corners…
> 
> The usual form of a mismatch access is a method-to-method call,
> but it can also be a field reference or a use of a class from a bytecode
> like instanceof or anewarray.
> 
> Related to method-to-method calls, there is also v-table slot packing,
> which is how method overriding is linked physically in HotSpot.
> A prepared v-table slot is really a redirected call site, to execute later.
> (A subclass inherits from its super not only a prefix of instance fields,
> but also a prefix of v-table slots; the subclass can extend either.)
> That's why we do all kinds of method analysis at preparation time,
> and why we can reasonably speak of v-table slots, as well as methods,
> as sources and targets of calls.  For me it helps it think of a v-table
> slot as a physical entity, like an instance field, but for the whole class.
> 
> This trick allows me to visualize override relations between methods.
> A v-table slot is inherited into a subclass from a superclass, and
> thus v-table slots (in their various locations) are entities with their
> own relations and properties.  When I read about "method overrides"
> the language is about relations between methods, and so it is
> hard then to reason about relations between *those* relations
> unless I can visualize them somehow.  Anyway, that's why I keep
> going on about v-tables, even though they are "just an implementation
> trick".
> 
> On Jul 10, 2018, at 11:24 AM, Karen Kinnear <karen.kinnear at oracle.com <mailto:karen.kinnear at oracle.com>> wrote:
>> 
>> Let me clarify the LW1 proposal. This model is designed to NOT take into account migration of value-based classes, which we will revisit
>> much farther downstream. 
>> 
>> For LW1 - the goal is to fail with an ICCE if there are inconsistencies with value type expectations. The question is when do we catch
>> the problem and fail, and do we allow field access and method invocation when the information is inaccurate. 
>> 
>> Ioi - here is the proposal 
>> http://cr.openjdk.java.net/~acorn/value-types-consistency-checking-details.pdf
>> (As John pointed out in another email - I should update it to describe the ValueTypes attribute - for now see the not-yet-out latest JVMS draft - 
>> 
>> http://cr.openjdk.java.net/~fparain/L-world/L-World-JVMS-4e.pdf
>> 
>> For local methods there are two parts:
>> 1) if a type in a descriptor is in the ValueTypes attribute, we will eagerly load it at preparation time. If it is not a value type, we will throw ICCE.
>> 2) if a type in a descriptor is NOT in the ValueTypes attribute: see *5 Method Invocation: VT Point NOT in ValueTypes attribute in the Consistency
>> Checking link. Extracted here - let me know if this does not make sense:
>> 
>> Point is a value type.
>> ClassA contains method m1(Point point). Point is NOT in Class A’s ValueTypes attribute. I think this matches Ioi’s and Tobias’ examples.
> 
> (Yes, I think so too, plus a call to m1 from some m2 in another class,
> which as you say is case 2 below.)
> 
>> case 1: method overriding
>> ClassA’s method m1 overrides ClassZZ’s method m1, and m1 is in ClassZZ’s ValueTypes attribute. Preparation of ClassA will fail with ICCE when creating the vtable.
> 
> ClassA inherits a v-table slot from ClassZZ that wants unboxed Points
> (passed as fields) but ClassA's method doesn't fit in that slot, since it
> uses Point refs (passed as oop).  Does that sound right?
> 
> Note that a calling sequence might not physically unbox a Point,
> because of some implementation heuristic or limitation.  However,
> the JVM must enforce ref/value consistency even in such a case,
> because the JVM needs to give a predictable model for error checks.
> If the JVM has the *option* to unbox a value, it needs to protect
> that option by throwing errors on inconsistent code even in cases
> where it doesn't *exercise* the option.
> 
> As a proxy for "could I unbox this Point value?", you can also ask
> "could I pass a null for this Point value?"  The two conditions are
> mutually exclusive, and the middle ground (of passing as oop
> but excluding null) is not visible to the user of the JVM; the user
> can't see physical calling sequences inside a virtual machine.
> 
> This is why nullability has arisen as a way to test whether a variable
> is "really a value" or is just passed by reference.  Nullability *also*
> has an independent role in managing certain legacy APIs where
> null is used as a sentinel value.  When we work out all the details,
> the JVMS might end up talking about identity as a property of
> the loaded type itself, but nullability as a property of various
> containers of that type.  Both identity and nullability need to be
> suppressed, via independent means in LW1, in order to fully
> flatten heap data and scalarize method arguments and returns.

> 
> (As instance fields and v-table slots are extended in parallel
> across a class hierarchy, value type optimizations follow along
> both avenues.  There is a deep duality between method parameters
> and object fields operating here.)
> 
>> ClassD extends ClassA, ClassD’s method m1 overrides ClassA’s m1 and m1 is in ClassD’s ValueTypes attribute. Preparation of ClassD will fail with ICCE when creating the vtable due to the mismatch.
>> Note ClassA keeps going - I don’t know of a way to bubble up the failure since ClassA may be in use.
> 
> So:  ClassA *creates* a v-table slot that wants Point refs.  Then when
> ClassD tries to fill that slot with a method that wants unboxed Points.
> 
> (Note that if A extends ZZ, ClassA already died during preparation,
> so ClassD will also die during preparation.  But here ZZ is not present,
> so A gets to determine the shape of the v-table slot.)
> 
> Alternate scenario:  ClassA inherits a v-table slot for m1 from ClassZZ that
> wants unboxed Points, and ClassA does *not* override m1.  ClassA commits
> no real foul, since it doesn't try to change the contents of the slot.  I think we
> let this pass, since we only look at overrides, but the details are always tricky.

> 
>> case 2:
>> ClassB has Point in ValueTypes attribute and invokes ClassA.m1.
>>  - ClassB can successfully create an instance of Point
>>  - at method resolution time, caller-callee consistency check will throw ICCE, so ClassB can not invoke ClassA.m1
>> (Tobias - I think this is your last paragraph “Now another compiled method)
> 
> Here ClassB wants to attach to ClassA's v-table slot for m1.  Just like
> m1 itself, the slot has a calling sequence.  In LW1 we aren't messing with
> on-the-fly adaptation, so when B tries to link to A.m1 (either the v-table
> slot or the method itself), the calling sequences don't align which causes
> ICCE.
> 
>> case 3:
>> ClassC does NOT have Point in ValueTypes attribute and invokes ClassA.m1
>>  ClassC is able to invoke ClassA - caller/callee check passed consistency (without checking reality)
>>  ClassC can pass a null Point
> 
> In that case old code just does the old thing.  One might ask how does
> the old code get a reference to the new value, if all the calling sequences
> are locked into alignment?  The answer is that value instances are very
> mobile in an LW* JVM; they can be laundered through java.lang.Object,
> or passed as array or field elements, or through JNI, or through untyped
> MH internals.  It's hard to close all the holes.
> 
> Thus, masking a Point as an Object allows old code to grab it.  But I think
> we do make it pretty difficult for the old code to re-type the thing as a non-value
> Point.  In order to do that, it must issue a checkcast (or equivalent), and in
> LW1 (as described in Karen's very thorough write-up) a checkcast *does*
> perform a consistency check, because it must resolve its CONSTANT_Class,
> and those are *always* checked against Reality.  That means the LW1 JVM
> will throw ICCE on old code that attempts to cast an Object to a Point (if Point
> is a value and the old code didn't know that).  The code for detecting such a
> mismatch is here in Fred's patch:
> 
>  http://cr.openjdk.java.net/~fparain/VTAttributeChecks/webrev.02/src/hotspot/share/oops/constantPool.cpp.udiff.html <http://cr.openjdk.java.net/~fparain/VTAttributeChecks/webrev.02/src/hotspot/share/oops/constantPool.cpp.udiff.html>
> 
> (Karen did I get that right?)

> 
>> So - from a compiler perspective - only classes that share the same ValueTypes attribute information about Point can call ClassA.m1.
>> 
>> From an execution perspective, we would like to ensure that neither ClassC nor ClassA can get their hands on an instance of Point, so
>> we are only passing null here. Tried to close all of these holes - could use review.
> 
> It's hard to close all the holes, as noted above.  And some holes we don't want
> to close:  Putting a Point into a List<Object> or Object[] means old code *can*
> grab it and operate on it, as long as the old code doesn't mention the type
> Point in its constant pool.  That's an important interoperability property we
> want to keep, if we can.
> 
>> Note to John - consistency checking for array elements
>> relative to ValueTypes attribute is part of closing this hole, i.e. ensuring that a class that has the wrong information about whether a type
>> is a value type can not get their hands on an instance of that value type.
> 
> OK, that makes sense.  Thanks for explaining.  If we want to allow old code
> to make a closer acquaintance with the point Point, then we have to implement
> a new kind of "view" of Point, a reference to a buffered value, which is nullable.
> And we are avoiding that in LW1, pending further analysis.
> 
> If we add nullable references to values in the JVM as a feature post-LW1, then
> we need to work out the various conversion and adaptation paths, as well as
> decide whether those two views of Point should be spelled differently (as JVM
> type descriptors), or spelled the same with contextual interpretation.  If they
> are different (say R-Point vs. Q-Point), then we need lots of tech for descriptor
> shifting adapters, and we also need to chose which spelling (if either) is the
> legacy spelling L-Point.  If they are the same, then it's the legacy spelling all
> around, and we need to be very careful to manage the side-channel information
> which indicates the Q/R distinction.  I hope to discuss this more at the JVMLS.

> 
>> Does this make sense for LW1?
> 
> Yes; hopefully my comments are correct also.
> 
>> thanks,
>> Karen
>> 
>> p.s. Frederic checked in round 1 of the consistency checking - but did not get round 2 in before he left for vacation. He is out this
>> week and the next. I attached his patch out for review if you want to use it to test with to see if that helps the compiler. He will be back I
>> believe the 23rd. I will check with Harold on Frederic’s verifier question - we may want to push this (without perhaps the new test) before
>> he gets back so you can use it.
>> 
>> Note - John asked for some refactoring - given how tight the time is before EA and the vacation schedule - that will be a postlw1 rfe.
> 
> As I think I said to Fred, that's fine with me.
> 
> — John