From karen.kinnear at oracle.com Wed Jan 3 16:11:19 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Wed, 3 Jan 2018 11:11:19 -0500 Subject: Valhalla EG minutes Dec 20, 2017 Message-ID: <131B1F8D-F49C-47F8-BE7C-A660CBE1FCB5@oracle.com> Happy New Year! Attendees: Remi, Dan H, Tobi, Dan S, John, Karen AIs: All - review nestmates JVMS invokeinterface selection changes All - review Dan?s proposal for constant pool structural descriptors Remi - proposal for T-Types - simpler approach to generic specialization I. Nestmates 1. JVMTI -agreed to modify JVMTI specification to disallow modifying NestHost or NestMembers attributes as part of class redefinition. 2. InvokeInterface selected method discussion - Karen sent an email http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2017-December/000486.html with an attached .pdf that essentially described the invokeinterface selection changes from pre-nestmates. (oops - .pdf was stripped) Need to revisit from further investigation - I will send a separate email. II. Constant Pool future changes Dan followed-up with an email: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2017-December/000490.html Dan S discussed a proposal to evolve Descriptors in the Constant Pool from strings to structured, in part motivated by template specialization. Today we use name mangling for arrays and for the Minimal Value Types prototype. Proposal is to have a Descriptor refer to a set of other Constant Pool entries which describe types, e.g. CONSTANT_ArrayType_info - would point to the type of its element e.g. CONSTANT_PrimitiveType_info - would have a byte which says which primitive e.g. CONSTANT_ParametersAndReturn_info - CP indices for return and parameters Goals: Dan H: request fixed length constant pool entries -> leading to proposal to contain the parameters and return value in an attribute rather than in the CONSTANT_ParametersAndReturn_info directly. Karen: ability for implementation to continue to use a String - so as to allow fast method lookup searches and reduce API changes, without having to do lots of constant pool resolution/class loading Karen: need ability to incorporate live types with different class loaders - this is a new challenge Remi: Already have L/D/I/F Would the new version be e.g. int.class? Today you can?t store an int in an object, but we can store a value type with an int - is it possible we don?t need the new primitives - or at least too soon to know? Is it possible JIT can do most of the specialization and we won?t need this? Dan S: other motivations John: wants this for template classes Remi: What if instead of template classes we had a T-Type descriptor - which is less powerful than template holes - substitution fills the CP holes AI: Remi - write a proposal - we would like to see how that might work Dan S: Cycle handling Array types can have cycles - rather than relying on ordering - perhaps tell tree depth - e.g. component type & dimensions - all in the CP John: perhaps modify to include the depth indicators in an attribute (separate proof of acyclicity) Remi: but if you insert a condy ? John: would need to discard the attribute Dan H: concern about a constant array for each dimension ? maybe some adhoc compression techniques Remi: why do you want to be cycle free? Dan S: hard to verify if types cyclical? Remi: why do you need to check in the verifier? John: maybe a structural constraint? Remi: if value - ok, but not for an array of values? e.g. Object[] with first entity as itself Dan: want array type convertible to finite string, benefit of attribute not tied to construct is it can evolve Dan H: Concerns about attributes - will this be as hard to maintain for e.g. class redefiners/bytecode generators as stackmaptables? No - you have to keep the CP indices in sync, but stackmaptables required abstract interpretation - transformer did not have the type information, and the compression mechanism was harder to implement and these would just require bumping indices. III LWorld Value Types - Remi Issue 1: Can Object[] store an array of VT? Dan S: NOT allow as part of subtyping Karen: missed that - Brian asked if X <: Object is X[] <: Object[] e.g. if X is a Value Type Why would this not be true? Dan S: performance concerns? (ed. note - I need to understand this better) Issue 2: putfield semantics aaload/aastore can do the same thing for value types and references if field is a ValueType can?t put a reference type in it Karen: for an array of VT - must have a VT to store in it for an object container that has a value type field - you can write a different value type in the field the restriction is on updating a field in an immutable value type itself Remi: when can the VM store a VT as an Object (ed. note: I presume he meant indirectly vs. flattened) Note: VM chooses when to do flattening - need to clarify in spec The author does NOT know if a value type has been flattened in a given container - reference, value type, array thanks, Karen From karen.kinnear at oracle.com Wed Jan 3 16:16:06 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Wed, 3 Jan 2018 11:16:06 -0500 Subject: Nestmates JVMS changes to selection and behavior implications In-Reply-To: <82E5D22B-C4DE-4C60-9AAA-62291CD9FDC2@oracle.com> References: <82E5D22B-C4DE-4C60-9AAA-62291CD9FDC2@oracle.com> Message-ID: After discussions with David Holmes, I have updated the description to note that there were three changes to invokeinterface selection - so we can discuss which ones we actually want. .pdf copied below since it was stripped in earlier emails. Focus of the discussion on potential changes is on invokeinterface selection changes. thanks, Karen ======= Invocation changes due to the December 2017 Nestmates JVMS proposal: http://cr.openjdk.java.net/~dlsmith/nestmates.html are described below. In studying the details of these changes, there are three sets of changes that change the behavior of invokeinterface. Invokeinterface is allowed to resolve to a private member of the reference class, and the resolved method becomes the selected method. The removal of the invokeinterface selection runtime access check, allows selection of a package-private or protected method, since they can override a public method from the resolution step. The combining of the invokeinterface selection and invokevirtual selection (and equivalent preparation modifications) add the concept of "overrides" to the invokeinterface selection lookup steps. Note that private methods never override and are never overridden. The consequence of this change is that the invokeinterface selection lookup will now SKIP private methods as invokevirtual does, where before it would find the private method and throw an IllegalAccessError. I am proposing that for nestmates we do not need the second change, and that we could continue to have invokeinterface and invokevirtual selection be aligned, while keeping the restriction that if the selection lookup procedure selects a method that is not public, invokeinterface throws an IllegalAccessError. (This would continue to allow resolution to a private member, select the same private member). This prevents adding additional cases in which the caller is able to invoke a method it can not directly access. For the 3rd change we have two ways to approach this. We can leave the existing invokeinterface selection/preparation alone, find private methods in the receiver and its superclasses, and throw an IllegalAccessError, or we can make the proposed changes and skip private methods as we do for invokevirtual (because of the term "overrides") and potentially find a public method that overrides the resolved method. Invokeinterface Selection Changes: JVMS 6.5 Invokeinterface changes Let C be the class of objectref. The actual method to be invoked is selected by the following lookup procedure: The selected method (5.4.5 ) for class C of an invocation of the referenced method is the actual method to be invoked. If C contains a declaration for an instance method with the same name and descriptor as the resolved method, then it is the method to be invoked. Otherwise, if C has a superclass, a search for a declaration of an instance method with the same name and descriptor as the resolved method is performed, starting with the direct superclass of C and continuing with the direct superclass of that class, and so forth, until a match is found or no further superclasses exist. If a match is found, then it is the method to be invoked. Otherwise, if there is exactly one maximally-specific method (5.4.3.3 ) in the superinterfaces of C that matches the resolved method's name and descriptor and is not abstract, then it is the method to be invoked... Linking Exceptions Otherwise, if the resolved method is static or private, the invokeinterface instruction throws an IncompatibleClassChangeError. Run-time Exceptions Otherwise, if step 1 or step 2 of the lookup procedure selects a method that is not public, invokeinterface throws an IllegalAccessError. This is the line that we believe could be restored, slightly modified to apply to the selection lookup procedure steps in 5.4.5, so that the only non-public selected method would be a private method which is the referenced method as well as the selected method. 5.4.5 Selection new section: Where an invokevirtual or an invokeinterface invocation refers to a resolved method mR, the selected method of the invocation for an instance of a class or interface C is determined as follows: If mR is marked ACC_PRIVATE, then it is the selected method. Otherwise, the selected method is determined by the following lookup procedure: If C contains a declaration for an instance method m that can override the resolved method, then m is the selected method. Otherwise, if C has a superclass, a search for a declaration of an instance method that can override the resolved method is performed, starting with the direct superclass of C and continuing with the direct superclass of that class, and so forth, until a method is found or no further superclasses exist. If a method is found, it is the selected method. Otherwise, if there is exactly one maximally-specific method (5.4.3.3 ) in the superinterfaces of C that matches the resolved method's name and descriptor and is not abstract, then it is the selected method. Note that any maximally-specific method selected in this step can override mR; there is no need to check this explicitly. While C will typically be a class, it may be an interface when these rules are applied during preparation (5.4.2 ). Beyond improving presentation, this change has the following effects: Addresses JDK-8098577 : if an invokevirtual resolves to an interface method, C or one of its superclasses may override the interface method. Prevents private methods from being selected by invokeinterface, unless the private method is also the referenced method. InvokeInterface Selection Table Summary: Current nestmates JVMS proposal: Invokeinterface has been modified to support private members as well as to be more consistent with invoke virtual by defining selection based on overriding, which does not change any success cases, and does change some IAE cases to become success cases. Color coding: The current proposal makes both the green and the red changes. The proposed modification would only make the green changes, i.e. only change behavior relative to private methods support for referenced method is private -> == selected method support for selection lookup skipping private methods in Resolved Method X Selected Method selected: local C.m public local C.m Prot/PP C.m private SuperC.m (includes Object.m) public SuperC.m (includes Object.m) Prot/PP SuperC.m Object.m private SuperJ.m Pub/def resolved: local I.m private old: IAE check, then ICCE new: I.m old:N/A new:I.m - - - - - - - local I.m public - C.m old:IAE new: C.m old: IAE new: skip SuperC.m old: IAE new: SuperC.m old: IAE new: skip max-specific Object public m - C.m old: IAE new: C.m old: IAE new: skip SuperC.m old: IAE new: SuperC.m old: IAE new: skip - SuperI: public default - C.m old: IAE new: C.m old: IAE new: skip SuperC.m old: IAE new: SuperC.m old: IAE new: skip max-specific SuperI: public (abstract) - C.m old: IAE new: C.m old: IAE new: skip SuperC.m old: IAE new: SuperC.m old: IAE new: skip max-specific Key: Rows are Resolved Method, Columns are available methods for selection SuperC is a super class, SuperI is a super interface of I, SuperJ is a super interface of C Not in this table: If the selected method in local, SuperC, Object or SuperJ is Abstract -> AME unchanged. If there is more than one maximally-specific method -> ICCE unchanged If there are zero maximally-specific methods -> AME unchanged If there are private methods in SuperI at resolution time, they are skipped -> unchanged If there are private methods in SuperJ at selection time, they are skipped -> unchanged Additional checks performed on source: double-check all the AME error handling I do not believe these cases have changed are there any error cases in which we change what error we throw? I do not believe so are there cases in preparation in which we check loader constraints, which actually throw exceptions? Hotspot does not check loader constraints for exception throwing cases Invokevirtual Selection Changes (8098577) 6.5 Invokevirtual Let C be the class of objectref. The actual method to be invoked is selected by the following lookup procedure: The selected method (5.4.5 ) for class C of an invocation of the referenced method is the actual method to be invoked. If C contains a declaration for an instance method m that overrides (5.4.5 ) the resolved method, then m is the method to be invoked. Otherwise, if C has a superclass, a search for a declaration of an instance method that overrides the resolved method is performed, starting with the direct superclass of C and continuing with the direct superclass of that class, and so forth, until an overriding method is found or no further superclasses exist. If an overriding method is found, it is the method to be invoked. Otherwise, if there is exactly one maximally-specific method (5.4.3.3 ) in the superinterfaces of C that matches the resolved method's name and descriptor and is not abstract, then it is the method to be invoked. JVMS 5.4.5 Overriding new section on common selection rules Addresses JDK-8098577 : if an invokevirtual resolves to an interface method, C or one of its superclasses may override the interface method. Invokevirtual behavior changes If a resolved method was private and in the same class as the caller, prior to this change, I do not see a clear selection definition since overriding does not apply to private methods. Implementation kept resolved == selected which is now clearly spelled out in the specification. Clarification: SuperC.m private, C extends SuperC. SuperC invoke virtual C.m, resolves to SuperC.m private Old and new specification should continue to allow invokevirtual of SuperC.m via C.m. New specification would extend this for nestmates. Addresses JDK-8098577 : if an invokevirtual resolves to an interface method, C or one of its superclasses may override the interface method. update from Dan H - this is not a behavioral change, this is a clarification in the specification. Preparation Changes: 5.4.2 Preparation (changes) During preparation of a class or interface C, the Java Virtual Machine also imposes loading constraints (5.3.4 ). Let L1 be the defining loader of C. For each instance method m declared in C that overrides can override (5.4.5 ) a an instance method declared in a superclass or superinterface , the Java Virtual Machine imposes the following loading constraints: ... Furthermore, if C implements a for each instance method m declared in a superinterface of C, but if C does not itself declare the method an instance method that can override m, then let be the superclass of C that declares the implementation of method m inherited by C. The if the selected method (5.4.5 ) for class or interface C of an invocation of m is declared in a class or interface , the Java Virtual Machine imposes the following constraints: ... Changes in Preparation relative to Selection Behavior: Summary: New specification is more accurate than the old one. It is still challenging to map to the implementation. One example, prior to nestmates, the hotspot implementation treated the second paragraph as if it said "if C implements an ACC_PUBLIC method". We did not perform loader constraints for cases in which the selected method would throw an exception, since the intention of preparation is loader constraint checking when creating a selection cache. With nestmates, and the modification to use the term override to match the common selection description, this would also allow ACC_PROTECTED and package private methods to be selected, and therefore have their loader constraints checked. All agreed that an interface method never overrides a class method (even for the java.lang.Object case). An interface method can override another interface method > On Dec 20, 2017, at 10:45 AM, Karen Kinnear wrote: > > David Holmes and I were studying the JVMS changes for nestmates - in particular the selection and preparation changes which > are designed to make invoke interface more like invoke virtual (JDK-8024806). > > I wanted to ensure that the behavior changes were all either error -> success, or error 1 -> error 2. > > As I went through this again, I had a couple more questions on the JVMS changes for selection preparation to sanity check. > > thanks, > Karen > > > > > From karen.kinnear at oracle.com Wed Jan 3 16:59:05 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Wed, 3 Jan 2018 11:59:05 -0500 Subject: LWorld1 initial phase proposal In-Reply-To: References: Message-ID: Many thanks. A couple of points: > > Example: A field of value type either needs an explicit flatten flag, > or a Q-descriptor. The flatten flag (ACC_VALUE) makes an unambiguous > context for the L-descriptor of the field type, so that it means a Q-type > rather than an L-type. Just to clarify: a value type is ?flattenable? - the JVM makes the decision to flatten or not - which may be implementation, platform, command-line-flag, etc. dependent. > >> II. Assumptions: >> 1. New root: LObject - for all references and value types > > java.lang.Object is a U-type. > Object classes (other than java.lang.Object) define R-types. > Value classes define Q-types. > Interfaces define U-types. > Abstract classes define R-types. > > The following are possibilities but I hope most of them can be false: > > Perhaps some object classes will *also* imply frozen versions which are Q-types; TBD. Could we possibly decouple the conversations about immutable classes from frozen instances? Or am I misunderstanding what the ?freeze? behavior would do? I thought the goal was to freeze a given instance (or array instance) - which does not imply any of the other value-based class characteristics that we can optimize for. > Perhaps some value classes will *also* imply heavy-boxed versions which are R-types; TBD. > Perhaps some interfaces will be *restricted* to Q-types or R-types; TBD. > Perhaps some abstract classes can help define Q-types or U-types; TBD. > >> flattenable when contained in reference, value type or array > > "contained in reference" is ambiguous. I suggest this language: > > flattenable when contained in a variable (instance field, static field, array element, or local) Much clearer. Thanks. Current proposal does not flatten in static fields - we don?t expect a significant benefit for that. > >> support interfaces > > Yes. Interfaces define U-types. (TBD whether there are any variations on this theme.) > > And we can say java.lang.Object is an "honorary interface". It is a U-type > which all classes implement. Like interfaces, it is a valid bound for all types > and all generic variables. Still trying to figure out what we mean by ?honorary interface?? Not allowed fields? Can be implemented by value classes and object classes? Other? more later, thanks, Karen From brian.goetz at oracle.com Wed Jan 3 20:29:19 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 3 Jan 2018 15:29:19 -0500 Subject: Received on valhalla-spec-comments: Generics and null In-Reply-To: References: Message-ID: <8d9585b3-0c9b-41f6-5207-dea73821a56e@oracle.com> The following was received on valhalla-spec-comments (archived at http://mail.openjdk.java.net/pipermail/valhalla-spec-comments/2017-December/000002.html). Yes, such an operation is necessary both for generic code ("return the default value for the type corresponding to type variable T"), and is also useful for non-generic code ("give me the default value for int").? The early Valhalla prototypes implement the syntax ".default" for this: ??? int zero = int.default; or ??? T zero() { return T.default; } The default is already well-defined for existing types (primitives and refs); value types similarly need a default, which currently is defined to be the value with "all default" values for its fields.? (This is a delicate choice; on the one hand, this is the obvious implementation -- write zeros everywhere -- but for some values, this bit pattern might not always have an obvious meaning, and this puts a burden on implementation code to deal with this.) Default values also implicitly arise from uninitialized fields and from newly allocated arrays. On 12/31/2017 2:17 PM, Jack wrote: > With generics soon to be expanded to also include primitive types, Value types, and data types, I'm under the impression that null (yes that elephant in the room that nobody likes to talk about) will be unsuitable as a default assignment for generic types. I'm not saying we need some sort of super null that encompasses all possible types, but how about a way to dynamically ask for the type-correct default value at run time? > Maybe something like this: T foo = assignmentCompatibleInstanceForThisGenericType T; > > Thank you for your time, > Jack From john.r.rose at oracle.com Thu Jan 4 00:46:45 2018 From: john.r.rose at oracle.com (John Rose) Date: Wed, 3 Jan 2018 16:46:45 -0800 Subject: LWorld1 initial phase proposal In-Reply-To: References: Message-ID: <9E3C640F-D4EF-4284-96B7-05BC97C27975@oracle.com> On Jan 3, 2018, at 8:59 AM, Karen Kinnear wrote: > > Many thanks. > > A couple of points: >> >> Example: A field of value type either needs an explicit flatten flag, >> or a Q-descriptor. The flatten flag (ACC_VALUE) makes an unambiguous >> context for the L-descriptor of the field type, so that it means a Q-type >> rather than an L-type. > Just to clarify: a value type is ?flattenable? - the JVM makes the decision to flatten or not - > which may be implementation, platform, command-line-flag, etc. dependent. Thanks. (The internals could sometimes?or even always?use the same non-flat representation as a heap buffer. The JVM's virtualization doesn't guarantee actual flattening, but allows it, and in order to do so the user of the JVM's virtual ISA must agree on a special distinction for "flattenable" types. One user model for this is that a "flattenable" type is "virtually flattened" which means it is impossible to prove or disprove, except maybe for resource constraints, whether the JVM has flattened or not. Similarly, it is impossible to prove or disprove whether the GC has deallocated an unreachable object, but it may be regarded as "virtually deallocated" the moment it becomes unreachable. Sound good?) >> >>> II. Assumptions: >>> 1. New root: LObject - for all references and value types >> >> java.lang.Object is a U-type. >> Object classes (other than java.lang.Object) define R-types. >> Value classes define Q-types. >> Interfaces define U-types. >> Abstract classes define R-types. >> >> The following are possibilities but I hope most of them can be false: >> >> Perhaps some object classes will *also* imply frozen versions which are Q-types; TBD. > Could we possibly decouple the conversations about immutable classes from frozen instances? Sorry, I'm not sure what you are asking here, since the concept of "immutable class" could have several meanings. I'm talking here about object classes, which are internally mutable, supporting stronger forms of immutability, notably "frozen" states. Such forms may be strong enough to be treated as Q-types. I'm happy to decouple stuff, but I don't see a clean line here yet. Object class immutability requires mutability control at the instance level, because final fields are sometimes assignable. Object classes must support mutable larval states for their instances while under construction, even if they are logically immutable (like jl.String). Value classes are inherently immutable, of course, but that's just because they work like values, even in private methods or constructors. Along the same lines, some mutable (not all) classes could be upgraded to support immutability on a per-instance basis. This can be viewed as a separate feature (instance "freezing") but IMO should be considered alongside the need for larval construction states of fully immutable object classes. > Or am I misunderstanding what the ?freeze? behavior would do? I thought the goal was to > freeze a given instance (or array instance) - which does not imply any of the other value-based > class characteristics that we can optimize for. My point is that at some point a frozen object, if frozen "solidly" enough, can be treated as if it were a heap-box for a corresponding value. When that happens, you could perform an R-to-Q conversion on it, and walk away with "just the value". How is that better than doing an R-to-U conversion? Not much better, but it *does* imply that the user of the resulting Q-value cannot accidentally witness the object identity, by synchronizing or acmp-ing it. It would be a safety feature, if we did it. Even if you had an unfrozen mutable object, such as an AtomicInteger or an int array, there would be some value in performing an R-to-Q conversion on it, as long as you pinky-promised that there were no future or outstanding concurrent writes to it. The memory model for this could be similar to that of today's arrays when reached by a final field, or @Stable fields: Something bad *could* happen if code had a buggy mutation operation, but it won't, because the code doesn't. Such an AtomicInteger could be optimized by loading its current int value and walking away with it, forgetting about the buffer. Passing it around under a Q-type would enable such optimizations, while passing it around under an R-type or U-type would not, since the JVM has to assume that someone might look at the box's identity at some time in the future. With a Q-type, the JVM knows that the box's identity may be suppressed and/or discarded. If, in addition, the AtomicInteger is marked by a JVM-enforced frozen bit, then the pinky-promise is enforceable. But the same optimizations apply in either case. We can deal just fine with unenforceable promises by saying that breaking an unenforced promise leads to a modicum of "indeterminate behavior", as with races in the JMM. > >> Perhaps some value classes will *also* imply heavy-boxed versions which are R-types; TBD. >> Perhaps some interfaces will be *restricted* to Q-types or R-types; TBD. >> Perhaps some abstract classes can help define Q-types or U-types; TBD. > > >> >>> flattenable when contained in reference, value type or array >> >> "contained in reference" is ambiguous. I suggest this language: >> >> flattenable when contained in a variable (instance field, static field, array element, or local) > Much clearer. Thanks. > Current proposal does not flatten in static fields - we don?t expect a significant benefit for that. That's an implementation matter; I'm talking mainly about the virtual machine. Static fields should be virtually flattenable, as a matter of design regularity. It's OK to ignore the ACC_VALUE bit for statics, if that simplifies the implementation. IMO it's not OK to say the ACC_VALUE bit is illegal on statics, since that just puts in a sharp edge where we don't need one. >> >>> support interfaces >> >> Yes. Interfaces define U-types. (TBD whether there are any variations on this theme.) >> >> And we can say java.lang.Object is an "honorary interface". It is a U-type >> which all classes implement. Like interfaces, it is a valid bound for all types >> and all generic variables. > Still trying to figure out what we mean by ?honorary interface?? I mean that getClass/toString/equals/hashCode API points are programmable from value types and generics and default methods just like compareTo, and under analogous conditions. So Object works like Comparable, where it matters. If Object or Comparable is my super, I can override its methods (in a value class or interface, as well as an object class). If a variable (including 'this') has Object or Comparable as a super, I can make a call against its API points. At some point we'll have to make it official with real spec. language. For now we limp along by saying "an interface" and then adding weasel words like "or Object, excluding certain methods like wait/notify/finalize". I suppose we could say that there is a notional type, an interface derived "in the obvious way" from Object, that looks like this: package java.lang; interface I$Object { Class getClass(); String toString(); boolean equals(Object x); int hashCode(); } (Maybe it's even an explicit type, coded in the JDK?) And then some occurrences of Object are rewritten as if they were I$Object. And we work the rules so that all types implement I$Object, and all calls to Object methods also mentioned in Object are rewritten to call against I$Object. > Not allowed fields? Nope; nobody wants fields on Object anyway. Not constructors either. The important thing is methods (override and call). > Can be implemented by value classes and object classes? Yes. > Other? Methods of [I$]Object are always available on 'this' in value classes, on 'this' in object classes, on 'this' in interface default methods, and on any variable of a generic type. From john.r.rose at oracle.com Thu Jan 4 02:40:04 2018 From: john.r.rose at oracle.com (John Rose) Date: Wed, 3 Jan 2018 18:40:04 -0800 Subject: LWorld1 initial phase proposal In-Reply-To: References: Message-ID: <34237B64-8A3D-44E5-A16A-8AE17FD1DC6F@oracle.com> (here's some of the "more later") On Dec 13, 2017, at 11:58 AM, Karen Kinnear wrote: > > ? > V. Expected Behaviors: > > ? > > 2. Java level APIs > isValue There are a few places where we can ask whether something is a value: 1. Given a U-type(*) variable x, is that variable's current value a value class? System.isValue(x) 2. Given a class mirror, is it a value class? Class.isValue(). 3. Given a reflected field, was it marked as (virtually) flattened, aka ACC_VALUE? Field.isValue(). (In L-world, a U-type appears to the JVM as an L-type, often an actual L-descriptor like "Lpkg.Foo;". The term U-type refers to a new way of using of L-type values. Likewise, R-types also appear to the JVM as L-types, but the term R-type refers to the legacy usage of legacy L-types. A Q-type enables a new way of passing or storing operands by value instead of by reference. In L-world, there are mainly just L-types and L-descriptors. If we need actual Q-descriptors in the JVM, it is only because sometimes there is no better way to encode the directive to use a Q-type, that is pass or store an operand only by value.) > isFlattened (for a reflection Field or array) Not sure what this is. Same as #3 above? Do we need, at the virtual level (not the physical level) two separate concepts, value and flattened? Or is this a distinction that applies only to the physical (sub-virtual) level? There is a low-level query we can make, which is to ask what the JVM has decided about physically (not virtually) flattening a field or a whole class. I would prefer to place that query out of reach, so that folks can't shoot themselves in the foot. It's a moral equivalent of @ForceInline, something that is very easy to misuse, and of little value in portable code. > isElementValue (for an array - not the same as is the array a Value) Yes, although the correct term is "component", not "element". Cf. getComponentType. field : array element :: Field.getType : Class.getComponentType field : array element :: Field.isValue : Class.isComponentValue We are touching on a weakness of reflection, where we want to use jl.Class to refer to types, but it isn't quite rich enough. Brian calls a hypothetically enriched Class a "crass". Such a thing can report not only the "proper" class associated with a type, but also its modifier (Q/R/U) or its template parameters. Given such a thing we can factor all the ancillary type queries onto the common type: interface CrassType { Class getProperClass(); boolean isValue(); // Q-type boolean isObject(); // R-type boolean isUniversal(); // U-type //boolean isTemplate(); //List getTemplateParameters(); } class Class implements ? CrassType { ? } The various in-context queries then turn into single crass-queries: CrassType Class.getComponentCrassType(); CrassType Field.getCrassType(); > ucmp - substitutability check (not overridable) Given the cost of this, I prefer to phrase it as a library call instead of a bytecode instruction: System.isSubstitutableValue and then something like: System.getSubstitutableHashCode These would be available to upgrade the various kinds of identity-sensitive collection types we have today, to wean them off of acmp and System.identityHashCode. > 3. Support beyond MVT > Allow Value Types in static fields (not flattened) Yep. (My comments above apply post-MVT!) > Method and method invocation support > Interface support: default methods - must handle both L-types and Q-types > > 4. LWorld bytecodes vs. JVMS 9 > Key Challenge: can we apply the same bytecodes to QTypes and RTypes? can they check dynamically without loss of performance? > special handling: > if_acmpeq/if_acmpne: false/true if either is a Qtype. I.e., JVM detects a "NaN-like" condition: Both operands are identical, *but* both are also values (the same value). In that case, we return false for acmpeq. > They should fall back to .equals "They" = "User code". It's important to say here that user code should fall back to Object.equals or a similar method (such as System.isSubstitutableValue) if identical values should be treated as identical. When the fall-back is Object.equals, we have encountered LIFE as we know it?the Legacy Idiom For Equality?which looks like "x==y||x.equals(y)". There are several known forms of LIFE, because of details like null processing. The JIT should optimize these bytecodes by detecting signs of LIFE (ooh, I gotta million of these) and transforming to cheaper machine code. For example, remove the value-detection part of the acmp semantics, and refrain from calling Object.equals in the case of a value being compared to itself. This transformation requires that we trust that Object.equals fulfills its contract, which requires a little more thought. It's probably enough to say "if the JVM can prove that two values are copies of each other, it may short-circuit calls to Object.equals on them to true". This basically enshrines a promise of the Object.equals API into an optional optimization in the JVM itself, which I think would be harmless, and it would enable us to make the above transformation with a clear conscience. > ifnull/ifnonnull: as if acmp vs. Null: false/true if QType Yep. That's cheap, as long as no value is encoded by a null buffer pointer. > needs dynamic different handling: > aaload/aastore: handle LType or QType dynamically This is worth experimenting with; maybe it's cheap enough. Something in the object's header (or one hop away) can say whether the array is flattened or not, and the interpreter or JIT can branch to the appropriate code. > aload/astore: handle LType or QType dynamically We have found that the cost here is small, since the dynamic check is only needed on return. With Loom, we might need a more complicated check on yield. That's when a number of frames are temporarily unwound from a thread, to be remounted later on some other thread. For that case, anything buffered in the thread needs to be rebuffered either on the fiber or in the heap. I prefer the heap for starters, since I am trying to resist all forms of mission-creep for fibers. But there's probably some way to eventually use the same fiber-local storage that the fiber uses to store the stack frames. > areturn: handle LType or QType dynamically That's where the cost for aload/astore goes. > exception if wrong kind: ICCE > putfield: QType exception: ICCE > monitorenter/exit: exception for QType: ICCE > new: exception for QType: ICCE (expects uninitialized state) Good. > aconst_null: exception for QType: ICCE This bytecode doesn't take a type parameter, so it can't object to a Q-type. I think you mean: null value: exception when passed as QType: ICCE There may be a role for NPE to play here. For example, we could have a u2q instruction which checks types *and* checks for nullness, through CCE *or* NPE. > vdefault: exception for LType: ICCE Or just return null. When we come to templated generics we'll have to reconsider this. > > withfield: exception for LType: ICCE Yes. > > unchanged or already implemented (in MVT) or should fall out: > getfield: handle LType or QType dynamically (already implemented) putfield: handle LType or QType dynamically (already implemented) > newarray/etc.: handle LType or QType dynamically (already implemented) > athrow: always LType (subtype of Error) - unchanged > invoke*: handle LType or QType dynamically (should fall out) > checkcast/instanceof: should fall out A checkcast to a value type should throw NPE if presented with a null. > ldc: should fall out > > V. Implementation use of explicit QType > 1. Field descriptors > Goal: not require verifier or class file parser to load all fields. > ICCE if misclaimed, at first runtime mismatch (kind constraints) > To allow flattening, want field and arrays to explicitly use QTypes at language level There's a fundamental choice here between denoting Q-types with descriptors or modifier bits. Let's explore using modifier bits as much as possible. Or do you want to put that off for later? For flattenable fields, it's a choice of Q-descriptor vs. ACC_VALUE modifier. > 2. Array descriptors: propose - yes > Remi: not needed - at array creation you know the element type I like Remi's suggestion, because I'm trying to get us to a uniform non-use of Q-descriptors. For flattenable arrays, it's a choice of Q-descriptor or dynamic "flattened" modifier on array. Not sure what the following is arguing for, but I'll jump in anyway! > Propsal: uniformity > - confusing to explain inconsistency Agree. Why do arrays need an extra flattening decoration? Isn't this just one more thing that I might get wrong or might change after javac-time? > - javac already knows the information and has done the work, why slow down? It knows this, but only at compile time. Java can change classes later. > - safety - kind constraints - could be checked by verifier If we do kind constraints, we should do them thoroughly, not "just for arrays". This suggests a bytecode to impose kind-constraints: kindcast [Q|R] And one to test them: iskind [Q|R]. The two could be coded with one code point, I suppose. Or, overload them onto checkcast and instanceof, with funny CP entries, Utf8["Q"], etc. Hmm; just make those be API points, for now. The JIT can intrinsify them. It's not clear which bytecodes would benefit from kind-constraints, though. > 3. Method descriptors : propose - no > 1) No indication of receiver type > 2) method descriptor parameters/return type - all use LTypes > Propose: do NOT support QTypes in Method Descriptors > challenge: descriptor mismatches based on migration > - for client inheritance/overriding, caller/callee matching Yes, good! (Pushing away those Q-descriptors, pushing, pushing, ? gone!) > Resolved Questions for LWorld1 > 1. Q: support other superclasses? > A: QType has no subclasses > A. for now - QType has only jlO as superclass, may be extended in future > (see if any that would break any optimizations) I wish we wouldn't perpetuate this use of jl.O; it doesn't work for either interfaces or value types. Suggestion: Consider allowing or requiring NULL (CP ref #0) as superclass of value types. And also of at least some interfaces. (And an interface with jlO as super is re-interpreted as having I$Object as one of its extends??) > > 2. Q:acmp behavior options: > a) failing: return false <- propose for try 1 (agree) > b) throw exception > c) field-equality using ucmp as "substitutable" - field-wise comparison > general bit equality including floating point > may need to recurse on values buffereed c is too expensive; let the user ask for it. b is also expensive in that it's harder to optimize in the presence of LIFE. and (my pet idea, just for the record): d) indeterminate (when given two identical values) Both a and d can be optimized in the presence of LIFE. d is marginally easier to optimize, while a is more deterministic BTW, Guy Steele notes that a is easy to justify by claiming that acmp really is a reference comparison, and that when either operand is a value, it is converted to a new reference by some sort of boxing. Since a new reference always compares distinct from a pre-existing one, then you get the described behavior: If either operand is a value, you get a false reference comparison. In the same vein, d can be justified by claiming that the boxing happens as above, *except* the JVM is allowed (but not required) to produce equivalent references for values it can prove to be equivalent. > A: LWorld1: if >= one operand is a QType: if_acmpeq -> false, if_acmpne -> true > A: null handling: as if acmp vs. NULL > if operand is a QType: ifnull -> false, ifnonnull->true > > 3. Q: Do we need to know if an LType is an old L-Type or a new LType? > A: be on the lookout - we have not yet identified any cases > - note: If we do, we have ClassFileVersion > > 4. Q: Any issues with argument passing/argument return handling if we have > runtime mismatches of kind? > A: If we have kind constraints then we should not get runtime mismatches > A: advanced error handling - check implications if verification is skipped I think we need to do some more work on this one. A classic example is a descriptor LFoo; which refers to a class Foo that is never loaded. Can you work with methods that mention Foo as an argument or return type? Sure you can: Just pass null always. Now suppose that Foo is a value-class, and it gets loaded an hour after the JIT has processed all the hot code including methods mentioning Foo. Now what do I do with all the nulls floating around in my system? And am I forbidden to pass Foo operands by value? > > 5. Q: Do we need a new carrier type? > A: TBD - so far requirement not identified. The premise behind L-world is that the L-carrier type is flexible enough to be extended to a U-type (and carry Q-values as well as R-values). > 6. Q: What does it mean for LObject to be more like an interface? > A: TBD (TODO - ask Dan?) > A1. disallow adding fields > A2. superclass of references and value types (with modified methods) (See above.) > > 7. Q: Do we need a fast way to distinguish an LType from a QType? in the JVM itself? > A. Initial prototype can work with instancKlass vs. valueKlass > Later - look at potential optimizations I think it's pretty clear that we need fast dynamic tests for all of the following: 1. a frame-local Q-value, for areturn (pass up to caller) 2. a thread-local Q-value, for aastore and putfield (convert to heap-buffer) 3. a Q-value (vs. an R-value), for acmp (short-circuit to false) The third one should also be a user-callable API (like isValue, above). > 8. Q: What does updated Object.hashcode do? > - field equality based hashcode > - assume cache in header optimization ? > A: Dan: call hashcode or identity hashcode (throw) - performance tradeoff The default is to call System.identityHashCode (or an internal equivalent). We can continue with that. And System.identityHashCode should probably throw, although it could also return the substitutability hash code. The performance should be OK since identityHashCode is a slow-path call, and we know we will have a fast test for Q-ness (because of acmp). > 9. Q: Do we need a fast way for Java to determine ValueType (isValue call?) > A: Frederic proposed: e.g. give all value types a common super interface > e.g. ValueMarker > - verifier or class file format checking at class loading > - ensure that this can't be a superinterface if not a value type > - this is probably temporary, but useful Simpler to have a value-bit on the class metadata block, and then do isValue() := this.$class_metadata.isValue_bit. (That's pseudo-code, of course. No real code on this alias.) Much faster than an interface check. > 10. Q: Migration QType->LType support? > Customers will try this > A: Need to ensure we catch failures > challenges: > instance creation: > value type - must have a private constructor - so new will fail IAE > - except for nestmates (dynamically added which are not same compilation) > - except Reflection.setAccessible > > 11. Q: Circularity handling for Field types? > A: today we get StackOverflowError - hotspot folks propose we leave that alone > to avoid complexity and costly testing OK for a prototype, but we need a proper ClassCircularityError eventually. We should repurpose the mechanism we use for today's ClassCircularityError to detect the additional kinds of circularity; they are all of the same kind. Principle stolen from C++: Having a flattened field is much like having a superclass, and vice versa. > 12. Q: Class.forName() and internal loaded class cache > A: NO L vs. Q type naming here, there is no ambiguity > > Unresolved Questions for LWorld1 > 1. Q: Reflection requirements/plans? For now, do the messy contextual isValue query points. And ask Brian for help designing a CrassType API, anticipating more messy contextual query points when templates arrive. > > 2. Q. What can the verifier check, what do we want to check later to avoid early class loading? > A. Expect to create kind constraints, if we have bytecodes that require a value type > which would be checked when we load that value type and throw ICCE It's not clear to me where we need kind constraints: They seem like a solution in search of a problem. But maybe I'm just missing the point. > A. Verifier can still check primitive vs. LType - leave those checks in (Yes; those are distinct "carrier types". The Q- and R-types are not distinct in that way, since they are carried by a common L-type in L-world.) > A. note: for loaded classes such as supertypes, value type fields or isAssignable checks, some classes are already loaded, so may not need a kind constraint > Q: Do we also need to do this for references - e.g. "new" bytecode? > Q: Is there a concern that by loading a class that uses the wrong bytecodes for someone else's kind - that could prevent a class from successfully loading? Yep. > =================== > Early experiment: > Add to JDK 11 (not MVT specific) > add dynamic checks for ValueBasedClasses > follow LWorld dynamic restrictions > add checks for ifacmp_eq/ne NOT followed by a call to .equals? > (todo: find Dan's corpus search results email) > todo: consider using JFR? This will be open in jdk11 > > Experimental steps: > 1. new repo - remove MVT parts, just include -XX:+EnableValhalla tests (already split) > > 2. LangTools: > 2.1 test bytecode generator > - generate LWorld bytecodes (from simp > > 2.2 Javac: > __ByValue for class declaration (for simplified JVMS bytecode changes document) > request: > super as java.lang.Object > generate LWorld bytecodes > LWorld restrictions checking > - e.g. allow superinterfaces , allow value types in static fields > > 2.2 new static utility class for new bytecodes > isValue > isElementValue (for array) > isFlattened (for field or array) > ucmp > > 2.3 java.lang.Object methods > Using isValue - rewrite > > 2.4 real JVMS LWorld1 proposal - Dan > > 3. Runtime > a. write up simplified JVMS bytecode changes document (not verifier yet) > b. interpreter bytecode changes - see list above > c. migrate VVT tests to LVT1 > d. verifier - > propose JVMS verifier changes > propose kind constraint handling - to not add any additional > class loading > - note: value type fields, supertypes and isAssignable checks already perform loading > e. method handle support > > 4. JIT: > a. bytecodes > b. migrate VVT tests to LVT1 > c. adaptor generation > d. optimizations > > > 5. Core libraries: > a. methodhandle support - e.g. changes to LambdaForms > - (simplified from MVT with no __Value) > - no boxing > b. Reflection/new reflection? > TODO - what are the requirements here? > > 6. Migration testing: > Try VBC -> value type -- run tests and see what breaks (This is a very good work-list. I can't think of anything else to add to it at the moment.) From john.r.rose at oracle.com Thu Jan 4 02:43:59 2018 From: john.r.rose at oracle.com (John Rose) Date: Wed, 3 Jan 2018 18:43:59 -0800 Subject: LWorld1 initial phase proposal In-Reply-To: <34237B64-8A3D-44E5-A16A-8AE17FD1DC6F@oracle.com> References: <34237B64-8A3D-44E5-A16A-8AE17FD1DC6F@oracle.com> Message-ID: On Jan 3, 2018, at 6:40 PM, John Rose wrote: > > (In L-world, a U-type appears to the JVM as an L-type, often an actual > L-descriptor like "Lpkg.Foo;". The term U-type refers to a new way of > using of L-type values. Likewise, R-types also appear to the JVM as > L-types, but the term R-type refers to the legacy usage of legacy L-types. > A Q-type enables a new way of passing or storing operands by value > instead of by reference. In L-world, there are mainly just L-types and > L-descriptors. If we need actual Q-descriptors in the JVM, it is only > because sometimes there is no better way to encode the directive to > use a Q-type, that is pass or store an operand only by value.) (Oh, boy, I think I made a hash of this paragraph. The point is that there are lots of types running around and only a few descriptors.) From karen.kinnear at oracle.com Fri Jan 5 15:04:02 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Fri, 5 Jan 2018 10:04:02 -0500 Subject: Valhalla EG meetings resuming Jan 17 Message-ID: Biweekly Valhalla EG meetings resuming Jan 17, 2018 at 9am PT/ noon ET/ 17 Dublin, 18 Grenoble/Stockholm, 20 St Petersburg https://oracle.zoom.us/j/5249803466 audio only: +1 646 558 8656 or +1 669 900 6833 US Toll Please look at the Nestmates JVMS Change to selection email - we have some decisions to make. thanks, Karen From daniel.smith at oracle.com Tue Jan 9 22:27:37 2018 From: daniel.smith at oracle.com (Dan Smith) Date: Tue, 9 Jan 2018 15:27:37 -0700 Subject: Nestmates JVMS changes to selection and behavior implications In-Reply-To: References: <82E5D22B-C4DE-4C60-9AAA-62291CD9FDC2@oracle.com> Message-ID: <49606948-D429-42AC-8F3C-385B62F94DC7@oracle.com> > On Jan 3, 2018, at 9:16 AM, Karen Kinnear wrote: > > In studying the details of these changes, there are three sets of changes that change the behavior of invokeinterface. > Invokeinterface is allowed to resolve to a private member of the reference class, and the resolved method becomes the selected method. > The removal of the invokeinterface selection runtime access check, allows selection of a package-private or protected method, since they can override a public method from the resolution step. > The combining of the invokeinterface selection and invokevirtual selection (and equivalent preparation modifications) add the concept of "overrides" to the invokeinterface selection lookup steps. Note that private methods never override and are never overridden. The consequence of this change is that the invokeinterface selection lookup will now SKIP private methods as invokevirtual does, where before it would find the private method and throw an IllegalAccessError. > I am proposing that for nestmates we do not need the second change, and that we could continue to have invokeinterface and invokevirtual selection be aligned, while keeping the restriction that if the selection lookup procedure selects a method that is not public, invokeinterface throws an IllegalAccessError. (This would continue to allow resolution to a private member, select the same private member). This prevents adding additional cases in which the caller is able to invoke a method it can not directly access. This is a good change; see details below. Eventually, I would like to entirely get rid of this rule, but to do so we need a more comprehensive solution to JDK-8021581. > For the 3rd change we have two ways to approach this. We can leave the existing invokeinterface selection/preparation alone, find private methods in the receiver and its superclasses, and throw an IllegalAccessError, or we can make the proposed changes and skip private methods as we do for invokevirtual (because of the term "overrides") and potentially find a public method that overrides the resolved method. > I have not yet heard a concrete proposal to make changes here, and continue to think the best thing to do is merge the selection logic into one set of rules, as described in the nestmates spec document. (It is *possible* to achieve our goals through other means, but there are many pitfalls, and I haven't seen anyone arguing for a specific alternative. From where I sit, the current proposed approach seems best.) > Run-time Exceptions > Otherwise, if step 1 or step 2 of the lookup procedure selects a method that is not public, invokeinterface throws an IllegalAccessError. > This is the line that we believe could be restored, slightly modified to apply to the selection lookup procedure steps in 5.4.5, so that the only non-public selected method would be a private method which is the referenced method as well as the selected method. Concretely, we would undo this deletion, and instead modify the rule as follows: "Otherwise, if step 1 or step 2 of the lookup procedure selects a method that is ~~not public~~ **neither public nor private**, invokeinterface throws an IllegalAccessError." The motivation here is that invokeinterface is uniquely able to i) resolve to a public interface method, and ii) select a protected/package method of a superclass that would otherwise be inaccessible to the caller. The fact that the referenced method is in an interface is important, because anyone with the ability to extend a class can also declare a fresh superinterface that includes any method names+descriptors they're interested in. ?Dan From karen.kinnear at oracle.com Wed Jan 10 14:34:10 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Wed, 10 Jan 2018 09:34:10 -0500 Subject: Nestmates JVMS changes to selection and behavior implications In-Reply-To: <49606948-D429-42AC-8F3C-385B62F94DC7@oracle.com> References: <82E5D22B-C4DE-4C60-9AAA-62291CD9FDC2@oracle.com> <49606948-D429-42AC-8F3C-385B62F94DC7@oracle.com> Message-ID: <9559C4EC-19B5-4F72-977F-6F3819F597ED@oracle.com> David Holmes and I are good with Dan?s proposal here. thanks, Karen > On Jan 9, 2018, at 5:27 PM, Dan Smith wrote: > >> On Jan 3, 2018, at 9:16 AM, Karen Kinnear > wrote: >> >> In studying the details of these changes, there are three sets of changes that change the behavior of invokeinterface. >> Invokeinterface is allowed to resolve to a private member of the reference class, and the resolved method becomes the selected method. >> The removal of the invokeinterface selection runtime access check, allows selection of a package-private or protected method, since they can override a public method from the resolution step. >> The combining of the invokeinterface selection and invokevirtual selection (and equivalent preparation modifications) add the concept of "overrides" to the invokeinterface selection lookup steps. Note that private methods never override and are never overridden. The consequence of this change is that the invokeinterface selection lookup will now SKIP private methods as invokevirtual does, where before it would find the private method and throw an IllegalAccessError. >> I am proposing that for nestmates we do not need the second change, and that we could continue to have invokeinterface and invokevirtual selection be aligned, while keeping the restriction that if the selection lookup procedure selects a method that is not public, invokeinterface throws an IllegalAccessError. (This would continue to allow resolution to a private member, select the same private member). This prevents adding additional cases in which the caller is able to invoke a method it can not directly access. > > This is a good change; see details below. > > Eventually, I would like to entirely get rid of this rule, but to do so we need a more comprehensive solution to JDK-8021581. > >> For the 3rd change we have two ways to approach this. We can leave the existing invokeinterface selection/preparation alone, find private methods in the receiver and its superclasses, and throw an IllegalAccessError, or we can make the proposed changes and skip private methods as we do for invokevirtual (because of the term "overrides") and potentially find a public method that overrides the resolved method. >> > > I have not yet heard a concrete proposal to make changes here, and continue to think the best thing to do is merge the selection logic into one set of rules, as described in the nestmates spec document. (It is *possible* to achieve our goals through other means, but there are many pitfalls, and I haven't seen anyone arguing for a specific alternative. From where I sit, the current proposed approach seems best.) >> Run-time Exceptions >> Otherwise, if step 1 or step 2 of the lookup procedure selects a method that is not public, invokeinterface throws an IllegalAccessError. >> This is the line that we believe could be restored, slightly modified to apply to the selection lookup procedure steps in 5.4.5, so that the only non-public selected method would be a private method which is the referenced method as well as the selected method. > > Concretely, we would undo this deletion, and instead modify the rule as follows: > > "Otherwise, if step 1 or step 2 of the lookup procedure selects a method that is ~~not public~~ **neither public nor private**, invokeinterface throws an IllegalAccessError." > > The motivation here is that invokeinterface is uniquely able to i) resolve to a public interface method, and ii) select a protected/package method of a superclass that would otherwise be inaccessible to the caller. The fact that the referenced method is in an interface is important, because anyone with the ability to extend a class can also declare a fresh superinterface that includes any method names+descriptors they're interested in. > > ?Dan From karen.kinnear at oracle.com Mon Jan 15 15:43:07 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Mon, 15 Jan 2018 10:43:07 -0500 Subject: RSVP please: Re: Nestmates JVMS changes to selection and behavior implications In-Reply-To: <9559C4EC-19B5-4F72-977F-6F3819F597ED@oracle.com> References: <82E5D22B-C4DE-4C60-9AAA-62291CD9FDC2@oracle.com> <49606948-D429-42AC-8F3C-385B62F94DC7@oracle.com> <9559C4EC-19B5-4F72-977F-6F3819F597ED@oracle.com> Message-ID: EG members - could you possibly review this change and reply in an email? We believe this is the last step to finalizing the nestmates JVMS changes. thanks, Karen p.s. changes are relative to: http://cr.openjdk.java.net/~dlsmith/nestmates.html > On Jan 10, 2018, at 9:34 AM, Karen Kinnear wrote: > > David Holmes and I are good with Dan?s proposal here. > > thanks, > Karen > >> On Jan 9, 2018, at 5:27 PM, Dan Smith > wrote: >> >>> On Jan 3, 2018, at 9:16 AM, Karen Kinnear > wrote: >>> >>> In studying the details of these changes, there are three sets of changes that change the behavior of invokeinterface. >>> Invokeinterface is allowed to resolve to a private member of the reference class, and the resolved method becomes the selected method. >>> The removal of the invokeinterface selection runtime access check, allows selection of a package-private or protected method, since they can override a public method from the resolution step. >>> The combining of the invokeinterface selection and invokevirtual selection (and equivalent preparation modifications) add the concept of "overrides" to the invokeinterface selection lookup steps. Note that private methods never override and are never overridden. The consequence of this change is that the invokeinterface selection lookup will now SKIP private methods as invokevirtual does, where before it would find the private method and throw an IllegalAccessError. >>> I am proposing that for nestmates we do not need the second change, and that we could continue to have invokeinterface and invokevirtual selection be aligned, while keeping the restriction that if the selection lookup procedure selects a method that is not public, invokeinterface throws an IllegalAccessError. (This would continue to allow resolution to a private member, select the same private member). This prevents adding additional cases in which the caller is able to invoke a method it can not directly access. >> >> This is a good change; see details below. >> >> Eventually, I would like to entirely get rid of this rule, but to do so we need a more comprehensive solution to JDK-8021581. >> >>> For the 3rd change we have two ways to approach this. We can leave the existing invokeinterface selection/preparation alone, find private methods in the receiver and its superclasses, and throw an IllegalAccessError, or we can make the proposed changes and skip private methods as we do for invokevirtual (because of the term "overrides") and potentially find a public method that overrides the resolved method. >>> >> >> I have not yet heard a concrete proposal to make changes here, and continue to think the best thing to do is merge the selection logic into one set of rules, as described in the nestmates spec document. (It is *possible* to achieve our goals through other means, but there are many pitfalls, and I haven't seen anyone arguing for a specific alternative. From where I sit, the current proposed approach seems best.) >>> Run-time Exceptions >>> Otherwise, if step 1 or step 2 of the lookup procedure selects a method that is not public, invokeinterface throws an IllegalAccessError. >>> This is the line that we believe could be restored, slightly modified to apply to the selection lookup procedure steps in 5.4.5, so that the only non-public selected method would be a private method which is the referenced method as well as the selected method. >> >> Concretely, we would undo this deletion, and instead modify the rule as follows: >> >> "Otherwise, if step 1 or step 2 of the lookup procedure selects a method that is ~~not public~~ **neither public nor private**, invokeinterface throws an IllegalAccessError." >> >> The motivation here is that invokeinterface is uniquely able to i) resolve to a public interface method, and ii) select a protected/package method of a superclass that would otherwise be inaccessible to the caller. The fact that the referenced method is in an interface is important, because anyone with the ability to extend a class can also declare a fresh superinterface that includes any method names+descriptors they're interested in. >> >> ?Dan > From karen.kinnear at oracle.com Mon Jan 15 15:48:54 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Mon, 15 Jan 2018 10:48:54 -0500 Subject: Valhalla EG meetings resuming Jan 31 In-Reply-To: References: Message-ID: <4CE4DBA0-2EBF-4181-8915-9420695C69CD@oracle.com> Apologies, we have a senior VP meeting conflict this week. Our next Valhalla EG meeting will be January 31 same time. Please RSVP in email relative to the Nestmates JVMS change so we can finalize the nestmates JVMS changes. Here is the email thread: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2018-January/000502.html thanks, Karen > On Jan 5, 2018, at 10:04 AM, Karen Kinnear wrote: > > Biweekly Valhalla EG meetings resuming Jan 17, 2018 > at 9am PT/ noon ET/ 17 Dublin, 18 Grenoble/Stockholm, 20 St Petersburg > > https://oracle.zoom.us/j/5249803466 > > audio only: +1 646 558 8656 or +1 669 900 6833 US Toll > > Please look at the Nestmates JVMS Change to selection email - we have some decisions > to make. > > thanks, > Karen From dl at cs.oswego.edu Mon Jan 15 23:55:51 2018 From: dl at cs.oswego.edu (Doug Lea) Date: Mon, 15 Jan 2018 18:55:51 -0500 Subject: RSVP please: Re: Nestmates JVMS changes to selection and behavior implications In-Reply-To: References: <82E5D22B-C4DE-4C60-9AAA-62291CD9FDC2@oracle.com> <49606948-D429-42AC-8F3C-385B62F94DC7@oracle.com> <9559C4EC-19B5-4F72-977F-6F3819F597ED@oracle.com> Message-ID: <00df52cf-8502-66f0-6bc4-0ebce3ebc71b@cs.oswego.edu> On 01/15/2018 10:43 AM, Karen Kinnear wrote: > EG members - could you possibly review this change and reply in an email? > We believe this is the last step to finalizing the nestmates JVMS changes. Seems OK to me. My only meta-thought is that nestmates might now hold the record for the most subtle details that needed addressing per unit of benefit to the platform... -Doug > > thanks, > Karen > > p.s. changes are relative > to:?http://cr.openjdk.java.net/~dlsmith/nestmates.html > >> On Jan 10, 2018, at 9:34 AM, Karen Kinnear > > wrote: >> >> David Holmes and I are good with Dan?s proposal here. >> >> thanks, >> Karen >> >>> On Jan 9, 2018, at 5:27 PM, Dan Smith >> > wrote: >>> >>>> On Jan 3, 2018, at 9:16 AM, Karen Kinnear >>> > wrote: >>>> >>>> In studying the details of these changes, there are three sets of >>>> changes that change the behavior of invokeinterface. >>>> >>>> 1. Invokeinterface is allowed to resolve to a private member of the >>>> reference class, and the resolved method becomes the selected >>>> method.? >>>> 2. The removal of the invokeinterface selection runtime access >>>> check, allows selection of a package-private or protected >>>> method, since they can override a public method from the >>>> resolution step. >>>> 3. The combining of the invokeinterface selection and invokevirtual >>>> selection (and equivalent preparation modifications) add the >>>> concept of "overrides" to the invokeinterface selection lookup >>>> steps. Note that private methods never override and are never >>>> overridden. The consequence of this change is that the >>>> invokeinterface selection lookup will now SKIP private methods >>>> as invokevirtual does, where before it would find the private >>>> method and throw an IllegalAccessError. >>>> >>>> I am proposing that for nestmates we do not need the second change, >>>> and that we could continue to have invokeinterface and invokevirtual >>>> selection be aligned, while keeping the restriction that if the >>>> selection lookup procedure selects a method that is not public, >>>> invokeinterface throws an IllegalAccessError. (This would continue >>>> to allow resolution to a private member, select the same private >>>> member). This prevents adding additional cases in which the caller >>>> is able to invoke a method it can not directly access. >>>> >>> >>> This is a good change; see details below. >>> >>> Eventually, I would like to entirely get rid of this rule, but to do >>> so we need a more comprehensive solution to JDK-8021581. >>> >>>> For the 3rd change we have two ways to approach this. We can leave >>>> the existing invokeinterface selection/preparation alone, find >>>> private methods in the receiver and its superclasses, and throw an >>>> IllegalAccessError, or we can make the proposed changes and skip >>>> private methods as we do for invokevirtual (because of the term >>>> "overrides") and potentially find a public method that overrides the >>>> resolved method. >>>> >>>> >>> >>> I have not yet heard a concrete proposal to make changes here, and >>> continue to think the best thing to do is merge the selection logic >>> into one set of rules, as described in the nestmates spec document. >>> (It is *possible* to achieve our goals through other means, but there >>> are many pitfalls, and I haven't seen anyone arguing for a specific >>> alternative. From where I sit, the current proposed approach seems best.) >>>> >>>> >>>> Run-time Exceptions >>>> >>>> Otherwise, if step 1 or step 2 of the lookup procedure selects a >>>> method that is not?|public|,?|invokeinterface|?throws >>>> an?|IllegalAccessError|. >>>> >>>> This is the line that we believe could be restored, >>>> slightly?modified to apply to the selection lookup procedure steps >>>> in 5.4.5, so that the only non-public selected method would be a >>>> private method which is the referenced method as well as the >>>> selected method. >>>> >>> >>> Concretely, we would undo this deletion, and instead modify the rule >>> as follows: >>> >>> "Otherwise, if step 1 or step 2 of the lookup procedure selects a >>> method that is ~~not public~~ **neither public nor private**, >>> invokeinterface throws an IllegalAccessError." >>> >>> The motivation here is that invokeinterface is uniquely able to i) >>> resolve to a public interface method, and ii) select a >>> protected/package method of a superclass that would otherwise be >>> inaccessible to the caller. The fact that the referenced method is in >>> an interface is important, because anyone with the ability to extend >>> a class can also declare a fresh superinterface that includes any >>> method names+descriptors they're interested in. >>> >>> ?Dan >> > From daniel.smith at oracle.com Fri Jan 19 00:14:19 2018 From: daniel.smith at oracle.com (Dan Smith) Date: Thu, 18 Jan 2018 17:14:19 -0700 Subject: Final CONSTANT_Dynamic spec Message-ID: A proposed final spec for CONSTANT_Dynamic is here: http://cr.openjdk.java.net/~dlsmith/constant-dynamic.html There are two significant changes: 5.4.3: Expanded the rule about concurrent resolution to account for nested resolution in a single thread 5.4.3.6: Added a resolution-time rule for detecting cycles in static arguments, with some additional discussion about cycles These are new rules for complex behavior, so could use some extra scrutiny. They are careful to distinguish between cycles in static arguments (explicitly checked by 5.4.3.6) and cycles in bootstrap method calls (which may overflow, or may eventually terminate). Some other, minor changes: 4.7.23: Removed a comment about the "typical" form of the bootstrap method handle, which some people read as both normative and noncommittal 5.4: Added a note that link-time errors are always instances of Error 5.4: Deleted an old line about OutOfMemoryErrors, explaining that it confusingly singles out one of many possible errors 5.4.3: Added a note that resolution errors are always instances of Error 5.4.3.6: Added a note about potential optimizations, such as skipping allocation of an argument array ?Dan From john.r.rose at oracle.com Fri Jan 19 01:43:59 2018 From: john.r.rose at oracle.com (John Rose) Date: Thu, 18 Jan 2018 17:43:59 -0800 Subject: Final CONSTANT_Dynamic spec In-Reply-To: References: Message-ID: <01D76829-AACD-456E-90CB-A83664A6E85C@oracle.com> On Jan 18, 2018, at 4:14 PM, Dan Smith wrote: > > A proposed final spec for CONSTANT_Dynamic is here: > > http://cr.openjdk.java.net/~dlsmith/constant-dynamic.html I gave it one more read. It is excellent. Let's do it. ? John From paul.sandoz at oracle.com Fri Jan 19 02:02:03 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 18 Jan 2018 18:02:03 -0800 Subject: Final CONSTANT_Dynamic spec In-Reply-To: <01D76829-AACD-456E-90CB-A83664A6E85C@oracle.com> References: <01D76829-AACD-456E-90CB-A83664A6E85C@oracle.com> Message-ID: <0CF93301-87A4-4E55-AC9B-B8C87ED730B3@oracle.com> > On 18 Jan 2018, at 17:43, John Rose wrote: > > On Jan 18, 2018, at 4:14 PM, Dan Smith wrote: >> >> A proposed final spec for CONSTANT_Dynamic is here: >> >> http://cr.openjdk.java.net/~dlsmith/constant-dynamic.html > > > I gave it one more read. It is excellent. Let's do it. ? John +1 thanks Dan, Paul. From karen.kinnear at oracle.com Fri Jan 19 17:36:07 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Fri, 19 Jan 2018 12:36:07 -0500 Subject: Final CONSTANT_Dynamic spec In-Reply-To: References: Message-ID: Looks great Dan - much clearer! thanks, Karen > On Jan 18, 2018, at 7:14 PM, Dan Smith wrote: > > A proposed final spec for CONSTANT_Dynamic is here: > > http://cr.openjdk.java.net/~dlsmith/constant-dynamic.html > > There are two significant changes: > > 5.4.3: Expanded the rule about concurrent resolution to account for nested resolution in a single thread > > 5.4.3.6: Added a resolution-time rule for detecting cycles in static arguments, with some additional discussion about cycles > > These are new rules for complex behavior, so could use some extra scrutiny. They are careful to distinguish between cycles in static arguments (explicitly checked by 5.4.3.6) and cycles in bootstrap method calls (which may overflow, or may eventually terminate). > > Some other, minor changes: > > 4.7.23: Removed a comment about the "typical" form of the bootstrap method handle, which some people read as both normative and noncommittal > > 5.4: Added a note that link-time errors are always instances of Error > > 5.4: Deleted an old line about OutOfMemoryErrors, explaining that it confusingly singles out one of many possible errors > > 5.4.3: Added a note that resolution errors are always instances of Error > > 5.4.3.6: Added a note about potential optimizations, such as skipping allocation of an argument array > > ?Dan From lois.foltan at oracle.com Fri Jan 19 18:02:20 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Fri, 19 Jan 2018 13:02:20 -0500 Subject: Final CONSTANT_Dynamic spec In-Reply-To: References: Message-ID: On 1/19/2018 12:36 PM, Karen Kinnear wrote: > Looks great Dan - much clearer! +1 for me too, I've read it through.? Thanks Dan! Lois > > thanks, > Karen > >> On Jan 18, 2018, at 7:14 PM, Dan Smith wrote: >> >> A proposed final spec for CONSTANT_Dynamic is here: >> >> http://cr.openjdk.java.net/~dlsmith/constant-dynamic.html >> >> There are two significant changes: >> >> 5.4.3: Expanded the rule about concurrent resolution to account for nested resolution in a single thread >> >> 5.4.3.6: Added a resolution-time rule for detecting cycles in static arguments, with some additional discussion about cycles >> >> These are new rules for complex behavior, so could use some extra scrutiny. They are careful to distinguish between cycles in static arguments (explicitly checked by 5.4.3.6) and cycles in bootstrap method calls (which may overflow, or may eventually terminate). >> >> Some other, minor changes: >> >> 4.7.23: Removed a comment about the "typical" form of the bootstrap method handle, which some people read as both normative and noncommittal >> >> 5.4: Added a note that link-time errors are always instances of Error >> >> 5.4: Deleted an old line about OutOfMemoryErrors, explaining that it confusingly singles out one of many possible errors >> >> 5.4.3: Added a note that resolution errors are always instances of Error >> >> 5.4.3.6: Added a note about potential optimizations, such as skipping allocation of an argument array >> >> ?Dan From daniel.smith at oracle.com Fri Jan 19 20:11:27 2018 From: daniel.smith at oracle.com (Dan Smith) Date: Fri, 19 Jan 2018 13:11:27 -0700 Subject: A "RecursiveConstants" attribute In-Reply-To: <131B1F8D-F49C-47F8-BE7C-A660CBE1FCB5@oracle.com> References: <131B1F8D-F49C-47F8-BE7C-A660CBE1FCB5@oracle.com> Message-ID: <2CEA5EE3-6F51-4806-9CA5-2BAFCF0620E8@oracle.com> We had the following discussion in the Dec 20 meeting: > On Jan 3, 2018, at 9:11 AM, Karen Kinnear wrote: > > Dan S: Cycle handling > Array types can have cycles > - rather than relying on ordering - perhaps tell tree depth - e.g. component type & dimensions - all in the CP > John: perhaps modify to include the depth indicators in an attribute (separate proof of acyclicity) > Remi: but if you insert a condy ? > John: would need to discard the attribute > Dan H: concern about a constant array for each dimension > ? maybe some adhoc compression techniques > Remi: why do you want to be cycle free? > Dan S: hard to verify if types cyclical? > Remi: why do you need to check in the verifier? > John: maybe a structural constraint? > Remi: if value - ok, but not for an array of values? > e.g. Object[] with first entity as itself > Dan: want array type convertible to finite string, benefit of attribute not tied to construct is it can evolve > > Dan H: Concerns about attributes - will this be as hard to maintain for e.g. class redefiners/bytecode generators as stackmaptables? > No - you have to keep the CP indices in sync, but stackmaptables required abstract interpretation - transformer > did not have the type information, and the compression mechanism was harder to implement and these > would just require bumping indices. To flesh the attribute idea out, here's a possible spec. Mentions CONSTANT_Dynamic for now, and would be modified in the future to name other constant types that allow cycles. Is this a happy solution to the problems raised in previous discussions about constant cycles? Any new problems introduced? ?Dan 4.7.28 The `RecursiveConstants` Attribute (new) ----------------------------------------- The `RecursiveConstants` attribute is a variable-length attribute in the `attributes` table of a `ClassFile` structure ([4.1]). The `RecursiveConstants` attribute facilitates checks that a `CONSTANT_Dynamic` entry in the `constant_pool` table does not refer, directly or indirectly, to itself ([4.4.13]). ~~~~ RecursiveConstants_attribute { u2 attribute_name_index; u4 attribute_length; u2 num_recursive_constants; { u2 constant_index; u2 max_recursion_depth; } recursive_constants[num_recursive_constants]; } ~~~~ The items in the `RecursiveConstants_attribute` structure are as follows: `attribute_name_index` : The value of the `attribute_name_index` item must be a valid index into the `constant_pool` table. The `constant_pool` entry at that index must be a `CONSTANT_Utf8_info` structure ([4.4.7]) representing the string `"RecursiveConstants"`. `attribute_length` : The value of the `attribute_length` item indicates the length of the attribute, excluding the initial six bytes. `num_recursive_constants` : The value of the `num_recursive_constants` item gives the number of entries in the `recursive_constants` array. `recursive_constants` : Each entry in the `recursive_constants` table contains the following two items: `constant_index` : The value of `constant_index` must be a valid index into the `constant_pool` table. The `constant_pool` entry at that index must be a `CONSTANT_Dynamic` structure ([4.4.13]). There may be at most one entry in the `recursive_constants` table for each `CONSTANT_Dynamic` structure in the `constant_pool`. `max_recursion_depth` : The value of `max_recursion_depth` designates a _maximum recursion depth_ for the referenced constant. 4.4.13 The `CONSTANT_Dynamic_info` Structure (modified) -------------------------------------------- The `CONSTANT_Dynamic_info` structure is used to describe a _dynamically-computed constant_, an arbitrary primitive or reference value produced by invocation of a _bootstrap method_ ([4.7.23]). The structure specifies i) a bootstrap method handle with an optional sequence of _static arguments_ and ii) a referenced name and type. ~~~~ CONSTANT_Dynamic_info { u1 tag; u2 bootstrap_method_attr_index; u2 name_and_type_index; } ~~~~ The items of the `CONSTANT_Dynamic_info` structure are as follows: `tag` : The `tag` item of the `CONSTANT_Dynamic_info` structure has the value `CONSTANT_Dynamic` (17). `bootstrap_method_attr_index` : The value of the `bootstrap_method_attr_index` item must be a valid index into the `bootstrap_methods` array of the bootstrap method table ([4.7.23]) of this class file. **The _maximum recursion depth_ of a `CONSTANT_Dynamic` structure is the value given by `max_recursion_depth` of an entry referencing the `CONSTANT_Dynamic` structure in the `recursive_constants` table of the `RecursiveConstants` attribute ([4.7.28]); or, if no such entry exists, 0. Let _d_ be the maximum recursion depth of this `CONSTANT_Dynamic_info` structure. For each item in the `bootstrap_arguments` array of the `bootstrap_methods` entry referenced by `bootstrap_method_attr_index`, if the item references a `CONSTANT_Dynamic_info` structure, the maximum recursion depth of that structure must be less than (and not equal to) _d_.** > **This check prevents a `CONSTANT_Dynamic_info` structure from referring to itself via one of its static arguments.** `name_and_type_index` : The value of the `name_and_type_index` item must be a valid index into the `constant_pool` table. The `constant_pool` entry at that index must be a `CONSTANT_NameAndType_info` structure ([4.4.6]) representing a name and field descriptor ([4.3.2]). From karen.kinnear at oracle.com Fri Jan 19 20:39:42 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Fri, 19 Jan 2018 15:39:42 -0500 Subject: A "RecursiveConstants" attribute In-Reply-To: <2CEA5EE3-6F51-4806-9CA5-2BAFCF0620E8@oracle.com> References: <131B1F8D-F49C-47F8-BE7C-A660CBE1FCB5@oracle.com> <2CEA5EE3-6F51-4806-9CA5-2BAFCF0620E8@oracle.com> Message-ID: Dan H, Thank you for the suggestion. I need to study this in more detail. Would it work for you if we finalize the latest Condy JVMS that Dan S sent out and look into recursion handling in more detail as part of a future cycle? http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2018-January/000505.html thanks, Karen > On Jan 19, 2018, at 3:11 PM, Dan Smith wrote: > > We had the following discussion in the Dec 20 meeting: > >> On Jan 3, 2018, at 9:11 AM, Karen Kinnear > wrote: >> >> Dan S: Cycle handling >> Array types can have cycles >> - rather than relying on ordering - perhaps tell tree depth - e.g. component type & dimensions - all in the CP >> John: perhaps modify to include the depth indicators in an attribute (separate proof of acyclicity) >> Remi: but if you insert a condy ? >> John: would need to discard the attribute >> Dan H: concern about a constant array for each dimension >> ? maybe some adhoc compression techniques >> Remi: why do you want to be cycle free? >> Dan S: hard to verify if types cyclical? >> Remi: why do you need to check in the verifier? >> John: maybe a structural constraint? >> Remi: if value - ok, but not for an array of values? >> e.g. Object[] with first entity as itself >> Dan: want array type convertible to finite string, benefit of attribute not tied to construct is it can evolve >> >> Dan H: Concerns about attributes - will this be as hard to maintain for e.g. class redefiners/bytecode generators as stackmaptables? >> No - you have to keep the CP indices in sync, but stackmaptables required abstract interpretation - transformer >> did not have the type information, and the compression mechanism was harder to implement and these >> would just require bumping indices. > > To flesh the attribute idea out, here's a possible spec. Mentions CONSTANT_Dynamic for now, and would be modified in the future to name other constant types that allow cycles. > > Is this a happy solution to the problems raised in previous discussions about constant cycles? Any new problems introduced? > > ?Dan > > > 4.7.28 The `RecursiveConstants` Attribute (new) > ----------------------------------------- > > The `RecursiveConstants` attribute is a variable-length attribute in the `attributes` table of a `ClassFile` structure ([4.1]). The `RecursiveConstants` attribute facilitates checks that a `CONSTANT_Dynamic` entry in the `constant_pool` table does not refer, directly or indirectly, to itself ([4.4.13]). > > ~~~~ > RecursiveConstants_attribute { > u2 attribute_name_index; > u4 attribute_length; > u2 num_recursive_constants; > { u2 constant_index; > u2 max_recursion_depth; > } recursive_constants[num_recursive_constants]; > } > ~~~~ > > The items in the `RecursiveConstants_attribute` structure are as follows: > > `attribute_name_index` > > : The value of the `attribute_name_index` item must be a valid index into the `constant_pool` table. The `constant_pool` entry at that index must be a `CONSTANT_Utf8_info` structure ([4.4.7]) representing the string `"RecursiveConstants"`. > > `attribute_length` > > : The value of the `attribute_length` item indicates the length of the attribute, excluding the initial six bytes. > > `num_recursive_constants` > > : The value of the `num_recursive_constants` item gives the number of entries in the `recursive_constants` array. > > `recursive_constants` > > : Each entry in the `recursive_constants` table contains the following two items: > > `constant_index` > > : The value of `constant_index` must be a valid index into the `constant_pool` table. The `constant_pool` entry at that index must be a `CONSTANT_Dynamic` structure ([4.4.13]). > > There may be at most one entry in the `recursive_constants` table for each `CONSTANT_Dynamic` structure in the `constant_pool`. > > `max_recursion_depth` > > : The value of `max_recursion_depth` designates a _maximum recursion depth_ for the referenced constant. > > > 4.4.13 The `CONSTANT_Dynamic_info` Structure (modified) > -------------------------------------------- > > The `CONSTANT_Dynamic_info` structure is used to describe a _dynamically-computed constant_, an arbitrary primitive or reference value produced by invocation of a _bootstrap method_ ([4.7.23]). The structure specifies i) a bootstrap method handle with an optional sequence of _static arguments_ and ii) a referenced name and type. > > ~~~~ > CONSTANT_Dynamic_info { > u1 tag; > u2 bootstrap_method_attr_index; > u2 name_and_type_index; > } > ~~~~ > > The items of the `CONSTANT_Dynamic_info` structure are as follows: > > `tag` > > : The `tag` item of the `CONSTANT_Dynamic_info` structure has the value > `CONSTANT_Dynamic` (17). > > `bootstrap_method_attr_index` > > : The value of the `bootstrap_method_attr_index` item must be a valid index into the `bootstrap_methods` array of the bootstrap method table ([4.7.23]) of this class file. > > **The _maximum recursion depth_ of a `CONSTANT_Dynamic` structure is the value given by `max_recursion_depth` of an entry referencing the `CONSTANT_Dynamic` structure in the `recursive_constants` table of the `RecursiveConstants` attribute ([4.7.28]); or, if no such entry exists, 0. Let _d_ be the maximum recursion depth of this `CONSTANT_Dynamic_info` structure. For each item in the `bootstrap_arguments` array of the `bootstrap_methods` entry referenced by `bootstrap_method_attr_index`, if the item references a `CONSTANT_Dynamic_info` structure, the maximum recursion depth of that structure must be less than (and not equal to) _d_.** > > > **This check prevents a `CONSTANT_Dynamic_info` structure from referring to itself via one of its static arguments.** > > `name_and_type_index` > > : The value of the `name_and_type_index` item must be a valid index into the `constant_pool` table. The `constant_pool` entry at that index must be a `CONSTANT_NameAndType_info` structure ([4.4.6]) representing a name and field descriptor ([4.3.2]). > From john.r.rose at oracle.com Fri Jan 19 21:24:10 2018 From: john.r.rose at oracle.com (John Rose) Date: Fri, 19 Jan 2018 13:24:10 -0800 Subject: A "RecursiveConstants" attribute In-Reply-To: <2CEA5EE3-6F51-4806-9CA5-2BAFCF0620E8@oracle.com> References: <131B1F8D-F49C-47F8-BE7C-A660CBE1FCB5@oracle.com> <2CEA5EE3-6F51-4806-9CA5-2BAFCF0620E8@oracle.com> Message-ID: On Jan 19, 2018, at 12:11 PM, Dan Smith wrote: > > To flesh the attribute idea out, here's a possible spec. Mentions CONSTANT_Dynamic for now, and would be modified in the future to name other constant types that allow cycles. > > Is this a happy solution to the problems raised in previous discussions about constant cycles? Any new problems introduced? Yes, this is about what I had in mind when I made the suggestion of an acyclicity proof. It reads well and looks like it would do the job. I like the touch of assigning zero to a missing condy; that gracefully covers the many non-recursive cases. Like Karen, I assume this is not an addendum to the final "minimal condy" spec you sent out. ? John From daniel.smith at oracle.com Fri Jan 19 21:40:39 2018 From: daniel.smith at oracle.com (Dan Smith) Date: Fri, 19 Jan 2018 14:40:39 -0700 Subject: A "RecursiveConstants" attribute In-Reply-To: References: <131B1F8D-F49C-47F8-BE7C-A660CBE1FCB5@oracle.com> <2CEA5EE3-6F51-4806-9CA5-2BAFCF0620E8@oracle.com> Message-ID: <2C2B2602-E1FB-4E77-9689-8F39BB1F787D@oracle.com> > On Jan 19, 2018, at 2:24 PM, John Rose wrote: > > On Jan 19, 2018, at 12:11 PM, Dan Smith wrote: >> >> To flesh the attribute idea out, here's a possible spec. Mentions CONSTANT_Dynamic for now, and would be modified in the future to name other constant types that allow cycles. >> >> Is this a happy solution to the problems raised in previous discussions about constant cycles? Any new problems introduced? > > Yes, this is about what I had in mind when I made the suggestion of an > acyclicity proof. It reads well and looks like it would do the job. I like > the touch of assigning zero to a missing condy; that gracefully covers > the many non-recursive cases. > > Like Karen, I assume this is not an addendum to the final "minimal condy" > spec you sent out. Nope, this isn't intended to be tied to condy. It's a separate effort. The followup question is: when should we try to deliver it? - SE 11: get it in when condy is introduced, avoid allowing any cyclical constant pools to escape into the wild - Sometime soon after 11: address compatibility by easing into it, perhaps following the path of StackMapTable (optional at first) - Next time we introduce a cyclical attribute: let condy be, don't disrupt anything until we need this check somewhere else The SE 11 option has the advantage of avoiding future historical baggage (e.g., otherwise, to support 55.0 class files, the resolution-time check must remain in place forever). It has the disadvantage that it's late in the game to be introducing new JVM attributes. ?Dan From john.r.rose at oracle.com Fri Jan 19 22:07:12 2018 From: john.r.rose at oracle.com (John Rose) Date: Fri, 19 Jan 2018 14:07:12 -0800 Subject: A "RecursiveConstants" attribute In-Reply-To: <2C2B2602-E1FB-4E77-9689-8F39BB1F787D@oracle.com> References: <131B1F8D-F49C-47F8-BE7C-A660CBE1FCB5@oracle.com> <2CEA5EE3-6F51-4806-9CA5-2BAFCF0620E8@oracle.com> <2C2B2602-E1FB-4E77-9689-8F39BB1F787D@oracle.com> Message-ID: On Jan 19, 2018, at 1:40 PM, Dan Smith wrote: > > The followup question is: when should we try to deliver it? > > - SE 11: get it in when condy is introduced, avoid allowing any cyclical constant pools to escape into the wild > - Sometime soon after 11: address compatibility by easing into it, perhaps following the path of StackMapTable (optional at first) > - Next time we introduce a cyclical attribute: let condy be, don't disrupt anything until we need this check somewhere else > > The SE 11 option has the advantage of avoiding future historical baggage (e.g., otherwise, to support 55.0 class files, the resolution-time check must remain in place forever). It has the disadvantage that it's late in the game to be introducing new JVM attributes. I don't mind fitting it in soon, as long as it doesn't slow the next increment of condy. The link-time circularity check is (I think) cheap enough to disregard as a downside. It can probably just stay in there as a belt-and-suspenders check. We can surely tweak the check if it ever shows up on Claes's performance tests. So the main downside of delaying it is the (small!) risk of cyclic CPs sneaking into the ecosystem, and then being rejected in the future. As you suggest, it could be rolled into a template classes design, where circularity checking is much more urgent. ? John From david.holmes at oracle.com Thu Jan 25 05:45:43 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 25 Jan 2018 15:45:43 +1000 Subject: API Update: 8193408: [Nestmates] Update Class.getNestMembers to allow for duplicates Message-ID: webrev: http://cr.openjdk.java.net/~dholmes/8193408/webrev/src/java.base/share/classes/java/lang/Class.java.cdiff.html (and inlined below) Further to earlier discussions: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2017-December/000464.html This is the proposed minor update to the specification for Class.getNestMembers() There are three changes: 1. Don't link to getNestHost when discussing nest membership validation and exceptions, because getNestHost() doesn't throw exceptions. 2. Note that duplicate nest member entries are permitted (including the nest host) and need not be removed. 3. Document that this implementation does not remove duplicates. Thanks, David --- old/src/java.base/share/classes/java/lang/Class.java 2018-01-25 00:26:45.139747770 -0500 +++ new/src/java.base/share/classes/java/lang/Class.java 2018-01-25 00:26:43.115631664 -0500 @@ -3904,8 +3904,15 @@ * of its nest. The nest host will always be the zeroeth element. * *

Each listed nest member must be validated by checking its own - * declared {@linkplain #getNestHost() nest host}. Any exceptions that occur - * as part of this process will be thrown. + * declared nest host. Any exceptions that occur as part of this process + * will be thrown. + * + *

The list of nest members in the classfile is permitted to + * contain duplicates, or to explicitly include the nest host. It is not + * required that an implementation of this method removes these duplicates. + * + * @implNote This implementation does not remove duplicate nest members if they + * are present. * * @return an array of all classes and interfaces in the same nest as * this class From maurizio.cimadamore at oracle.com Thu Jan 25 14:36:13 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 25 Jan 2018 14:36:13 +0000 Subject: API Update: 8193408: [Nestmates] Update Class.getNestMembers to allow for duplicates In-Reply-To: References: Message-ID: Looks good Maurizio On 25/01/18 05:45, David Holmes wrote: > webrev: > http://cr.openjdk.java.net/~dholmes/8193408/webrev/src/java.base/share/classes/java/lang/Class.java.cdiff.html > (and inlined below) > > Further to earlier discussions: > > http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2017-December/000464.html > > > This is the proposed minor update to the specification for > Class.getNestMembers() > > There are three changes: > > 1. Don't link to getNestHost when discussing nest membership > validation and exceptions, because getNestHost() doesn't throw > exceptions. > > 2. Note that duplicate nest member entries are permitted (including > the nest host) and need not be removed. > > 3. Document that this implementation does not remove duplicates. > > Thanks, > David > > --- old/src/java.base/share/classes/java/lang/Class.java 2018-01-25 > 00:26:45.139747770 -0500 > +++ new/src/java.base/share/classes/java/lang/Class.java 2018-01-25 > 00:26:43.115631664 -0500 > @@ -3904,8 +3904,15 @@ > ????? * of its nest. The nest host will always be the zeroeth element. > ????? * > ????? *

Each listed nest member must be validated by checking its own > -???? * declared {@linkplain #getNestHost() nest host}. Any exceptions > that occur > -???? * as part of this process will be thrown. > +???? * declared nest host. Any exceptions that occur as part of this > process > +???? * will be thrown. > +???? * > +???? *

The list of nest members in the classfile is permitted to > +???? * contain duplicates, or to explicitly include the nest host. It > is not > +???? * required that an implementation of this method removes these > duplicates. > +???? * > +???? * @implNote This implementation does not remove duplicate nest > members if they > +???? * are present. > ????? * > ????? * @return an array of all classes and interfaces in the same > nest as > ????? * this class From david.holmes at oracle.com Tue Jan 30 09:55:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 30 Jan 2018 19:55:23 +1000 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access Message-ID: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> I've gone through the API specifications for core reflection, MethodHandles and VarHandles to see what changes are needed to accommodate nestmates and the related invocation rule changes. Turns out there is very little needed and most of what there is is non-normative, just correcting or clarifying particular things. For VarHandle nothing is needed as it defers all access checking to MethodHandle.Lookup. Bug: https://bugs.openjdk.java.net/browse/JDK-8191116 Webrev: http://cr.openjdk.java.net/~dholmes/8191116/webrev/ Core reflection changes: These are minimal as core reflection already expresses all access control in terms of "Java language access control" which already allows for nestmate access. - java/lang/reflect/AccessibleObject.java *

Java language access control prevents use of private members outside - * their class; package access members outside their package; protected members + * their top-level class; package access members outside their package; protected members This corrects the definition of private access in the Java language. - java/lang/reflect/Method.java * using dynamic method lookup as documented in The Java Language - * Specification, Second Edition, section 15.12.4.4; in particular, - * overriding based on the runtime type of the target object will occur. + * Specification, section 15.12.4.4; in particular, + * overriding based on the runtime type of the target object may occur. Removed unnecessary reference to "Second Edition". Changed 'will occur' to 'may occur' to account for the different forms of invocation that may apply. MethodHandle API Changes: - java/lang/invoke/MethodHandle.java * A non-virtual method handle to a specific virtual method implementation * can also be created. These do not perform virtual lookup based on * receiver type. Such a method handle simulates the effect of - * an {@code invokespecial} instruction to the same method. + * an {@code invokespecial} instruction to the same non-private method; + * or an {@code invokevirtual} or {@code invokeinterface} instruction to the + * same private method (as applicable). I tried to clarify that non-virtual invocations are not limited to invokespecial - as private invocations via invokevirtual or invokeinterface are also non-virtual. - java/lang/invoke/MethodHandles.java *

- * In some cases, access between nested classes is obtained by the Java compiler by creating - * an wrapper method to access a private method of another class - * in the same top-level declaration. + * Since JDK 11 the relationship between nested types can be expressed directly through the + * {@code NestHost} and {@code NestMembers} attributes. + * (See the Java Virtual Machine Specification, sections 4.7.28 and 4.7.29.) + * In that case, the lookup class has direct access to private members of all its nestmates, and + * that is true of the associated {@code Lookup} object as well. + * Otherwise, access between nested classes is obtained by the Java compiler creating + * a wrapper method to access a private method of another class in the same nest. * For example, a nested class {@code C.D} Updated the nested classes description to cover legacy approach and new nestmate approach. - * {@code C.E} would be unable to those private members. + * {@code C.E} would be unable to access those private members. Fixed typo: "access" was missing. * Discussion of private access: * We say that a lookup has private access * if its {@linkplain #lookupModes lookup modes} - * include the possibility of accessing {@code private} members. + * include the possibility of accessing {@code private} members + * (which includes the private members of nestmates). * As documented in the relevant methods elsewhere, * only lookups with private access possess the following capabilities: *

    - *
  • access private fields, methods, and constructors of the lookup class + *
  • access private fields, methods, and constructors of the lookup class and its nestmates Clarify that private access includes nestmate access. - * access all members of the caller's class, all public types in the caller's module, + * access all members of the caller's class and nestmates, all public types in the caller's module, Ditto. * When called, the handle will treat the first argument as a receiver - * and dispatch on the receiver's type to determine which method + * and, for non-private methods, dispatch on the receiver's type to determine which method * implementation to enter. + * For private methods the named method in {@code refc} will be invoked on the receiver. Clarify dispatch process for private versus non-private. * @throws IllegalAccessException if access checking fails, * or if the method is {@code static}, - * or if the method is {@code private} method of interface, This is the only actual spec change - it's no longer illegal for findVirtual to be used to find a private interface method. Thanks, David From david.holmes at oracle.com Tue Jan 30 20:51:32 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 31 Jan 2018 06:51:32 +1000 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> Message-ID: Hi Dan, Thanks for looking at this so quickly. On 31/01/2018 12:19 AM, Daniel Heidinga wrote: > Thanks David. > These changes all seem reasonable. ?I was going to complain about > changing?'will occur' to 'may occur' as 'may' make makes it difficult to > determine when it will vs won't occur but I think it's correct in this case. It's really non-normative anyway. The real process is described in JLS 15.12.4.4, so if you wanted to know for sure case-by-case then you'd need to refer back to that. David ----- > --Dan > > ----- Original message ----- > From: David Holmes > Sent by: "valhalla-spec-experts" > > To: valhalla-spec-experts at openjdk.java.net > Cc: > Subject: API Updates: 8191116: [Nestmates] Update core reflection, > MethodHandle and varhandle APIs to allow for nestmate access > Date: Tue, Jan 30, 2018 4:55 AM > > I've gone through the API specifications for core reflection, > MethodHandles and VarHandles to see what changes are needed to > accommodate nestmates and the related invocation rule changes. Turns > out there is very little needed and most of what there is is > non-normative, just correcting or clarifying particular things. For > VarHandle nothing is needed as it defers all access checking to > MethodHandle.Lookup. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8191116 > > > Webrev: http://cr.openjdk.java.net/~dholmes/8191116/webrev/ > > > > Core reflection changes: > > These are minimal as core reflection already expresses all access > control in terms of "Java language access control" which already > allows for nestmate access. > > - java/lang/reflect/AccessibleObject.java > ??*

    Java language access control prevents use of private > members outside > - * their class; package access members outside their package; > protected members > + * their top-level class; package access members outside their > package; protected members > > This corrects the definition of private access in the Java language. > > - java/lang/reflect/Method.java > ??* using dynamic method lookup as documented in The Java Language > - * Specification, Second Edition, section 15.12.4.4; in particular, > - * overriding based on the runtime type of the target object will > occur. > + * Specification, section 15.12.4.4; in particular, > + * overriding based on the runtime type of the target object may occur. > > Removed unnecessary reference to "Second Edition". Changed 'will > occur' to 'may occur' to account for the different forms of > invocation that may apply. > > > MethodHandle API Changes: > > - java/lang/invoke/MethodHandle.java > ??* A non-virtual method handle to a specific virtual method > implementation > ??* can also be created. ?These do not perform virtual lookup based on > ??* receiver type. ?Such a method handle simulates the effect of > - * an {@code invokespecial} instruction to the same method. > + * an {@code invokespecial} instruction to the same non-private method; > + * or an {@code invokevirtual} or {@code invokeinterface} > instruction to the > + * same private method (as applicable). > > I tried to clarify that non-virtual invocations are not limited to > invokespecial - as private invocations via invokevirtual or > invokeinterface are also non-virtual. > > - java/lang/invoke/MethodHandles.java > > ?? ? ?*

    > - ? ? * In some cases, access between nested classes is obtained by > the Java compiler by creating > - ? ? * an wrapper method to access a private method of another class > - ? ? * in the same top-level declaration. > + ? ? * Since JDK 11 the relationship between nested types can be > expressed directly through the > + ? ? * {@code NestHost} and {@code NestMembers} attributes. > + ? ? * (See the Java Virtual Machine Specification, sections 4.7.28 > and 4.7.29.) > + ? ? * In that case, the lookup class has direct access to private > members of all its nestmates, and > + ? ? * that is true of the associated {@code Lookup} object as well. > + ? ? * Otherwise, access between nested classes is obtained by the > Java compiler creating > + ? ? * a wrapper method to access a private method of another class > in the same nest. > ?? ? ?* For example, a nested class {@code C.D} > > Updated the nested classes description to cover legacy approach and > new nestmate approach. > > - ? ? * {@code C.E} would be unable to those private members. > + ? ? * {@code C.E} would be unable to access those private members. > > Fixed typo: "access" was missing. > > ?? ? ?* Discussion of private access: > ?? ? ?* We say that a lookup has private access > ?? ? ?* if its {@linkplain #lookupModes lookup modes} > - ? ? * include the possibility of accessing {@code private} members. > + ? ? * include the possibility of accessing {@code private} members > + ? ? * (which includes the private members of nestmates). > ?? ? ?* As documented in the relevant methods elsewhere, > ?? ? ?* only lookups with private access possess the following > capabilities: > ?? ? ?*

      > - ? ? *
    • access private fields, methods, and constructors of the > lookup class > + ? ? *
    • access private fields, methods, and constructors of the > lookup class and its nestmates > > Clarify that private access includes nestmate access. > > - ? ? * ?access all members of the caller's class, all public types > in the caller's module, > + ? ? * ?access all members of the caller's class and nestmates, all > public types in the caller's module, > > Ditto. > > ?? ? ?* When called, the handle will treat the first argument as a > receiver > - ? ? * and dispatch on the receiver's type to determine which method > + ? ? * and, for non-private methods, dispatch on the receiver's > type to determine which method > ?? ? ?* implementation to enter. > + ? ? * For private methods the named method in {@code refc} will be > invoked on the receiver. > > Clarify dispatch process for private versus non-private. > > ?? ? ?* @throws IllegalAccessException if access checking fails, > ?? ? ?* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?or if the method is {@code > static}, > - ? ? * ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?or if the method is {@code > private} method of interface, > > This is the only actual spec change - it's no longer illegal for > findVirtual to be used to find a private interface method. > > Thanks, > > David > > From karen.kinnear at oracle.com Tue Jan 30 22:36:26 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Tue, 30 Jan 2018 17:36:26 -0500 Subject: Valhalla EG Jan 31, 2018 Message-ID: <5DB470C1-EDF6-4B38-9B90-1B6E6A09FF3F@oracle.com> Reminder: meeting resumes Jan 31 at 9am PT/ noon ET/ 17 Dublin, 18 Grenoble/Stockholm, 20 St Petersburg https://oracle.zoom.us/j/5249803466 audio only: +1 646 558 8656 or +1 669 900 6833 US Toll Initial agenda items: 1. Condy: JVMS final ConstantDynamic spec: last call for feedback: http://cr.openjdk.java.net/~dlsmith/constant-dynamic.html summary of final changes: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2018-January/000505.html 2. Nestmates: JVMS changes: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2018-January/000500.html Summary: Changes from: http://cr.openjdk.java.net/~dlsmith/nestmates.html JVMS 6 Invokeinterface: Run-time Exceptions > Otherwise, if step 1 or step 2 of the lookup procedure selects a method that is not public, invokeinterface throws an IllegalAccessError. > This is the line that we believe could be restored, slightly modified to apply to the selection lookup procedure steps in 5.4.5, so that the only non-public selected method would be a private method which is the referenced method as well as the selected method. Concretely, we would undo this deletion, and instead modify the rule as follows: "Otherwise, if step 1 or step 2 of the lookup procedure selects a method that is ~~not public~~ **neither public nor private**, invokeinterface throws an IllegalAccessError." The motivation here is that invokeinterface is uniquely able to i) resolve to a public interface method, and ii) select a protected/package method of a superclass that would otherwise be inaccessible to the caller. The fact that the referenced method is in an interface is important, because anyone with the ability to extend a class can also declare a fresh superinterface that includes any method names+descriptors they're interested in. 3. LWorld Value Types: http://cr.openjdk.java.net/~acorn/LWorldValueTypes.pdf Updated proposal - walk through key changes: signature handling, nullability thanks, Karen From paul.sandoz at oracle.com Wed Jan 31 02:24:35 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 30 Jan 2018 18:24:35 -0800 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> Message-ID: <30772B5A-46C4-43D1-A78D-B598EB011C73@oracle.com> > On Jan 30, 2018, at 1:55 AM, David Holmes wrote: > MethodHandle API Changes: > > - java/lang/invoke/MethodHandle.java > * A non-virtual method handle to a specific virtual method implementation > * can also be created. These do not perform virtual lookup based on > * receiver type. Such a method handle simulates the effect of > - * an {@code invokespecial} instruction to the same method. > + * an {@code invokespecial} instruction to the same non-private method; > + * or an {@code invokevirtual} or {@code invokeinterface} instruction to the > + * same private method (as applicable). > I tried to clarify that non-virtual invocations are not limited to invokespecial - as private invocations via invokevirtual or invokeinterface are also non-virtual. > > Why s/same method/same non-private method/ for the invokespecial? It?s possible to look up a private method within the same class using Lookup.invokespecial and invoke it (and also look up a private constructor and invoke it). Paul. From david.holmes at oracle.com Wed Jan 31 03:22:07 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 31 Jan 2018 13:22:07 +1000 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: <30772B5A-46C4-43D1-A78D-B598EB011C73@oracle.com> References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> <30772B5A-46C4-43D1-A78D-B598EB011C73@oracle.com> Message-ID: <4824a94b-81dd-8ede-6b2d-dda1ae8b6d48@oracle.com> Hi Paul, On 31/01/2018 12:24 PM, Paul Sandoz wrote: >> On Jan 30, 2018, at 1:55 AM, David Holmes > > wrote: >> >> >> MethodHandle API Changes: >> >> - java/lang/invoke/MethodHandle.java >> * A non-virtual method handle to a specific virtual method implementation >> * can also be created. These do not perform virtual lookup based on >> * receiver type. Such a method handle simulates the effect of >> - * an {@code invokespecial} instruction to the same method. >> + * an {@code invokespecial} instruction to the same non-private method; >> + * or an {@code invokevirtual} or {@code invokeinterface} instruction to the >> + * same private method (as applicable). >> >> I tried to clarify that non-virtual invocations are not limited to >> invokespecial - as private invocations via invokevirtual or >> invokeinterface are also non-virtual. >> >> > > Why s/same method/same non-private method/ for the invokespecial? > > It?s possible to look up a private method within the same class using > Lookup.invokespecial and invoke it (and also look up a private > constructor and invoke it). Yes you are right, but this text is not trying to describe how an invokespecial might be used, but rather how "A non-virtual method handle to a specific virtual method implementation can also be created." But the notion of "specific virtual method implementation" is perhaps not applicable to private methods in the first place. Perhaps I just need to remove this change altogether and leave it as-is. Thanks, David > Paul. From paul.sandoz at oracle.com Wed Jan 31 16:48:26 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 31 Jan 2018 08:48:26 -0800 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: <4824a94b-81dd-8ede-6b2d-dda1ae8b6d48@oracle.com> References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> <30772B5A-46C4-43D1-A78D-B598EB011C73@oracle.com> <4824a94b-81dd-8ede-6b2d-dda1ae8b6d48@oracle.com> Message-ID: <56C82B88-0F86-44F8-9E57-4669260257B3@oracle.com> > On Jan 30, 2018, at 7:22 PM, David Holmes wrote: > > Hi Paul, > > On 31/01/2018 12:24 PM, Paul Sandoz wrote: >>> On Jan 30, 2018, at 1:55 AM, David Holmes > wrote: >>> >>> >>> MethodHandle API Changes: >>> >>> - java/lang/invoke/MethodHandle.java >>> * A non-virtual method handle to a specific virtual method implementation >>> * can also be created. These do not perform virtual lookup based on >>> * receiver type. Such a method handle simulates the effect of >>> - * an {@code invokespecial} instruction to the same method. >>> + * an {@code invokespecial} instruction to the same non-private method; >>> + * or an {@code invokevirtual} or {@code invokeinterface} instruction to the >>> + * same private method (as applicable). >>> >>> I tried to clarify that non-virtual invocations are not limited to invokespecial - as private invocations via invokevirtual or invokeinterface are also non-virtual. >>> >>> >> Why s/same method/same non-private method/ for the invokespecial? >> It?s possible to look up a private method within the same class using Lookup.invokespecial and invoke it (and also look up a private constructor and invoke it). > > Yes you are right, but this text is not trying to describe how an invokespecial might be used, but rather how "A non-virtual method handle to a specific virtual method implementation can also be created.? > Ok, i see now. > But the notion of "specific virtual method implementation" is perhaps not applicable to private methods in the first place. > > Perhaps I just need to remove this change altogether and leave it as-is. > I would be inclined just to leave it as is. Paul. From frederic.parain at oracle.com Wed Jan 31 19:38:47 2018 From: frederic.parain at oracle.com (Frederic Parain) Date: Wed, 31 Jan 2018 14:38:47 -0500 Subject: JVMS draft for L-world value types with support for nullability Message-ID: <8AD16BD6-2C65-4C4B-BBA1-4FEB99764209@oracle.com> Here?s a draft of the JVMS proposing a way to implement the L-world value types, with support for value-based classes migration to value classes (essentially, support for nullability): http://cr.openjdk.java.net/~fparain/L-world/L-World-JVMS-3.pdf The assumptions and key properties are listed in Karen?s document: http://cr.openjdk.java.net/~acorn/LWorldValueTypesjan31.pdf Feedback and comments are welcome. Fred From brian.goetz at oracle.com Wed Jan 31 20:39:00 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 31 Jan 2018 15:39:00 -0500 Subject: A "RecursiveConstants" attribute In-Reply-To: <2CEA5EE3-6F51-4806-9CA5-2BAFCF0620E8@oracle.com> References: <131B1F8D-F49C-47F8-BE7C-A660CBE1FCB5@oracle.com> <2CEA5EE3-6F51-4806-9CA5-2BAFCF0620E8@oracle.com> Message-ID: <95f19422-86a4-115b-eae7-d274ab57e072@oracle.com> It seems to me that the rational thing to do is, in SE 11, to implicitly assume max-depth is zero, and be done.? (IE, pretend to implement this proposal, but don't actually allow classfiles to have the attribute.)? Then we won't get any classfiles that have cycles, and the full spectrum of cycle-accepting approaches is still on the table. This proposal is essentially about *enabling* recursive constants in the classfile; one would not need a RC attribute if one has no self-reference in the CP.? But allowing cycles merely creates a bigger problem down the road -- how should bytecode readers present such a recursive constant using the symbolic reference API?? You can't construct such a thing with the API we have, and that's by design; the API has been designed to be value-based (and let compilers intern away the duplication).? So allowing controlled cycles in the classfile still doesn't address the question of how to build a symbolic reference for them.? The result will probably be classfiles that blow up when you try to parse them -- not good. What use cases are we really concerned about here?? I may have missed the meeting where these cases were brought up, but it feels a little bit like we're extrapolating from a single example (or maybe a single Future). On 1/19/2018 3:11 PM, Dan Smith wrote: > We had the following discussion in the Dec 20 meeting: > >> On Jan 3, 2018, at 9:11 AM, Karen Kinnear > > wrote: >> >> Dan S: Cycle handling >> Array types can have cycles >> ? - rather than relying on ordering - perhaps tell tree depth - e.g. >> component type & dimensions - all in the CP >> John: perhaps modify to include the depth indicators in an attribute >> (separate proof of acyclicity) >> Remi: but if you insert a condy ? >> John: would need to discard the attribute >> Dan H: concern about a constant array for each dimension >> ? maybe some adhoc compression techniques >> Remi: why do you want to be cycle free? >> Dan S: hard to verify if types cyclical? >> Remi: why do you need to check in the verifier? >> John: maybe a structural constraint? >> Remi: if value - ok, but not for an array of values? >> ? e.g. Object[] with first entity as itself >> Dan: want array type convertible to finite string, benefit of >> attribute not tied to construct is it can evolve >> >> Dan H: Concerns about attributes - will this be as hard to maintain >> for e.g. class redefiners/bytecode generators as stackmaptables? >> No - you have to keep the CP indices in sync, but stackmaptables >> required abstract interpretation - transformer >> did not have the type information, and the compression mechanism was >> harder to implement and these >> would just require bumping indices. > > To flesh the attribute idea out, here's a possible spec. Mentions > CONSTANT_Dynamic for now, and would be modified in the future to name > other constant types that allow cycles. > > Is this a happy solution to the problems raised in previous > discussions about constant cycles? Any new problems introduced? > > ?Dan > > > 4.7.28 The `RecursiveConstants` Attribute (new) > ----------------------------------------- > > The `RecursiveConstants` attribute is a variable-length attribute in > the `attributes` table of a `ClassFile` structure ([4.1]). The > `RecursiveConstants` attribute facilitates checks that a > `CONSTANT_Dynamic` entry in the `constant_pool` table does not refer, > directly or indirectly, to itself ([4.4.13]). > > ~~~~ > RecursiveConstants_attribute { > ? ? u2 attribute_name_index; > ? ? u4 attribute_length; > ? ? u2 num_recursive_constants; > ? ? { ? u2 constant_index; > ? ? ? ? u2 max_recursion_depth; > ? ? } recursive_constants[num_recursive_constants]; > } > ~~~~ > > The items in the `RecursiveConstants_attribute` structure are as follows: > > `attribute_name_index` > > : ?The value of the `attribute_name_index` item must be a valid index > into the `constant_pool` table. The `constant_pool` entry at that > index must be a `CONSTANT_Utf8_info` structure ([4.4.7]) representing > the string `"RecursiveConstants"`. > > `attribute_length` > > : The value of the `attribute_length` item indicates the length of the > attribute, excluding the initial six bytes. > > `num_recursive_constants` > > : The value of the `num_recursive_constants` item gives the number of > entries in the `recursive_constants` array. > > `recursive_constants` > > : Each entry in the `recursive_constants` table contains the following > two items: > > ? ? `constant_index` > ? ? : The value of `constant_index` must be a valid index into the > `constant_pool` table. The `constant_pool` entry at that index must be > a `CONSTANT_Dynamic` structure ([4.4.13]). > ? ? ? ? There may be at most one entry in the `recursive_constants` > table for each `CONSTANT_Dynamic` structure in the `constant_pool`. > ? ? `max_recursion_depth` > ? ? : The value of `max_recursion_depth` designates a _maximum > recursion depth_ for the referenced constant. > > > 4.4.13 The `CONSTANT_Dynamic_info` Structure (modified) > -------------------------------------------- > > The `CONSTANT_Dynamic_info` structure is used to describe a > _dynamically-computed constant_, an arbitrary primitive or reference > value produced by invocation of a _bootstrap method_ ([4.7.23]). The > structure specifies i) a bootstrap method handle with an optional > sequence of _static arguments_ and ii) a referenced name and type. > > ~~~~ > CONSTANT_Dynamic_info { > ? ?u1 tag; > ? ?u2 bootstrap_method_attr_index; > ? ?u2 name_and_type_index; > } > ~~~~ > > The items of the `CONSTANT_Dynamic_info` structure are as follows: > > `tag` > > : The `tag` item of the `CONSTANT_Dynamic_info` structure has the value > `CONSTANT_Dynamic` (17). > > `bootstrap_method_attr_index` > > : The value of the `bootstrap_method_attr_index` item must be a valid > index into the `bootstrap_methods` array of the bootstrap method table > ([4.7.23]) of this class file. > > ? ? **The _maximum recursion depth_ of a `CONSTANT_Dynamic` structure > is the value given by `max_recursion_depth` of an entry referencing > the `CONSTANT_Dynamic` structure in the `recursive_constants` table of > the `RecursiveConstants` attribute ([4.7.28]); or, if no such entry > exists, 0. Let _d_ be the maximum recursion depth of this > `CONSTANT_Dynamic_info` structure. For each item in the > `bootstrap_arguments` array of the `bootstrap_methods` entry > referenced by `bootstrap_method_attr_index`, if the item references a > `CONSTANT_Dynamic_info` structure, the maximum recursion depth of that > structure must be less than (and not equal to) _d_.** > > ? ? > **This check prevents a `CONSTANT_Dynamic_info` structure from > referring to itself via one of its static arguments.** > > `name_and_type_index` > > : The value of the `name_and_type_index` item must be a valid index > into the `constant_pool` table. The `constant_pool` entry at that > index must be a `CONSTANT_NameAndType_info` structure ([4.4.6]) > representing a name and field descriptor ([4.3.2]). > From daniel.smith at oracle.com Wed Jan 31 20:47:27 2018 From: daniel.smith at oracle.com (Dan Smith) Date: Wed, 31 Jan 2018 13:47:27 -0700 Subject: Final CONSTANT_Dynamic spec In-Reply-To: References: Message-ID: <80E6B9AF-AF53-416D-8710-5DCCED3A8EB5@oracle.com> > On Jan 18, 2018, at 5:14 PM, Dan Smith wrote: > > A proposed final spec for CONSTANT_Dynamic is here: > > http://cr.openjdk.java.net/~dlsmith/constant-dynamic.html > > There are two significant changes: > > 5.4.3: Expanded the rule about concurrent resolution to account for nested resolution in a single thread > > 5.4.3.6: Added a resolution-time rule for detecting cycles in static arguments, with some additional discussion about cycles Discussing this cycle-detection rule with Alex, we agreed that its nondeterminism was not helpful, and that we can make it deterministic without putting undue constraints on implementations. So here's a tweaked version of the rule we'd like to use instead: ----- Let X be the symbolic reference currently being resolved, and let Y be a static argument of X to be resolved as described above. If X and Y are both dynamically-computed constants, and if Y is either the same as X or has a static argument that references X through its static arguments, directly or indirectly, resolution fails with a StackOverflowError **at the point where re-resolution of X would be required**. ~~This rule allows some leeway in the sequencing of the error check: the implementation may resolve some of the static arguments of Y, or may not.~~ ----- Note that an implementation is free to try to re-resolve X multiple times and keep looping until the stack actually overflows?once you've reached the nested X the first time, all further computation is not observable by users. ?Dan From paul.sandoz at oracle.com Wed Jan 31 20:50:10 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 31 Jan 2018 12:50:10 -0800 Subject: JVMS draft for L-world value types with support for nullability In-Reply-To: <8AD16BD6-2C65-4C4B-BBA1-4FEB99764209@oracle.com> References: <8AD16BD6-2C65-4C4B-BBA1-4FEB99764209@oracle.com> Message-ID: Hi Fred, Some basic questions, which might be because the specification is transitioning from value-based to value classes. > The ACC_ENUM flag indicates that this class or its superclass is declared as an enumerated type. A class file must not have both ACC_ENUM and ACC_VALUE_TYPE flags set. Why can?t enum classes be values classes, where enum values are actual values? I think the answer may be below. > The ACC_NON_NULLABLE flag indicates that this field must never store the null reference. The field signature must be the signature of a class. The class specified in the field?s signature is loaded during the loading phase of the class declaring this field. The class of the field must be a value class. This field must be initialized with the default value of this value class. I suppose in theory this attribute could be applied to a non-value class? > A field must not have both ACC_STATIC and ACC_NON_NULLABLE flags set. This would rule out the static fields of a enum class, although in this case i would presume an enum class is such that after static initializer block all static fields would be assigned since they are also marked final. So, if static fields can be final then why not non-nullable? Thanks, Paul. > On Jan 31, 2018, at 11:38 AM, Frederic Parain wrote: > > Here?s a draft of the JVMS proposing a way to implement the L-world value types, > with support for value-based classes migration to value classes (essentially, support > for nullability): > > http://cr.openjdk.java.net/~fparain/L-world/L-World-JVMS-3.pdf > > The assumptions and key properties are listed in Karen?s document: > > http://cr.openjdk.java.net/~acorn/LWorldValueTypesjan31.pdf > > > Feedback and comments are welcome. > > Fred > From daniel.smith at oracle.com Wed Jan 31 21:13:15 2018 From: daniel.smith at oracle.com (Dan Smith) Date: Wed, 31 Jan 2018 14:13:15 -0700 Subject: A "RecursiveConstants" attribute In-Reply-To: <95f19422-86a4-115b-eae7-d274ab57e072@oracle.com> References: <131B1F8D-F49C-47F8-BE7C-A660CBE1FCB5@oracle.com> <2CEA5EE3-6F51-4806-9CA5-2BAFCF0620E8@oracle.com> <95f19422-86a4-115b-eae7-d274ab57e072@oracle.com> Message-ID: <60BB9395-3B8B-4901-A8AD-99B3C209A005@oracle.com> > On Jan 31, 2018, at 1:39 PM, Brian Goetz wrote: > > It seems to me that the rational thing to do is, in SE 11, to implicitly assume max-depth is zero, and be done. (IE, pretend to implement this proposal, but don't actually allow classfiles to have the attribute.) Then we won't get any classfiles that have cycles, and the full spectrum of cycle-accepting approaches is still on the table. > > This proposal is essentially about *enabling* recursive constants in the classfile; one would not need a RC attribute if one has no self-reference in the CP. But allowing cycles merely creates a bigger problem down the road -- how should bytecode readers present such a recursive constant using the symbolic reference API? You can't construct such a thing with the API we have, and that's by design; the API has been designed to be value-based (and let compilers intern away the duplication). So allowing controlled cycles in the classfile still doesn't address the question of how to build a symbolic reference for them. The result will probably be classfiles that blow up when you try to parse them -- not good. > > What use cases are we really concerned about here? I may have missed the meeting where these cases were brought up, but it feels a little bit like we're extrapolating from a single example (or maybe a single Future). Sorry, I didn't provide enough context for those who weren't part of previous discussions. You're misunderstanding the restriction. A CONSTANT_Dynamic with recursion depth 0 would not be allowed to have a CONSTANT_Dynamic as its static argument. Its static arguments could only be strings, numbers, classes, etc. A CONSTANT_Dynamic with recursion depth 1 would be allowed to have a CONSTANT_Dynamic as its static argument, but only if the argument has a recursion depth of 0. A CONSTANT_Dynamic with recursion depth 2 would be allowed to have a CONSTANT_Dynamic as its static argument, but only if the argument has a recursion depth of 0 or 1. Etc. Under no circumstances would a CONSTANT_Dynamic be allowed to refer to itself?because in order to do so, at some point in the cycle you'd have a CONSTANT_Dynamic with a static argument of an illegal depth. To restate my point about layering on this restriction after 11: when enforced, this rule gives us the property that there are no structural cycles in a well-formed (per 4.8) constant pool. So if, say, the symbolic reference API is building a structure from a loaded constant pool, it can assume no cycles exist. However, if we don't enforce the rule on version 55.0 class files, we can't retroactively enforce it later, and downstream tools won't get strong guarantees about CONSTANT_Dynamic (however, they will have strong guarantees about future structures like CONSTANT_ArrayType). ?Dan From frederic.parain at oracle.com Wed Jan 31 21:38:05 2018 From: frederic.parain at oracle.com (Frederic Parain) Date: Wed, 31 Jan 2018 16:38:05 -0500 Subject: JVMS draft for L-world value types with support for nullability In-Reply-To: References: <8AD16BD6-2C65-4C4B-BBA1-4FEB99764209@oracle.com> Message-ID: Hi Paul, > On Jan 31, 2018, at 15:50, Paul Sandoz wrote: > > Hi Fred, > > Some basic questions, which might be because the specification is transitioning from value-based to value classes. > > >> The ACC_ENUM flag indicates that this class or its superclass is declared as an enumerated type. A class file must not have both ACC_ENUM and ACC_VALUE_TYPE flags set. > > > Why can?t enum classes be values classes, where enum values are actual values? I think the answer may be below. First of all, there?s a backward compatibility issue with old enums. They have been defined with full identity, which is incompatible with being a value type. There?s also an issue with the super-type. The super-type of all enums is the abstract class java.lang.Enum, but the super-type of a value class must be java.lang.Object. One advantage of values types is that the JVM can flattened them, it is possible because of two properties: being identity-less and having a default value. Without a default value, the JVM cannot initialized a non-nullable field, so flattened cannot be performed, > > >> The ACC_NON_NULLABLE flag indicates that this field must never store the null reference. The field signature must be the signature of a class. The class specified in the field?s signature is loaded during the loading phase of the class declaring this field. The class of the field must be a value class. This field must be initialized with the default value of this value class. > > > I suppose in theory this attribute could be applied to a non-value class? No as it is define today, because of the lack of non-null default value for non-value class, which makes the initialization of such field impossible for the JVM. A future project might be to enable ACC_NON_NULLABLE for non-value class, but it would require a much complex initialization scheme, probably involving indy or condy. This is way beyond the scope of this draft. > > >> A field must not have both ACC_STATIC and ACC_NON_NULLABLE flags set. > > This would rule out the static fields of a enum class, although in this case i would presume an enum class is such that after static initializer block all static fields would be assigned since they are also marked final. > > So, if static fields can be final then why not non-nullable? This is an open question. The rational of the current choice is to avoid some circularity errors with some common construct for static fields. The ACC_NON_NULLABLE flag is a marker for the JVM that indicates where flattening is possible. The gain of flattening static fields is very small compared to the gains of flattening of instance fields or array components. And the risk of circularity errors is high. For instance, it is impossible to for a value class to have a static field with ACC_NON_NULLABLE of its own type. Static fields are initialized during the preparation phase (JVMS 5.4.2), which means for these fields to store the default value of the class, but the class has not been fully initialized yet, so it is still impossible to create such default values. The problem exists also with indirect references to the current class (cycles). So allowing ACC_NON_NULLABLE for static fields would require to write a new set of restrictions in the language about when the flag can be used and and it cannot. We saw theses constraints as a useless burden for the language (because the purpose of the flag is to enable some implementation optimizations), so we proposed to simply forbid them. If the language team thinks there?s more value in allowing ACC_NON_NULLABLE, we can revisit the current choice. Fred > > >> On Jan 31, 2018, at 11:38 AM, Frederic Parain wrote: >> >> Here?s a draft of the JVMS proposing a way to implement the L-world value types, >> with support for value-based classes migration to value classes (essentially, support >> for nullability): >> >> http://cr.openjdk.java.net/~fparain/L-world/L-World-JVMS-3.pdf >> >> The assumptions and key properties are listed in Karen?s document: >> >> http://cr.openjdk.java.net/~acorn/LWorldValueTypesjan31.pdf >> >> >> Feedback and comments are welcome. >> >> Fred >> > From brian.goetz at oracle.com Wed Jan 31 21:50:43 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 31 Jan 2018 16:50:43 -0500 Subject: A "RecursiveConstants" attribute In-Reply-To: <60BB9395-3B8B-4901-A8AD-99B3C209A005@oracle.com> References: <131B1F8D-F49C-47F8-BE7C-A660CBE1FCB5@oracle.com> <2CEA5EE3-6F51-4806-9CA5-2BAFCF0620E8@oracle.com> <95f19422-86a4-115b-eae7-d274ab57e072@oracle.com> <60BB9395-3B8B-4901-A8AD-99B3C209A005@oracle.com> Message-ID: OK, so the goal is clearly to *prohibit* cyclic constants by enforcing a depth constraint on condy-in-condy nesting.? I like this goal much better, and I see how the solution addresses it. Still, I have to wonder whether its worth the complexity of capturing it explicitly with a new (stackmap-like) attribute, which must be specified, and gives us more chances for the classfile to be out of sync with itself, for the goal of preventing something that can't usefully happen anyway -- as we discussed previously, trying to actually resolve a cyclic constant will likely result in SOE, so such classfiles are pretty much useless and therefore not likely to spread very far. > Sorry, I didn't provide enough context for those who weren't part of previous discussions. You're misunderstanding the restriction. > > A CONSTANT_Dynamic with recursion depth 0 would not be allowed to have a CONSTANT_Dynamic as its static argument. Its static arguments could only be strings, numbers, classes, etc. > > A CONSTANT_Dynamic with recursion depth 1 would be allowed to have a CONSTANT_Dynamic as its static argument, but only if the argument has a recursion depth of 0. > > A CONSTANT_Dynamic with recursion depth 2 would be allowed to have a CONSTANT_Dynamic as its static argument, but only if the argument has a recursion depth of 0 or 1. > > Etc. > > Under no circumstances would a CONSTANT_Dynamic be allowed to refer to itself?because in order to do so, at some point in the cycle you'd have a CONSTANT_Dynamic with a static argument of an illegal depth. > > To restate my point about layering on this restriction after 11: when enforced, this rule gives us the property that there are no structural cycles in a well-formed (per 4.8) constant pool. So if, say, the symbolic reference API is building a structure from a loaded constant pool, it can assume no cycles exist. However, if we don't enforce the rule on version 55.0 class files, we can't retroactively enforce it later, and downstream tools won't get strong guarantees about CONSTANT_Dynamic (however, they will have strong guarantees about future structures like CONSTANT_ArrayType). > > ?Dan