From john.r.rose at oracle.com Thu Feb 1 14:34:28 2018 From: john.r.rose at oracle.com (John Rose) Date: Thu, 1 Feb 2018 14:34:28 +0000 Subject: JVMS draft for L-world value types with support for nullability In-Reply-To: References: <8AD16BD6-2C65-4C4B-BBA1-4FEB99764209@oracle.com> Message-ID: <603DF4A9-2F9A-4FA9-8EF8-954991FBF358@oracle.com> On Jan 31, 2018, at 9:38 PM, Frederic Parain wrote: > > Hi Paul, > >> On Jan 31, 2018, at 15:50, Paul Sandoz wrote: >> >> Hi Fred, >> >> Some basic questions, which might be because the specification is transitioning from value-based to value classes. >> >> >>> The ACC_ENUM flag indicates that this class or its superclass is declared as an enumerated type. A class file must not have both ACC_ENUM and ACC_VALUE_TYPE flags set. >> >> >> Why can?t enum classes be values classes, where enum values are actual values? I think the answer may be below. > > First of all, there?s a backward compatibility issue with old enums. They have been defined with full identity, > which is incompatible with being a value type. > There?s also an issue with the super-type. The super-type of all enums is the abstract class java.lang.Enum, > but the super-type of a value class must be java.lang.Object. > One advantage of values types is that the JVM can flattened them, it is possible because of two properties: > being identity-less and having a default value. Without a default value, the JVM cannot initialized a non-nullable > field, so flattened cannot be performed, Good summary. Although it is desirable to evolve enums to value types, we would have to figure out a migration strategy that would allow us to recompile them into value types, but also allow old code to operate correctly on them. I don't think this is an easy problem, in L-world or any other world. A JVM could do a heroic optimization on a field of enum type, if the enum type were loaded before the class containing the field. That would be an instructive project, I think, but it is not an important one. > >> >> >>> The ACC_NON_NULLABLE flag indicates that this field must never store the null reference. The field signature must be the signature of a class. The class specified in the field?s signature is loaded during the loading phase of the class declaring this field. The class of the field must be a value class. This field must be initialized with the default value of this value class. >> >> >> I suppose in theory this attribute could be applied to a non-value class? > > No as it is define today, because of the lack of non-null default value for non-value class, which makes the initialization > of such field impossible for the JVM. > > A future project might be to enable ACC_NON_NULLABLE for non-value class, but > it would require a much complex initialization scheme, probably involving indy or condy. > This is way beyond the scope of this draft. I agree. The JVM would have to somehow track the initialization state of a non-nullable field, forcing each constructor to initialize it, and preventing access before initialization. (Idea: Use null internally, and have getfield throw NPE if there is accidental access before construction is complete.) > >> >>> A field must not have both ACC_STATIC and ACC_NON_NULLABLE flags set. >> >> This would rule out the static fields of a enum class, although in this case i would presume an enum class is such that after static initializer block all static fields would be assigned since they are also marked final. >> >> So, if static fields can be final then why not non-nullable? > > This is an open question. > > The rational of the current choice is to avoid some circularity errors with some > common construct for static fields. Don't we have similar circularity problems with non-static fields? Why are static fields worse? One answer: You have to create the Class mirror very early (during preparation), and we use Class mirrors to store static field values. value class A { static flat B BVAL; } value class B { static flat A AVAL; } Thus, preparing A requires the layout of B, and vice versa. To me this seems like an inconvenience, not an inherent flaw of flat statics. Maybe a crude but effective way to break the bootstrap cycle is to store BVAL and AVAL with internal references, initialized to null. Then, have getstatic silently convert the internal nulls to default values. It's a cheat, but nobody would know. Or, more simply, convert flat statics into one-element arrays, again as an internal cheat: value class A { static B[] $INTERNAL$ARRAY$BVAL = new B[1]; } Then getstatic would secretly load the first (and only) element of the one-element array which stores the static. It might also have to recognize a null array reference, during bootstrapping, and "load" a default value from the not-yet-existing array. > > The ACC_NON_NULLABLE flag is a marker for the JVM that indicates where flattening > is possible. The gain of flattening static fields is very small compared to the gains of > flattening of instance fields or array components. And the risk of circularity errors is high. > For instance, it is impossible to for a value class to have a static field with ACC_NON_NULLABLE > of its own type. I hope we can use a different term: It is not impossible; it is unimplemented. The example is: value class A { static flat A AVAL; } > Static fields are initialized during the preparation phase (JVMS 5.4.2), > which means for these fields to store the default value of the class, but the class > has not been fully initialized yet, so it is still impossible to create such default values. After the class is loaded, and before it is prepared, its instance layout is known. At that point the default value can be created. I think what this puzzle shows us is that default values, like nulls, can exist before the initialization of a class. (Or am I missing some vicious circularity?) > The problem exists also with indirect references to the current class (cycles). > > So allowing ACC_NON_NULLABLE for static fields would require to write a new set > of restrictions in the language about when the flag can be used and and it cannot. Languages can do whatever they want; it's up to them to use a translation strategy that produces the designed semantics. If the JVM refused to supply flat statics, the language could still get them with a translation strategy that used hidden arrays and null-to-default conversions. But I think the JVM should not refuse. > We saw theses constraints as a useless burden for the language (because the purpose > of the flag is to enable some implementation optimizations), so we proposed > to simply forbid them. > > If the language team thinks there?s more value in allowing ACC_NON_NULLABLE, we > can revisit the current choice. I don't see this as a language decision: The question is whether the two modifier bits (static, non-nullable) should constrain each other or whether they should act orthogonally. That's a JVM design issue, and I think we would regret linking them together in this way, because it's not a clean design. (That's assuming I'm right about the absence of a true logical circularity here.) The corresponding language question is whether value-statics must be nullable. It would be very surprising to me if the language team decided that was desirable. ? John From john.r.rose at oracle.com Thu Feb 1 14:37:10 2018 From: john.r.rose at oracle.com (John Rose) Date: Thu, 1 Feb 2018 14:37:10 +0000 Subject: JVMS draft for L-world value types with support for nullability In-Reply-To: <8AD16BD6-2C65-4C4B-BBA1-4FEB99764209@oracle.com> References: <8AD16BD6-2C65-4C4B-BBA1-4FEB99764209@oracle.com> Message-ID: <51A87423-0717-4351-9AC8-3013662D5C1A@oracle.com> On Jan 31, 2018, at 7:38 PM, Frederic Parain wrote: > > http://cr.openjdk.java.net/~acorn/LWorldValueTypesjan31.pdf > > > Feedback and comments are welcome. A couple more comments about Object as an "honorary interface": As your document suggests, I don't think we have a real option to make a new top type I$Object that is a proper interface. Instead, we have to get into the habit of treating Object like an interface. The cost of this is pretty small, I think. Mainly we have to treat the "messy" API of Object (wait/notify/finalize) as a set of partial functions, which can throw exceptions (or have some other empty behavior) when applied to values. ? John From paul.sandoz at oracle.com Thu Feb 1 17:24:54 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 1 Feb 2018 09:24:54 -0800 Subject: JVMS draft for L-world value types with support for nullability In-Reply-To: <603DF4A9-2F9A-4FA9-8EF8-954991FBF358@oracle.com> References: <8AD16BD6-2C65-4C4B-BBA1-4FEB99764209@oracle.com> <603DF4A9-2F9A-4FA9-8EF8-954991FBF358@oracle.com> Message-ID: <0209503F-E77B-458F-BD18-01452A075818@oracle.com> Hi Fred, John, Thanks for the explanations, very informative. Regarding enums i was not thinking of the case of implicitly retrofitting all enum classes to be value classes (that would be tricky as you point out). I was wondering if it might be possible for an enum class to also explicitly be a value class, e.g.: __Value enum MyEnumValue { V1, V2, V3; } (placing aside compatibility issues of an explicit transition, and even though it inherits from abstract class Enum), since enum is anyway special cased in some ways, but perhaps not sufficiently to make it easy to do. Thanks, Paul. > On Feb 1, 2018, at 6:34 AM, John Rose wrote: > > On Jan 31, 2018, at 9:38 PM, Frederic Parain wrote: >> >> Hi Paul, >> >>> On Jan 31, 2018, at 15:50, Paul Sandoz wrote: >>> >>> Hi Fred, >>> >>> Some basic questions, which might be because the specification is transitioning from value-based to value classes. >>> >>> >>>> The ACC_ENUM flag indicates that this class or its superclass is declared as an enumerated type. A class file must not have both ACC_ENUM and ACC_VALUE_TYPE flags set. >>> >>> >>> Why can?t enum classes be values classes, where enum values are actual values? I think the answer may be below. >> >> First of all, there?s a backward compatibility issue with old enums. They have been defined with full identity, >> which is incompatible with being a value type. >> There?s also an issue with the super-type. The super-type of all enums is the abstract class java.lang.Enum, >> but the super-type of a value class must be java.lang.Object. >> One advantage of values types is that the JVM can flattened them, it is possible because of two properties: >> being identity-less and having a default value. Without a default value, the JVM cannot initialized a non-nullable >> field, so flattened cannot be performed, > > Good summary. Although it is desirable to evolve enums to value types, > we would have to figure out a migration strategy that would allow us to > recompile them into value types, but also allow old code to operate > correctly on them. I don't think this is an easy problem, in L-world or > any other world. > > A JVM could do a heroic optimization on a field of enum type, if the > enum type were loaded before the class containing the field. That > would be an instructive project, I think, but it is not an important one. > >> >>> >>> >>>> The ACC_NON_NULLABLE flag indicates that this field must never store the null reference. The field signature must be the signature of a class. The class specified in the field?s signature is loaded during the loading phase of the class declaring this field. The class of the field must be a value class. This field must be initialized with the default value of this value class. >>> >>> >>> I suppose in theory this attribute could be applied to a non-value class? >> >> No as it is define today, because of the lack of non-null default value for non-value class, which makes the initialization >> of such field impossible for the JVM. >> >> A future project might be to enable ACC_NON_NULLABLE for non-value class, but >> it would require a much complex initialization scheme, probably involving indy or condy. >> This is way beyond the scope of this draft. > > I agree. The JVM would have to somehow track the initialization state > of a non-nullable field, forcing each constructor to initialize it, and preventing > access before initialization. (Idea: Use null internally, and have getfield > throw NPE if there is accidental access before construction is complete.) > >> >>> >>>> A field must not have both ACC_STATIC and ACC_NON_NULLABLE flags set. >>> >>> This would rule out the static fields of a enum class, although in this case i would presume an enum class is such that after static initializer block all static fields would be assigned since they are also marked final. >>> >>> So, if static fields can be final then why not non-nullable? >> >> This is an open question. >> >> The rational of the current choice is to avoid some circularity errors with some >> common construct for static fields. > > Don't we have similar circularity problems with non-static fields? Why are > static fields worse? One answer: You have to create the Class mirror > very early (during preparation), and we use Class mirrors to store static > field values. > > value class A { static flat B BVAL; } > value class B { static flat A AVAL; } > > Thus, preparing A requires the layout of B, and vice versa. To me this > seems like an inconvenience, not an inherent flaw of flat statics. > > Maybe a crude but effective way to break the bootstrap cycle is to > store BVAL and AVAL with internal references, initialized to null. > Then, have getstatic silently convert the internal nulls to default > values. It's a cheat, but nobody would know. > > Or, more simply, convert flat statics into one-element arrays, > again as an internal cheat: > > value class A { static B[] $INTERNAL$ARRAY$BVAL = new B[1]; } > > Then getstatic would secretly load the first (and only) element of the > one-element array which stores the static. It might also have to > recognize a null array reference, during bootstrapping, and "load" > a default value from the not-yet-existing array. > >> >> The ACC_NON_NULLABLE flag is a marker for the JVM that indicates where flattening >> is possible. The gain of flattening static fields is very small compared to the gains of >> flattening of instance fields or array components. And the risk of circularity errors is high. >> For instance, it is impossible to for a value class to have a static field with ACC_NON_NULLABLE >> of its own type. > > I hope we can use a different term: It is not impossible; it is unimplemented. > > The example is: > > value class A { static flat A AVAL; } > > >> Static fields are initialized during the preparation phase (JVMS 5.4.2), >> which means for these fields to store the default value of the class, but the class >> has not been fully initialized yet, so it is still impossible to create such default values. > > After the class is loaded, and before it is prepared, its instance layout is known. > At that point the default value can be created. I think what this puzzle shows > us is that default values, like nulls, can exist before the initialization of a class. > > (Or am I missing some vicious circularity?) > >> The problem exists also with indirect references to the current class (cycles). >> >> So allowing ACC_NON_NULLABLE for static fields would require to write a new set >> of restrictions in the language about when the flag can be used and and it cannot. > > Languages can do whatever they want; it's up to them to use a translation strategy > that produces the designed semantics. If the JVM refused to supply flat statics, > the language could still get them with a translation strategy that used hidden arrays > and null-to-default conversions. But I think the JVM should not refuse. > >> We saw theses constraints as a useless burden for the language (because the purpose >> of the flag is to enable some implementation optimizations), so we proposed >> to simply forbid them. >> >> If the language team thinks there?s more value in allowing ACC_NON_NULLABLE, we >> can revisit the current choice. > > I don't see this as a language decision: The question is whether the two modifier > bits (static, non-nullable) should constrain each other or whether they should > act orthogonally. That's a JVM design issue, and I think we would regret linking > them together in this way, because it's not a clean design. (That's assuming I'm > right about the absence of a true logical circularity here.) > > The corresponding language question is whether value-statics must be nullable. > It would be very surprising to me if the language team decided that was desirable. > > ? John From john.r.rose at oracle.com Thu Feb 1 19:05:06 2018 From: john.r.rose at oracle.com (John Rose) Date: Thu, 1 Feb 2018 19:05:06 +0000 Subject: JVMS draft for L-world value types with support for nullability In-Reply-To: <0209503F-E77B-458F-BD18-01452A075818@oracle.com> References: <8AD16BD6-2C65-4C4B-BBA1-4FEB99764209@oracle.com> <603DF4A9-2F9A-4FA9-8EF8-954991FBF358@oracle.com> <0209503F-E77B-458F-BD18-01452A075818@oracle.com> Message-ID: <37D843A0-D615-48E8-B00C-49049103DF21@oracle.com> On Feb 1, 2018, at 5:24 PM, Paul Sandoz wrote: > > an enum class to also explicitly be a value class Yes, it's probably doable, but we would have to work out the migration story, and also figure out how to manage the Enum supertype. Remember that one of the ongoing challenges of VT's is the role of Object. For value-enums, I think we would reprise the role for Enum. I'd rather have template classes under my belt before tackling that, so I could make Enum behave differently as an object vs. a value supertype. From john.r.rose at oracle.com Sun Feb 4 17:16:50 2018 From: john.r.rose at oracle.com (John Rose) Date: Sun, 4 Feb 2018 18:16:50 +0100 Subject: Final CONSTANT_Dynamic spec In-Reply-To: References: <80E6B9AF-AF53-416D-8710-5DCCED3A8EB5@oracle.com> Message-ID: On Feb 4, 2018, at 5:59 PM, Daniel Heidinga wrote: > > all further computation is not observable by users. Yeah, that?s pretty loose. I don?t think anyone wants to propose rollback for this corner case. Can anyone provide a better rephrase of the point? (Who puts the dan in pedantic??) From john.r.rose at oracle.com Sun Feb 4 18:25:34 2018 From: john.r.rose at oracle.com (John Rose) Date: Sun, 4 Feb 2018 19:25:34 +0100 Subject: Fwd: optimizing acmp in L-World References: <47625659-9B0E-481C-B755-4C5C4E2DE23C@oracle.com> Message-ID: <656A690F-8A94-496B-B12E-96A8F83DBB7E@oracle.com> FTR, here are some observations on optimizing new acmp semantics which will help motivate adopting the L-world design. From: John Rose Date: January 29, 2018 at 9:42:49 PM GMT+1 To: valhalla-dev Subject: optimizing acmp in L-World Tobias, Roland, and I had a chat in Santa Clara last week about optimizing the acmp operation in the JVM. Here are my notes on it. Background: The acmp operation, also known as reference comparison, is overloaded in L-World to handle values as well, with a specific semantics of returning false if either operand is a value (similar to NaN behavior for fcmp). The rationale for this is that acmp is reference comparison, even in L-World, and since values have no references, they can never be equal as references. (Alternatively, it is *as if* both operands were converted *to references* before the acmp operation, which means that either operand, if it is a value, *would be* boxed to a reference value, which then, being a new reference, is automatically unequal to any other reference. This tricky account may be appealing in some circumstances.) OK, so how do we make it fast? Ideally, we would like to strength-reduce the enhanced acmp instruction down to the original acmp instruction, which after all is just a single instruction (such as cmpq) on all platforms. The first thing to notice, before going into details, is that in L-world all Q-values are buffered in the interpreter. This means that, at least for interpreter-generated values, all values can be treated as physical pointers, even if they are logically Q-values. (Crib sheet: In L-world, all kinds of values except primitives are formally carried by L-values, under L-descriptors. True references can be referred to as R-values of R-type, reference type, or object class. New value types are referred to as Q-values of Q-type, value type, or value class. When you don't know what you have, you can say it's a U-type, which just like a Q-type or R-type is carried by an L-value under an L-descriptor.) This means that the old acmp semantics are almost correct, in many cases. If you have two L-values and do a pointer comparison on their physical pointers, and if the pointers differ, you are done. If the physical pointers compare equal, then there is one more thing you have to do: Make sure that the L-value carried by both physical pointers is in fact a reference, an R-value. So the logic looks like this, and the new semantics are in the second "if" statement: bool new_acmp(ptr x, ptr y) { if (!old_acmp(x, y)) return false; // x != y => false if (is_value_ptr(x)) return false; // x.is_value => false return true; } bool old_acmp(ptr x, ptr y) { return x == y; // cmpq etc. } Note that since acmp is symmetric, the JVM has the option to silently test either x or y for "is_value_ptr" if the two physical pointers compare equal. This leads immediately to the first set of optimizations, which probably are enough to get us where we want to go. (There is a second set we can add later.) All the JVM has to do is prove or detect that one or the other physical pointer either *is certainly* a Q-value, or *is certainly not* a Q-value. If either physical pointer is certainly a Q-value, then the new_acmp instruction is a constant false. If either physical pointer is certainly not a Q-value, then the new_acmp instruction strength reduces to the old_acmp instruction. The strength reduction to false can be done in any place where the type of either operand is known to be a Q-type. The interpreter doesn't track types, and the acmp instruction isn't resolved so it can't be quickened to a type context, but the JIT has lots of type information. Thus, if either operand of acmp is statically known (or accurately speculated) to be a value type, then the acmp instruction constant folds to false. Likewise, if either operand is known to be an R-type (true reference) the acmp may be strength reduced to a simple reference comparison. This also is likely to happen in the JIT. It also happens in the interpreter in the acmp_null instruction, where (obviously) the second operand is a reference (i.e., null). (Note that the type Object is a U-type in L-world, so Object-to-Object comparisons such as are found in generic collection code do not benefit from this optimization. More later on that.) The JIT can also track, in its profile, whether various instructions "see" only Q-values or only R-values. (The classic HotSpot already tracks "sightings" of null for similar purposes.) In that case, speculative narrowings (only Q-types or only R-types) can be used to short circuit new_acmp down to old_acmp. This is useful because the speculation can sometimes be verified out of a loop body, where the actual acmp instruction is executed in a loop. Remember that the JIT can choose which operand of acmp to test for value-ness; if it chooses a loop invariant, and the invariant is always Q or always R, then the acmp inside the loop folds up, no matter what the dynamic value of the other operand. For that matter, even if speculation fails, loop unswitching can get to a similar result, at the cost of two copies of the loop. So, in many cases, contextual information can allow the JVM to fold up new_acmp to old_acmp or constant false. In cases such folding fails, there is as simple trick which can make things easier. Recall that folding fails when *both* operands are statically U-types, *neither* operand folding to a Q-type or R-type. In that case, the JVM must pick one operand or the other (it doesn't matter which) and perform the is_value_ptr operation on it, in such a way that the result of that operand affects the final result in the correct way. A naive implementation of new_acmp adds an extra test-and-branch to fold in the is_value_ptr test, but a clever might be able to avoid it. Suppose there is a branch free technique for implementing is_value_ptr, such as extracting a bit from the physical pointer to the U-value, or (after a null check) extracting a bit from the physical pointer of the class metadata pointer in the value's header, or (after two indirections) extracting a bit from the structure of the class metadata. (Mr. Simm's prototype puts such a bit in the metadata pointer, in the object header.) If such a bit can be obtained in a branch-free manner, then it can be used to perturb the result of the old_acmp in a useful manner. For example: bool new_acmp(ptr x, ptr y) { if (x == null) return y == null; int bit = (x->class_in_header & Q_TYPE_BIT); ptr x1 = x | swar_broadcast_bit(bit != 0); return old_acmp(x1, y); } int swar_broadcast_bit(bool z) { return z ? -1 : 0; // use ALU ops } The idea is to extract a bit which signals the presence of a Q-type (in either operand) and use it to "mess up" the equality comparison, using and, or, or xor as needed. This perturbation technique has two costs: First, it requires a null check (unless the Q-bit is in the actual physical reference). Second, it requires an ALU operation or two to spread the perturbation. But these costs will probably be negligible, except (perhaps) in very tight loops in generic code. Generic code loops performing frequent acmp operations on untyped (Object) operands will need to perform extra null checks and/or value/reference detections if they are not already present (by luck) in the loop context. In this case, loop unswitching may be useful. There is one more contextual operation which may be helpful in such cases, if all else fails (and loop unswitching is not desirable). Honestly, I haven't seen any real-world code yet that needs it, but it is comforting to have as a fall-back. The idea is this: If the acmp is contextually followed by a call to Object.equals, in the usual Legacy Idiom For Equality (L.I.F.E.), then there is one more trick we can play. We'd like to force the new_acmp down to an old_acmp in these cases. Can we? Here's the choice: bool dutiful_life(ptr x, ptr y) { if (new_acmp(x, y)) return true; return x->Object_equals(y); } bool clever_life(ptr x, ptr y) { if (old_acmp(x, y)) return true; return x->Object_equals(y); } Note that the only semantic difference between old_acmp and new_acmp arises when *both* operands are the *same* value. (Work out the cases: In every other case, the simple pointer comparison detects the difference and produces the correct "false" answer.) Thus, in the L.I.F.E., the only case where old_acmp gives the wrong answer produces an immediate answer of "true", instead of following up with a call to x.equals(x) on the Q-value. To put it in a nutshell, the only difference between dutiful_life and clever_life is that the former always calls Object::equals if either operand is a value, and the the latter calls Object::equals in a subset of those cases (when the physical pointers differ). When the physical pointers do *not* differ, there is no call to Object::equals. But clever_life always delivers the same answer as dutiful_life. So is there a problem? Not much, although it requires us to treat Object::equals as an intrinsic with some predictable semantics. The crucial inside that allows us to adopt clever_life is that, if you call x.equals(x) on a non-null x, the result must always be true. Now, this is not enforced by the JVM, but it is clearly documented in the API specification for Object::equals, and lots of stuff would be already broken if that were ever false. Thus, if we are willing to rule out implementations of Object::equals for which a value can ever compare *not equal* to itself, then we can substitute clever_life for dutiful_life. We also have to allow one more thing: That we can silently drop calls to equals (at least in the context of L.I.F.E.) when the JVM can prove that the contract of equals allows the JVM to predict the outcome. This means that if you put side effects (like print statements) into an equals method on a value type, you might not see the side effects, after the JVM has optimized things. This corner case may require further thought in order to straighten it out. Note that L.I.F.E. is frequently found in generic code, where instead of operating on Object values it operates on type parameter types. (The JVM just sees Object, after erasure.) In that case, if the JVM were to specialize the generic code separately for some types (such as value types), along the lines of the template class proposals, then the L.I.F.E. code would fold up in the JIT using the rules given at the top of this note. The generic expression "a == b", where a or b was a value of a specialized type parameter T, would fold up to false in every specialization where T was a Q-type. Likewise, as noted above, if L.I.F.E. is applied to a true R-type (or specialized to such a type), then any new_acmp on that type can be short-circuited to the faster old_acmp. All in all, it appears that the cost of the new acmp semantics can, for all practical purposes, be pushed down into the noise. ? John From frederic.parain at oracle.com Mon Feb 5 19:05:55 2018 From: frederic.parain at oracle.com (Frederic Parain) Date: Mon, 5 Feb 2018 14:05:55 -0500 Subject: JVMS draft for L-world value types with support for nullability In-Reply-To: <37D843A0-D615-48E8-B00C-49049103DF21@oracle.com> References: <8AD16BD6-2C65-4C4B-BBA1-4FEB99764209@oracle.com> <603DF4A9-2F9A-4FA9-8EF8-954991FBF358@oracle.com> <0209503F-E77B-458F-BD18-01452A075818@oracle.com> <37D843A0-D615-48E8-B00C-49049103DF21@oracle.com> Message-ID: <9B2F324F-F2EF-4702-90F1-CDC3B6BE7FF8@oracle.com> Here?s an update of the JVMS draft: http://cr.openjdk.java.net/~fparain/L-world/L-World-JVMS-4.pdf with the following changes: * Renamed ACC_NON_NULLABLE with ACC_FLATTENABLE * Removed restriction that ACC_FLATTENABLE cannot be used on static fields * Added rules about circularity restrictions for fields with the ACC_FLATTENABLE flag set * Added rules about initialization of classes of flattenable fields Fred From karen.kinnear at oracle.com Mon Feb 5 23:31:29 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Mon, 5 Feb 2018 18:31:29 -0500 Subject: JVMS draft for L-world value types with support for nullability In-Reply-To: <9B2F324F-F2EF-4702-90F1-CDC3B6BE7FF8@oracle.com> References: <8AD16BD6-2C65-4C4B-BBA1-4FEB99764209@oracle.com> <603DF4A9-2F9A-4FA9-8EF8-954991FBF358@oracle.com> <0209503F-E77B-458F-BD18-01452A075818@oracle.com> <37D843A0-D615-48E8-B00C-49049103DF21@oracle.com> <9B2F324F-F2EF-4702-90F1-CDC3B6BE7FF8@oracle.com> Message-ID: <717CE742-BB91-4DF9-9561-D05154591AE2@oracle.com> Frederic, Detailed review of the updated JVMS draft - many are just minor typos/edits. A couple are bigger. Many thanks for the latest changes. thanks, Karen 1. 2.4 Reference Types and Values " whose all instances are identity-less and immutable" "whose" -> "for which" 2. 2.11.5 Instance Creation and Manipulation 4th bullet: non-staticfields - need a space) 3. 4.1 The Classfile Structure "If none of the ACC_VALUE_TYPE or the ACC_INTERFACE flag is set" -> "If neither the ACC_VALUE_TYPE ..." 4. 4.4.1 The CONSTANT_Class_info Structure Because arrays are objects, ... "but not the opcode new" - perhaps "but not the opcodes new or defaultvalue". 5. 4.9.1. Static Constraints getfield, putfield, getstatic, putstatic -- also withfield - must refer to a CONSTANT_FieldRef 6. 4.9.1 Static Constraints operands of instanceof, checkcast, new ... - also include defaultvalue 7. 4.9.1 Static Constraints No defaulvalue instruction may reference a constant pool entry ... - fix indentation 8. 5.3.5 Deriving a Class from a class File Representation 3. Note that if C is an interface it must have Object as its direct superclass, ... change to "if C is an interface or a value class" 9. 5.3.5 Deriving a Class from a class File Representation 3. "If the class or interface named as the direct superclass of C is in fact an interface, ... change to "is in fact an interface or value class" -> ICCE 10. 5.3.5 Deriving a Class from a class File Representation 5. Change ACC_VALUE_TYPE flag to ACC_FLATTENABLE 5. Perhaps word: If C has any non-static field FC with the ACC_FLATTENABLE flag set, in order to use F below in the exception case 5. Exceptions: Rather than triggering a StackOverflwError, I was expecting a ClassCircularityError * If any of the fields marked as ACC_FLATTENABLE is not in fact a value class, loading throws an ICCE * Otherwise, if any of the ACC_FLATTENABLE fields contains directly or indirectly the class C or the class FC, loading throws a ClassCircularityError 11. 5.3.5 Deriving a Class from a class File Representation 5. So it looks as though you have made a change to allow ACC_FLATTENABLE on an object class. I recognize there have been requests to support that in the future, but I think that will require additional discussions before I would make a JVMS change. Perhaps a bit of non-normative text here say in italics? i.e. we do not allow a value class to migrate to an object class - intention is to allow declaration of a ACC_FLATTENABLE to imply non-nullable for object classes as well as value classes? - I would leave this out until we have had further detailed discussions 12. 5.5. Initialization Thank you for adding step 8 to pre-initialize an ACC_FLATTENABLE field. Can you also add this to the bullets under "may be initialized only as a result of: * If C is a reference type and a containing class declares a field C with the ACC_FLATTENABLE flag set ... 13. General question Would it make sense to disallow an Interface to define a field with ACC_FLATTENABLE? 14. Chapter 6: putfield "withe" -> "with the" I need to double-check - I was expecting a runtime, not a linktime NPE due to ACC_FLATTENABLE. Does linktime actually know the value you are writing? 15. Chapter 6: putstatic mentions null for value class type throws NPE, but I think that is intended to be if ACC_FLATTENABLE, and again - isn't this a run-time exception? 16. Chapter 6: new: if finds interface, abstract class or value class -> throws InstantiationError defaultvalue: if interface or abstract class, throws InstantiationError if object class, throws ICCE We can ask Dan Smith what he prefers here. Given it is ok to migrate object class to value class, but not value class to object class, perhaps it is appropriate to have asymmetric errors. 17. Chapter 6: withfield typo: vwithtfield =204 same question: NPE - isn't that a runtime exception > On Feb 5, 2018, at 2:05 PM, Frederic Parain wrote: > > Here?s an update of the JVMS draft: > > http://cr.openjdk.java.net/~fparain/L-world/L-World-JVMS-4.pdf > > with the following changes: > > * Renamed ACC_NON_NULLABLE with ACC_FLATTENABLE > * Removed restriction that ACC_FLATTENABLE cannot be used on static fields > * Added rules about circularity restrictions for fields with the ACC_FLATTENABLE flag set > * Added rules about initialization of classes of flattenable fields > > Fred From frederic.parain at oracle.com Tue Feb 6 01:29:05 2018 From: frederic.parain at oracle.com (Frederic Parain) Date: Mon, 5 Feb 2018 20:29:05 -0500 Subject: JVMS draft for L-world value types with support for nullability In-Reply-To: <717CE742-BB91-4DF9-9561-D05154591AE2@oracle.com> References: <8AD16BD6-2C65-4C4B-BBA1-4FEB99764209@oracle.com> <603DF4A9-2F9A-4FA9-8EF8-954991FBF358@oracle.com> <0209503F-E77B-458F-BD18-01452A075818@oracle.com> <37D843A0-D615-48E8-B00C-49049103DF21@oracle.com> <9B2F324F-F2EF-4702-90F1-CDC3B6BE7FF8@oracle.com> <717CE742-BB91-4DF9-9561-D05154591AE2@oracle.com> Message-ID: <02C2D0E4-4807-4136-B59C-CC2735C21C97@oracle.com> Karen, Thank you for this very detailed review. I?ve fixed all the issues (see comments below), and the updated draft is available here: http://cr.openjdk.java.net/~fparain/L-world/L-World-JVMS-4a.pdf Fred > On Feb 5, 2018, at 18:31, Karen Kinnear wrote: > > Frederic, > > Detailed review of the updated JVMS draft - many are just minor typos/edits. A couple > are bigger. Many thanks for the latest changes. > > thanks, > Karen > > 1. 2.4 Reference Types and Values > " whose all instances are identity-less and immutable" > "whose" -> "for which Fixed > > 2. 2.11.5 Instance Creation and Manipulation > 4th bullet: non-staticfields - need a space) > Fixed > 3. 4.1 The Classfile Structure > "If none of the ACC_VALUE_TYPE or the ACC_INTERFACE flag is set" > -> "If neither the ACC_VALUE_TYPE ?? Fixed > 4. 4.4.1 The CONSTANT_Class_info Structure > > Because arrays are objects, ... "but not the opcode new" - perhaps "but not the opcodes new or defaultvalue?. Fixed > > 5. 4.9.1. Static Constraints > getfield, putfield, getstatic, putstatic -- also withfield - must refer to a CONSTANT_FieldRef > Fixed > 6. 4.9.1 Static Constraints > operands of instanceof, checkcast, new ... > - also include defaultvalue > Fixed > 7. 4.9.1 Static Constraints > No defaulvalue instruction may reference a constant pool entry ... > - fix indentation Fixed > 8. 5.3.5 Deriving a Class from a class File Representation > 3. Note that if C is an interface it must have Object as its direct superclass, ... > change to "if C is an interface or a value class? > Fixed > 9. 5.3.5 Deriving a Class from a class File Representation > 3. "If the class or interface named as the direct superclass of C is in fact an interface, ... > change to "is in fact an interface or value class" -> ICCE > Fixed > > 10. 5.3.5 Deriving a Class from a class File Representation > 5. Change ACC_VALUE_TYPE flag to ACC_FLATTENABLE > Fixed > 5. Perhaps word: If C has any non-static field FC with the ACC_FLATTENABLE flag set, in order > to use F below in the exception case > > 5. Exceptions: > Rather than triggering a StackOverflwError, I was expecting a ClassCircularityError > * If any of the fields marked as ACC_FLATTENABLE is not in fact a value class, loading throws > an ICCE > * Otherwise, if any of the ACC_FLATTENABLE fields contains directly or indirectly the class C or the > class FC, loading throws a ClassCircularityError > > I?ve re-worked this section. It will have to be reviewed again. > 11. 5.3.5 Deriving a Class from a class File Representation > 5. So it looks as though you have made a change to allow ACC_FLATTENABLE on an object class. > I recognize there have been requests to support that in the future, but I think that will > require additional discussions before I would make a JVMS change. > My bad. This is not a change, just a leftover of a previous iterations I forgot to update. Current semantic is that it is illegal to set the ACC_FLATTENABLE flag for a field which is an object class. I?ve fixed this section too. > Perhaps a bit of non-normative text here say in italics? > i.e. we do not allow a value class to migrate to an object class > - intention is to allow declaration of a ACC_FLATTENABLE to imply non-nullable for > object classes as well as value classes? > - I would leave this out until we have had further detailed discussions I?d prefer to keep this idea in our discussion, and not write it down in the spec, to avoid making wrong expectations. > > 12. 5.5. Initialization > Thank you for adding step 8 to pre-initialize an ACC_FLATTENABLE field. > Can you also add this to the bullets under "may be initialized only as a result of: > > * If C is a reference type and a containing class declares a field C with the > ACC_FLATTENABLE flag set ? > Added > > 13. General question > Would it make sense to disallow an Interface to define a field with ACC_FLATTENABLE? Yes, if you want to prevent > > 14. Chapter 6: > putfield "withe" -> "with the? Fixed > I need to double-check - I was expecting a runtime, not a linktime NPE due to ACC_FLATTENABLE. > Does linktime actually know the value you are writing? You?re right, it?s a runtime exception. > > 15. Chapter 6: > putstatic > mentions null for value class type throws NPE, but I think that is intended to be if > ACC_FLATTENABLE, and again - isn't this a run-time exception? Fixed > > 16. Chapter 6: > new: if finds interface, abstract class or value class -> throws InstantiationError > defaultvalue: if interface or abstract class, throws InstantiationError > if object class, throws ICCE > > We can ask Dan Smith what he prefers here. Given it is ok to migrate object class to value class, but > not value class to object class, perhaps it is appropriate to have asymmetric errors. I will let Dan answer this one, I don?t have a preference or a good argument here. > > 17. Chapter 6: > withfield > typo: vwithtfield =204 > same question: NPE - isn't that a runtime exception > Fixed > > >> On Feb 5, 2018, at 2:05 PM, Frederic Parain wrote: >> >> Here?s an update of the JVMS draft: >> >> http://cr.openjdk.java.net/~fparain/L-world/L-World-JVMS-4.pdf >> >> with the following changes: >> >> * Renamed ACC_NON_NULLABLE with ACC_FLATTENABLE >> * Removed restriction that ACC_FLATTENABLE cannot be used on static fields >> * Added rules about circularity restrictions for fields with the ACC_FLATTENABLE flag set >> * Added rules about initialization of classes of flattenable fields >> >> Fred > From david.holmes at oracle.com Wed Feb 7 08:36:16 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 7 Feb 2018 18:36:16 +1000 Subject: [Nestmates] Spec updates to JVM TI, JDWP, JDI and java.lang.instrument for Class redefinition/retransformation Message-ID: <163f03e7-ae95-7f43-8420-57c20ef56b26@oracle.com> webrev: http://cr.openjdk.java.net/~dholmes/8191118/webrev/ I tried to produce specdiffs but specdiff seems broken. The links below at least highlight the changed text but you can't navigate directly to it. For java.lang.instrument the only change is in Instrumentation.java as per the webrev: - Specify that some attributes can be explicitly excluded from redefinition and retransformation - Specify the NestHost and NestMembers attributes are excluded JVM TI spec: http://cr.openjdk.java.net/~dholmes/8010319/specs/JVM-TI/diff.html - Specify that some attributes can be explicitly excluded from redefinition and retransformation - Specify the NestHost and NestMembers attributes are excluded - Add new error code JDWP spec: http://cr.openjdk.java.net/~dholmes/8010319/specs/JDWP/diff.html - Clarify what the "restricted redefinitions" are - Add changing the NestHost/NestMembers attributes to the set of restricted redefinitions - Add new error code --- For JDI the only change is in com/sun/jdi/VirtualMachine.java as per the webrev: - Add changing the NestHost/NestMembers attributes to the set of restricted redefinitions - clarify what "unrestricted redefinition" means - fix existing typo that refers to subclasses instead of superclasses Thanks, David ----- From karen.kinnear at oracle.com Wed Feb 7 14:35:18 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Wed, 7 Feb 2018 09:35:18 -0500 Subject: JVMS draft for L-world value types with support for nullability In-Reply-To: <37D843A0-D615-48E8-B00C-49049103DF21@oracle.com> References: <8AD16BD6-2C65-4C4B-BBA1-4FEB99764209@oracle.com> <603DF4A9-2F9A-4FA9-8EF8-954991FBF358@oracle.com> <0209503F-E77B-458F-BD18-01452A075818@oracle.com> <37D843A0-D615-48E8-B00C-49049103DF21@oracle.com> Message-ID: Paul, Just to make sure we are in sync for now: I think we are all in agreement that current Enums can not migrate to be value types: 1. enums have identity 2. enums have java.lang.Enum abstract class as super-class, not java.lang.Object 3. there is no clear default value 4. enums have mutable fields. What I think you are wondering about is if there is a role for a new kind of type, value-enums, that have value type characteristics. I totally agree with John that the is a future exercise. And if you can figure out a migration story in future, more power to you, but we are not designing value types around that requirement. A note - at least from the hotspot perspective, enums are not special-cased and as you can imagine we are trying to minimize special cases since they tend to be sources of bugs, so we would like to keep it that way. thanks, Karen > On Feb 1, 2018, at 2:05 PM, John Rose wrote: > > On Feb 1, 2018, at 5:24 PM, Paul Sandoz > wrote: >> >> an enum class to also explicitly be a value class > > Yes, it's probably doable, but we would have to work out the > migration story, and also figure out how to manage the Enum > supertype. Remember that one of the ongoing challenges > of VT's is the role of Object. For value-enums, I think we > would reprise the role for Enum. I'd rather have template > classes under my belt before tackling that, so I could make > Enum behave differently as an object vs. a value supertype. > From paul.sandoz at oracle.com Wed Feb 7 16:37:57 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 7 Feb 2018 08:37:57 -0800 Subject: JVMS draft for L-world value types with support for nullability In-Reply-To: References: <8AD16BD6-2C65-4C4B-BBA1-4FEB99764209@oracle.com> <603DF4A9-2F9A-4FA9-8EF8-954991FBF358@oracle.com> <0209503F-E77B-458F-BD18-01452A075818@oracle.com> <37D843A0-D615-48E8-B00C-49049103DF21@oracle.com> Message-ID: <5D41F308-28D0-4671-80FD-5DA57C09F908@oracle.com> > On Feb 7, 2018, at 6:35 AM, Karen Kinnear wrote: > > Paul, > > Just to make sure we are in sync for now: > We are. > I think we are all in agreement that current Enums can not migrate to be > value types: > 1. enums have identity > 2. enums have java.lang.Enum abstract class as super-class, not java.lang.Object > 3. there is no clear default value > 4. enums have mutable fields. > > What I think you are wondering about is if there is a role for a new kind of type, > value-enums, that have value type characteristics. Yes, it was a thought exercise which also conveniently pushed on some other areas like static fields. Thanks, Paul. > I totally agree with John > that the is a future exercise. And if you can figure out a migration story in > future, more power to you, but we are not designing value types around that > requirement. > > A note - at least from the hotspot perspective, enums are not special-cased > and as you can imagine we are trying to minimize special cases since they > tend to be sources of bugs, so we would like to keep it that way. > > thanks, > Karen > >> On Feb 1, 2018, at 2:05 PM, John Rose > wrote: >> >> On Feb 1, 2018, at 5:24 PM, Paul Sandoz > wrote: >>> >>> an enum class to also explicitly be a value class >> >> Yes, it's probably doable, but we would have to work out the >> migration story, and also figure out how to manage the Enum >> supertype. Remember that one of the ongoing challenges >> of VT's is the role of Object. For value-enums, I think we >> would reprise the role for Enum. I'd rather have template >> classes under my belt before tackling that, so I could make >> Enum behave differently as an object vs. a value supertype. >> > From daniel.smith at oracle.com Wed Feb 7 23:55:40 2018 From: daniel.smith at oracle.com (Dan Smith) Date: Wed, 7 Feb 2018 16:55:40 -0700 Subject: Final CONSTANT_Dynamic spec In-Reply-To: References: <80E6B9AF-AF53-416D-8710-5DCCED3A8EB5@oracle.com> Message-ID: <50F0A11C-476D-4C2B-8AE2-6BD2E27F891A@oracle.com> > On Feb 4, 2018, at 9:59 AM, Daniel Heidinga wrote: > > Dan, > > Can you clarify this sentence: > ---- > Note that an implementation is free to try to re-resolve X multiple times and keep looping until the stack actually overflows?once you've reached the nested X the first time, all further computation is not observable by users. > ---- > > In particular this statement "not observable by users" has some unpleasant implications. Does that mean that side-effects that occur during the re-resolution should be undone?That the StackOverflowError's backtrace should only include a single loop of the re-resolution? > > There are a lot of ways that users could observe further computation and if the JVM needs to detect and prevent them from seeing the effects, this actually mandates early detection. Was that the intention or am I being overly pedantic? The dashes (-----) are to delineate the actual spec. The sentence you're asking about is just me explaining for this mailing list how this rule interacts with our current implementation strategy. So, nothing being mandated here. That said, what side-effects are you concerned about? My claim is that re-resolution in this scenario would not, for example, trigger any class loading or bootstrap invocations. My use if "observable to users" was not meant to include things that are implementation details anyway, like stack traces, CPU usage, debugger interactions, etc. ?Dan From daniel.smith at oracle.com Wed Feb 7 23:48:28 2018 From: daniel.smith at oracle.com (Dan Smith) Date: Wed, 7 Feb 2018 16:48:28 -0700 Subject: ClassLoaders throwing exceptions Message-ID: We tried, over a few rounds of the condy spec, to make claims about what kind of exceptions can be thrown during resolution. Specifically, that anything thrown will be an Error. Bootstrap methods execute arbitrary code, so we wrap any non-Error that gets thrown in a BootstrapMethodError. I tried finding the equivalent rule for ClassLoaders, and could not. With some testing, it turns out that, in fact, exceptions thrown by ClassLoaders are propagated without wrapping. See the test below. So, it turns out, resolution (of classes) can result in *any* kind of exception being thrown. This is very old behavior, and I'm pretty sure we don't want to perturb it, but I thought it would be good to raise awareness of it. I'll make sure JVMS, as modified for condy, treats it appropriately. ?Dan Test code: ~~~ public class Test { public static void main(String... args) throws Exception { PreemptingClassLoader l = new PreemptingClassLoader("A", "B"); Class c = l.loadClass("A"); Runnable r = (Runnable) c.getDeclaredConstructor().newInstance(); r.run(); } } ~~~ public class A implements Runnable { public void run() { System.out.println(B.foo); } } ~~~ public class B { public static String foo = "foo"; } ~~~ import java.util.*; import java.io.*; public class PreemptingClassLoader extends ClassLoader { private final Set names = new HashSet<>(); public PreemptingClassLoader(String... names) { for (String n : names) this.names.add(n); } protected Class loadClass(String name, boolean resolve) throws ClassNotFoundException { if (!names.contains(name)) return super.loadClass(name, resolve); Class result = findLoadedClass(name); if (result == null) { if (name.equals("B")) throw new RuntimeException("Hi, mom"); // INSERTED TO TRIGGER EXCEPTION String filename = name.replace('.', '/') + ".class"; try (InputStream data = getResourceAsStream(filename)) { if (data == null) throw new ClassNotFoundException(); try (ByteArrayOutputStream buffer = new ByteArrayOutputStream()) { int b; do { b = data.read(); if (b >= 0) buffer.write(b); } while (b >= 0); byte[] bytes = buffer.toByteArray(); result = defineClass(name, bytes, 0, bytes.length); } } catch (IOException e) { throw new ClassNotFoundException("Error reading class file", e); } } if (resolve) resolveClass(result); return result; } } From david.holmes at oracle.com Mon Feb 12 01:19:19 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Feb 2018 11:19:19 +1000 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: <56C82B88-0F86-44F8-9E57-4669260257B3@oracle.com> References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> <30772B5A-46C4-43D1-A78D-B598EB011C73@oracle.com> <4824a94b-81dd-8ede-6b2d-dda1ae8b6d48@oracle.com> <56C82B88-0F86-44F8-9E57-4669260257B3@oracle.com> Message-ID: <2540bae2-3ae9-82ee-7a4b-2f52bb6d9a19@oracle.com> Hi Paul, On 1/02/2018 2:48 AM, Paul Sandoz wrote: >> On Jan 30, 2018, at 7:22 PM, David Holmes wrote: >> On 31/01/2018 12:24 PM, Paul Sandoz wrote: >>>> On Jan 30, 2018, at 1:55 AM, David Holmes > wrote: >>>> MethodHandle API Changes: >>>> >>>> - java/lang/invoke/MethodHandle.java >>>> * A non-virtual method handle to a specific virtual method implementation >>>> * can also be created. These do not perform virtual lookup based on >>>> * receiver type. Such a method handle simulates the effect of >>>> - * an {@code invokespecial} instruction to the same method. >>>> + * an {@code invokespecial} instruction to the same non-private method; >>>> + * or an {@code invokevirtual} or {@code invokeinterface} instruction to the >>>> + * same private method (as applicable). >>>> >>>> I tried to clarify that non-virtual invocations are not limited to invokespecial - as private invocations via invokevirtual or invokeinterface are also non-virtual. >>>> >>>> >>> Why s/same method/same non-private method/ for the invokespecial? >>> It?s possible to look up a private method within the same class using Lookup.invokespecial and invoke it (and also look up a private constructor and invoke it). >> >> Yes you are right, but this text is not trying to describe how an invokespecial might be used, but rather how "A non-virtual method handle to a specific virtual method implementation can also be created.? >> > > Ok, i see now. > > >> But the notion of "specific virtual method implementation" is perhaps not applicable to private methods in the first place. >> >> Perhaps I just need to remove this change altogether and leave it as-is. >> > > I would be inclined just to leave it as is. I decided to restore the original text and then add the following - as I do want to make it clear that non-virtual is not restricted to invokespecial: * A non-virtual method handle can also be created to simulate the effect * of an {@code invokevirtual} or {@code invokeinterface} instruction on * a private method (as applicable). I have updated specdiffs for all of the changes here: http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang/overview-summary.html http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.invoke/overview-summary.html http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.reflect/overview-summary.html Please ignore the spurious "version" changes. Thanks, David ----- > Paul. > From paul.sandoz at oracle.com Mon Feb 12 18:52:47 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 12 Feb 2018 10:52:47 -0800 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: <2540bae2-3ae9-82ee-7a4b-2f52bb6d9a19@oracle.com> References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> <30772B5A-46C4-43D1-A78D-B598EB011C73@oracle.com> <4824a94b-81dd-8ede-6b2d-dda1ae8b6d48@oracle.com> <56C82B88-0F86-44F8-9E57-4669260257B3@oracle.com> <2540bae2-3ae9-82ee-7a4b-2f52bb6d9a19@oracle.com> Message-ID: <8750AD50-8B8D-4822-AF21-7A5FE89B0462@oracle.com> > On Feb 11, 2018, at 5:19 PM, David Holmes wrote: > > Hi Paul, > > On 1/02/2018 2:48 AM, Paul Sandoz wrote: >>> On Jan 30, 2018, at 7:22 PM, David Holmes wrote: >>> On 31/01/2018 12:24 PM, Paul Sandoz wrote: >>>>> On Jan 30, 2018, at 1:55 AM, David Holmes > wrote: >>>>> MethodHandle API Changes: >>>>> >>>>> - java/lang/invoke/MethodHandle.java >>>>> * A non-virtual method handle to a specific virtual method implementation >>>>> * can also be created. These do not perform virtual lookup based on >>>>> * receiver type. Such a method handle simulates the effect of >>>>> - * an {@code invokespecial} instruction to the same method. >>>>> + * an {@code invokespecial} instruction to the same non-private method; >>>>> + * or an {@code invokevirtual} or {@code invokeinterface} instruction to the >>>>> + * same private method (as applicable). >>>>> >>>>> I tried to clarify that non-virtual invocations are not limited to invokespecial - as private invocations via invokevirtual or invokeinterface are also non-virtual. >>>>> >>>>> >>>> Why s/same method/same non-private method/ for the invokespecial? >>>> It?s possible to look up a private method within the same class using Lookup.invokespecial and invoke it (and also look up a private constructor and invoke it). >>> >>> Yes you are right, but this text is not trying to describe how an invokespecial might be used, but rather how "A non-virtual method handle to a specific virtual method implementation can also be created.? >>> >> Ok, i see now. >>> But the notion of "specific virtual method implementation" is perhaps not applicable to private methods in the first place. >>> >>> Perhaps I just need to remove this change altogether and leave it as-is. >>> >> I would be inclined just to leave it as is. > > I decided to restore the original text and then add the following - as I do want to make it clear that non-virtual is not restricted to invokespecial: > > * A non-virtual method handle can also be created to simulate the effect > * of an {@code invokevirtual} or {@code invokeinterface} instruction on > * a private method (as applicable). > > I have updated specdiffs for all of the changes here: > > http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang/overview-summary.html If a class has no nestmates is it implicitly a nest of itself? (is that considered something valid?) Class c = ? // some class with no nest mates assert c.getNestHost() == c assert c.isNestmateOf(c) assert Stream.of(c.getNestMembers()).anyMatch(_c -> _c == c); ? If class nh is a nest host with nest mates (in addition to itself): Class nh = ? // some class that is a nest host assert nh.getNestHost() == c assert nh.isNestmateOf(nh) assert Stream.of(nh.getNestMembers()).anyMatch(_c -> _c == nh); assert Stream.of(nh.getNestMembers()).allMatch(_c -> _c.isNestmakeOf(nh)); assert Stream.of(nh.getNestMembers()).allMatch( _c -> Stream.of(nh.getNestMembers()).allMatch(__c -> __c.isNestmakeOf(_c))); ? getNestMembers ? "The list of nest members in the classfile is permitted to contain duplicates, or to explicitly include the nest host. It is not required that an implementation of this method removes these duplicates." The "or to explicitly include the nest host? suggests it might not include the nest host, but a prior statement says it will be present in the zeroth element. Why the ambiguity over duplicates? is this motivated by performance? this may just push the cost to clients that have to always remove duplicates to function reliably and may be cause bugs if duplicates are rare and induced by certain relationships or loading patterns. My inclination would be for the returned array to not contain duplicates. > http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.invoke/overview-summary.html > http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.reflect/overview-summary.html > Looks good to me. Paul. > Please ignore the spurious "version" changes. > > Thanks, > David > ----- > >> Paul. From john.r.rose at oracle.com Mon Feb 12 19:20:02 2018 From: john.r.rose at oracle.com (John Rose) Date: Mon, 12 Feb 2018 11:20:02 -0800 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: <8750AD50-8B8D-4822-AF21-7A5FE89B0462@oracle.com> References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> <30772B5A-46C4-43D1-A78D-B598EB011C73@oracle.com> <4824a94b-81dd-8ede-6b2d-dda1ae8b6d48@oracle.com> <56C82B88-0F86-44F8-9E57-4669260257B3@oracle.com> <2540bae2-3ae9-82ee-7a4b-2f52bb6d9a19@oracle.com> <8750AD50-8B8D-4822-AF21-7A5FE89B0462@oracle.com> Message-ID: <95A25755-A73D-4A73-AABC-F63B47485449@oracle.com> On Feb 12, 2018, at 10:52 AM, Paul Sandoz wrote: > > If a class has no nestmates is it implicitly a nest of itself? Logically, every class is exactly one nest. In the absence of Nest* attributes, the nest is a singleton. There are sometimes secondary reasons to distinguish an explicit singleton from an implicit one, reasons which don't affect member access control. The Lookup::in API wants to be sensitive the distinction in some way, so that it can use InnerClasses for old code, but ditch that in favor of NestMates in new code. We could ask reflection to show that distinction directly, or (maybe better) we could have Lookup::in grub it directly out of some secret channel. I'm on the fence about this. ? John From david.holmes at oracle.com Mon Feb 12 21:55:49 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Feb 2018 07:55:49 +1000 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: <8750AD50-8B8D-4822-AF21-7A5FE89B0462@oracle.com> References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> <30772B5A-46C4-43D1-A78D-B598EB011C73@oracle.com> <4824a94b-81dd-8ede-6b2d-dda1ae8b6d48@oracle.com> <56C82B88-0F86-44F8-9E57-4669260257B3@oracle.com> <2540bae2-3ae9-82ee-7a4b-2f52bb6d9a19@oracle.com> <8750AD50-8B8D-4822-AF21-7A5FE89B0462@oracle.com> Message-ID: <786c8570-bc46-c499-e22d-3deca90cba91@oracle.com> Hi Paul, On 13/02/2018 4:52 AM, Paul Sandoz wrote: > > >> On Feb 11, 2018, at 5:19 PM, David Holmes wrote: >> >> Hi Paul, >> >> On 1/02/2018 2:48 AM, Paul Sandoz wrote: >>>> On Jan 30, 2018, at 7:22 PM, David Holmes wrote: >>>> On 31/01/2018 12:24 PM, Paul Sandoz wrote: >>>>>> On Jan 30, 2018, at 1:55 AM, David Holmes > wrote: >>>>>> MethodHandle API Changes: >>>>>> >>>>>> - java/lang/invoke/MethodHandle.java >>>>>> * A non-virtual method handle to a specific virtual method implementation >>>>>> * can also be created. These do not perform virtual lookup based on >>>>>> * receiver type. Such a method handle simulates the effect of >>>>>> - * an {@code invokespecial} instruction to the same method. >>>>>> + * an {@code invokespecial} instruction to the same non-private method; >>>>>> + * or an {@code invokevirtual} or {@code invokeinterface} instruction to the >>>>>> + * same private method (as applicable). >>>>>> >>>>>> I tried to clarify that non-virtual invocations are not limited to invokespecial - as private invocations via invokevirtual or invokeinterface are also non-virtual. >>>>>> >>>>>> >>>>> Why s/same method/same non-private method/ for the invokespecial? >>>>> It?s possible to look up a private method within the same class using Lookup.invokespecial and invoke it (and also look up a private constructor and invoke it). >>>> >>>> Yes you are right, but this text is not trying to describe how an invokespecial might be used, but rather how "A non-virtual method handle to a specific virtual method implementation can also be created.? >>>> >>> Ok, i see now. >>>> But the notion of "specific virtual method implementation" is perhaps not applicable to private methods in the first place. >>>> >>>> Perhaps I just need to remove this change altogether and leave it as-is. >>>> >>> I would be inclined just to leave it as is. >> >> I decided to restore the original text and then add the following - as I do want to make it clear that non-virtual is not restricted to invokespecial: >> >> * A non-virtual method handle can also be created to simulate the effect >> * of an {@code invokevirtual} or {@code invokeinterface} instruction on >> * a private method (as applicable). >> >> I have updated specdiffs for all of the changes here: >> >> http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang/overview-summary.html > > If a class has no nestmates is it implicitly a nest of itself? (is that considered something valid?) Yes - as John said. Every class is a member of some nest, even if that is a singelton nest consistenting only of itself. That is the default if there is no nesthost attribute. > Class c = ? // some class with no nest mates > assert c.getNestHost() == c > assert c.isNestmateOf(c) > assert Stream.of(c.getNestMembers()).anyMatch(_c -> _c == c); > > ? Yes. > If class nh is a nest host with nest mates (in addition to itself): > > Class nh = ? // some class that is a nest host > > assert nh.getNestHost() == c No, nh.getNestHost == nh > assert nh.isNestmateOf(nh) Yes. > assert Stream.of(nh.getNestMembers()).anyMatch(_c -> _c == nh); > assert Stream.of(nh.getNestMembers()).allMatch(_c -> _c.isNestmakeOf(nh)); > assert Stream.of(nh.getNestMembers()).allMatch( > _c -> Stream.of(nh.getNestMembers()).allMatch(__c -> __c.isNestmakeOf(_c))); > > ? Yes - ignoring your typo and assuming I'm reading those expressions right. > getNestMembers > ? > > "The list of nest members in the classfile is permitted to contain duplicates, or to explicitly include the nest host. It is not required that an implementation of this method removes these duplicates." > > The "or to explicitly include the nest host? suggests it might not include the nest host, but a prior statement says it will be present in the zeroth element. The "or" pertains to the list of nest members in the classfile - ie the contents of the NestMembers attribute. The returned array of nestmembers will always contain the nesthost as the zeroeth element, but may also contain it somewhere else if the classfile explicitly listed it in nestmembers. > Why the ambiguity over duplicates? is this motivated by performance? this may just push the cost to clients that have to always remove duplicates to function reliably and may be cause bugs if duplicates are rare and induced by certain relationships or loading patterns. My inclination would be for the returned array to not contain duplicates. Yes performance. Having to check for duplicates adds a cost to every single well formed call to this API to account for something that the specification allows to happen but which we don't expect to happen and which javac will never produce. This has all been discussed previously. > >> http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.invoke/overview-summary.html >> http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.reflect/overview-summary.html >> > > Looks good to me. Thanks, David ----- > Paul. > >> Please ignore the spurious "version" changes. >> >> Thanks, >> David >> ----- >> >>> Paul. > From paul.sandoz at oracle.com Tue Feb 13 01:45:18 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 12 Feb 2018 17:45:18 -0800 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: <786c8570-bc46-c499-e22d-3deca90cba91@oracle.com> References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> <30772B5A-46C4-43D1-A78D-B598EB011C73@oracle.com> <4824a94b-81dd-8ede-6b2d-dda1ae8b6d48@oracle.com> <56C82B88-0F86-44F8-9E57-4669260257B3@oracle.com> <2540bae2-3ae9-82ee-7a4b-2f52bb6d9a19@oracle.com> <8750AD50-8B8D-4822-AF21-7A5FE89B0462@oracle.com> <786c8570-bc46-c499-e22d-3deca90cba91@oracle.com> Message-ID: <152030BC-93F1-475E-ACC4-C7DADB7BF30D@oracle.com> > On Feb 12, 2018, at 1:55 PM, David Holmes wrote: >> getNestMembers >> ? >> "The list of nest members in the classfile is permitted to contain duplicates, or to explicitly include the nest host. It is not required that an implementation of this method removes these duplicates." >> The "or to explicitly include the nest host? suggests it might not include the nest host, but a prior statement says it will be present in the zeroth element. > > The "or" pertains to the list of nest members in the classfile - ie the contents of the NestMembers attribute. The returned array of nestmembers will always contain the nesthost as the zeroeth element, but may also contain it somewhere else if the classfile explicitly listed it in nestmembers. > I see, it?s easy to misread, well i did :-) Perhaps call out the attribute and provide a link to the JVMS? >> Why the ambiguity over duplicates? is this motivated by performance? this may just push the cost to clients that have to always remove duplicates to function reliably and may be cause bugs if duplicates are rare and induced by certain relationships or loading patterns. My inclination would be for the returned array to not contain duplicates. > > Yes performance. Having to check for duplicates adds a cost to every single well formed call to this API to account for something that the specification allows to happen but which we don't expect to happen and which javac will never produce. This has all been discussed previously. > Ok, it?s unfortunate that the cost will be placed on the developer who has to code defensively in case there might be duplicates i.e. the performance cost is pushed to an unbounded set of places (or places where bugs may lurk). Paul. From david.holmes at oracle.com Tue Feb 13 02:24:49 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Feb 2018 12:24:49 +1000 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: <152030BC-93F1-475E-ACC4-C7DADB7BF30D@oracle.com> References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> <30772B5A-46C4-43D1-A78D-B598EB011C73@oracle.com> <4824a94b-81dd-8ede-6b2d-dda1ae8b6d48@oracle.com> <56C82B88-0F86-44F8-9E57-4669260257B3@oracle.com> <2540bae2-3ae9-82ee-7a4b-2f52bb6d9a19@oracle.com> <8750AD50-8B8D-4822-AF21-7A5FE89B0462@oracle.com> <786c8570-bc46-c499-e22d-3deca90cba91@oracle.com> <152030BC-93F1-475E-ACC4-C7DADB7BF30D@oracle.com> Message-ID: <0435cd18-8a83-cf1c-2b07-2b5e52563ae8@oracle.com> On 13/02/2018 11:45 AM, Paul Sandoz wrote: >> On Feb 12, 2018, at 1:55 PM, David Holmes wrote: >>> getNestMembers >>> ? >>> "The list of nest members in the classfile is permitted to contain duplicates, or to explicitly include the nest host. It is not required that an implementation of this method removes these duplicates." >>> The "or to explicitly include the nest host? suggests it might not include the nest host, but a prior statement says it will be present in the zeroth element. >> >> The "or" pertains to the list of nest members in the classfile - ie the contents of the NestMembers attribute. The returned array of nestmembers will always contain the nesthost as the zeroeth element, but may also contain it somewhere else if the classfile explicitly listed it in nestmembers. >> > > I see, it?s easy to misread, well i did :-) Perhaps call out the attribute and provide a link to the JVMS? I can add a link to JVMS if that is what we normally do. As for misreading ... the subject of the sentence is "The list of nest members in the classfile". ;-) > >>> Why the ambiguity over duplicates? is this motivated by performance? this may just push the cost to clients that have to always remove duplicates to function reliably and may be cause bugs if duplicates are rare and induced by certain relationships or loading patterns. My inclination would be for the returned array to not contain duplicates. >> >> Yes performance. Having to check for duplicates adds a cost to every single well formed call to this API to account for something that the specification allows to happen but which we don't expect to happen and which javac will never produce. This has all been discussed previously. >> > > Ok, it?s unfortunate that the cost will be placed on the developer who has to code defensively in case there might be duplicates i.e. the performance cost is pushed to an unbounded set of places (or places where bugs may lurk). There's really no expectation that the developer will need to program defensively here as we don't expect compilers to produce such classfiles. But I, for one, prefer a "user pays" scheme over an "everyone pays" scheme (which is what disallowing duplicates would also be). David > Paul. > From paul.sandoz at oracle.com Tue Feb 13 02:39:24 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 12 Feb 2018 18:39:24 -0800 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: <0435cd18-8a83-cf1c-2b07-2b5e52563ae8@oracle.com> References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> <30772B5A-46C4-43D1-A78D-B598EB011C73@oracle.com> <4824a94b-81dd-8ede-6b2d-dda1ae8b6d48@oracle.com> <56C82B88-0F86-44F8-9E57-4669260257B3@oracle.com> <2540bae2-3ae9-82ee-7a4b-2f52bb6d9a19@oracle.com> <8750AD50-8B8D-4822-AF21-7A5FE89B0462@oracle.com> <786c8570-bc46-c499-e22d-3deca90cba91@oracle.com> <152030BC-93F1-475E-ACC4-C7DADB7BF30D@oracle.com> <0435cd18-8a83-cf1c-2b07-2b5e52563ae8@oracle.com> Message-ID: > On Feb 12, 2018, at 6:24 PM, David Holmes wrote: > > On 13/02/2018 11:45 AM, Paul Sandoz wrote: >>> On Feb 12, 2018, at 1:55 PM, David Holmes wrote: >>>> getNestMembers >>>> ? >>>> "The list of nest members in the classfile is permitted to contain duplicates, or to explicitly include the nest host. It is not required that an implementation of this method removes these duplicates." >>>> The "or to explicitly include the nest host? suggests it might not include the nest host, but a prior statement says it will be present in the zeroth element. >>> >>> The "or" pertains to the list of nest members in the classfile - ie the contents of the NestMembers attribute. The returned array of nestmembers will always contain the nesthost as the zeroeth element, but may also contain it somewhere else if the classfile explicitly listed it in nestmembers. >>> >> I see, it?s easy to misread, well i did :-) Perhaps call out the attribute and provide a link to the JVMS? > > I can add a link to JVMS if that is what we normally do. As for misreading ... the subject of the sentence is "The list of nest members in the classfile". ;-) > I know :-) most developers will not be thinking at the classfile level so some link for those that are interested or care is helpful i think. >>>> Why the ambiguity over duplicates? is this motivated by performance? this may just push the cost to clients that have to always remove duplicates to function reliably and may be cause bugs if duplicates are rare and induced by certain relationships or loading patterns. My inclination would be for the returned array to not contain duplicates. >>> >>> Yes performance. Having to check for duplicates adds a cost to every single well formed call to this API to account for something that the specification allows to happen but which we don't expect to happen and which javac will never produce. This has all been discussed previously. >>> >> Ok, it?s unfortunate that the cost will be placed on the developer who has to code defensively in case there might be duplicates i.e. the performance cost is pushed to an unbounded set of places (or places where bugs may lurk). > > There's really no expectation that the developer will need to program defensively here as we don't expect compilers to produce such classfiles. But the developer will not know that since they will be reading the JavaDoc. > But I, for one, prefer a "user pays" scheme over an "everyone pays" scheme (which is what disallowing duplicates would also be). It?s an awkward situation, experience suggests this type of thing bites back later on, so my inclination is to take the hit in the JDK and not return dups. The implementation might be able to cache information on the internal ReflectionData class. Paul. From david.holmes at oracle.com Tue Feb 13 02:51:49 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Feb 2018 12:51:49 +1000 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> <30772B5A-46C4-43D1-A78D-B598EB011C73@oracle.com> <4824a94b-81dd-8ede-6b2d-dda1ae8b6d48@oracle.com> <56C82B88-0F86-44F8-9E57-4669260257B3@oracle.com> <2540bae2-3ae9-82ee-7a4b-2f52bb6d9a19@oracle.com> <8750AD50-8B8D-4822-AF21-7A5FE89B0462@oracle.com> <786c8570-bc46-c499-e22d-3deca90cba91@oracle.com> <152030BC-93F1-475E-ACC4-C7DADB7BF30D@oracle.com> <0435cd18-8a83-cf1c-2b07-2b5e52563ae8@oracle.com> Message-ID: <0fa3e841-d94d-3a71-c7c1-5bcc2e9f77de@oracle.com> On 13/02/2018 12:39 PM, Paul Sandoz wrote: >> On Feb 12, 2018, at 6:24 PM, David Holmes wrote: >> >> On 13/02/2018 11:45 AM, Paul Sandoz wrote: >>>> On Feb 12, 2018, at 1:55 PM, David Holmes wrote: >>>>> getNestMembers >>>>> ? >>>>> "The list of nest members in the classfile is permitted to contain duplicates, or to explicitly include the nest host. It is not required that an implementation of this method removes these duplicates." >>>>> The "or to explicitly include the nest host? suggests it might not include the nest host, but a prior statement says it will be present in the zeroth element. >>>> >>>> The "or" pertains to the list of nest members in the classfile - ie the contents of the NestMembers attribute. The returned array of nestmembers will always contain the nesthost as the zeroeth element, but may also contain it somewhere else if the classfile explicitly listed it in nestmembers. >>>> >>> I see, it?s easy to misread, well i did :-) Perhaps call out the attribute and provide a link to the JVMS? >> >> I can add a link to JVMS if that is what we normally do. As for misreading ... the subject of the sentence is "The list of nest members in the classfile". ;-) >> > > I know :-) most developers will not be thinking at the classfile level so some link for those that are interested or care is helpful i think. Okay will see what I can reasonably add. > >>>>> Why the ambiguity over duplicates? is this motivated by performance? this may just push the cost to clients that have to always remove duplicates to function reliably and may be cause bugs if duplicates are rare and induced by certain relationships or loading patterns. My inclination would be for the returned array to not contain duplicates. >>>> >>>> Yes performance. Having to check for duplicates adds a cost to every single well formed call to this API to account for something that the specification allows to happen but which we don't expect to happen and which javac will never produce. This has all been discussed previously. >>>> >>> Ok, it?s unfortunate that the cost will be placed on the developer who has to code defensively in case there might be duplicates i.e. the performance cost is pushed to an unbounded set of places (or places where bugs may lurk). >> >> There's really no expectation that the developer will need to program defensively here as we don't expect compilers to produce such classfiles. > > But the developer will not know that since they will be reading the JavaDoc. > > >> But I, for one, prefer a "user pays" scheme over an "everyone pays" scheme (which is what disallowing duplicates would also be). > > It?s an awkward situation, experience suggests this type of thing bites back later on, so my inclination is to take the hit in the JDK and not return dups. The implementation might be able to cache information on the internal ReflectionData class. Please see John's response on this: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2017-December/000465.html Thanks, David > Paul. > From david.holmes at oracle.com Tue Feb 13 04:04:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Feb 2018 14:04:12 +1000 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: <0fa3e841-d94d-3a71-c7c1-5bcc2e9f77de@oracle.com> References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> <30772B5A-46C4-43D1-A78D-B598EB011C73@oracle.com> <4824a94b-81dd-8ede-6b2d-dda1ae8b6d48@oracle.com> <56C82B88-0F86-44F8-9E57-4669260257B3@oracle.com> <2540bae2-3ae9-82ee-7a4b-2f52bb6d9a19@oracle.com> <8750AD50-8B8D-4822-AF21-7A5FE89B0462@oracle.com> <786c8570-bc46-c499-e22d-3deca90cba91@oracle.com> <152030BC-93F1-475E-ACC4-C7DADB7BF30D@oracle.com> <0435cd18-8a83-cf1c-2b07-2b5e52563ae8@oracle.com> <0fa3e841-d94d-3a71-c7c1-5bcc2e9f77de@oracle.com> Message-ID: On 13/02/2018 12:51 PM, David Holmes wrote: > On 13/02/2018 12:39 PM, Paul Sandoz wrote: >>> On Feb 12, 2018, at 6:24 PM, David Holmes >>> wrote: >>> >>> On 13/02/2018 11:45 AM, Paul Sandoz wrote: >>>>> On Feb 12, 2018, at 1:55 PM, David Holmes >>>>> wrote: >>>>>> getNestMembers >>>>>> ? >>>>>> "The list of nest members in the classfile is permitted to contain >>>>>> duplicates, or to explicitly include the nest host. It is not >>>>>> required that an implementation of this method removes these >>>>>> duplicates." >>>>>> The "or to explicitly include the nest host? suggests it might not >>>>>> include the nest host, but a prior statement says it will be >>>>>> present in the zeroth element. >>>>> >>>>> The "or" pertains to the list of nest members in the classfile - ie >>>>> the contents of the NestMembers attribute. The returned array of >>>>> nestmembers will always contain the nesthost as the zeroeth >>>>> element, but may also contain it somewhere else if the classfile >>>>> explicitly listed it in nestmembers. >>>>> >>>> I see, it?s easy to misread, well i did :-) Perhaps call out the >>>> attribute and provide a link to the JVMS? >>> >>> I can add a link to JVMS if that is what we normally do. As for >>> misreading ... the subject of the sentence is "The list of nest >>> members in the classfile". ;-) >>> >> >> I know :-) most developers will not be thinking at the classfile level >> so some link for those that are interested or care is helpful i think. > > Okay will see what I can reasonably add. Added JVMS ref: *

The list of nest members in the classfile (JVMS 4.1) is permitted to * contain duplicates, or to explicitly include the nest host. David ----- > >> >>>>>> Why the ambiguity over duplicates? is this motivated by >>>>>> performance? this may just push the cost to clients that have to >>>>>> always remove duplicates to function reliably and may be cause >>>>>> bugs if duplicates are rare and induced by certain relationships >>>>>> or loading patterns. My inclination would be for the returned >>>>>> array to not contain duplicates. >>>>> >>>>> Yes performance. Having to check for duplicates adds a cost to >>>>> every single well formed call to this API to account for something >>>>> that the specification allows to happen but which we don't expect >>>>> to happen and which javac will never produce. This has all been >>>>> discussed previously. >>>>> >>>> Ok, it?s unfortunate that the cost will be placed on the developer >>>> who has to code defensively in case there might be duplicates i.e. >>>> the performance cost is pushed to an unbounded set of places (or >>>> places where bugs may lurk). >>> >>> There's really no expectation that the developer will need to program >>> defensively here as we don't expect compilers to produce such >>> classfiles. >> >> But the developer will not know that since they will be reading the >> JavaDoc. >> >> >>> But I, for one, prefer a "user pays" scheme over an "everyone pays" >>> scheme (which is what disallowing duplicates would also be). >> >> It?s an awkward situation, experience suggests this type of thing >> bites back later on, so my inclination is to take the hit in the JDK >> and not return dups. The implementation might be able to cache >> information on the internal ReflectionData class. > > Please see John's response on this: > > http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2017-December/000465.html > > > Thanks, > David > >> Paul. >> From karen.kinnear at oracle.com Tue Feb 13 15:59:46 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Tue, 13 Feb 2018 10:59:46 -0500 Subject: Updated Valhalla LWorld proposal Message-ID: As part of the Valhalla EG meeting tomorrow, Feb 14, we would like to review the LWorld Value Types latest proposal: http://cr.openjdk.java.net/~acorn/LWorldValueTypesFeb13.pdf And the latest rough draft JVMS: http://cr.openjdk.java.net/~fparain/L-world/L-World-JVMS-4b.pdf thanks, Karen From paul.sandoz at oracle.com Tue Feb 13 16:45:18 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 13 Feb 2018 08:45:18 -0800 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> <30772B5A-46C4-43D1-A78D-B598EB011C73@oracle.com> <4824a94b-81dd-8ede-6b2d-dda1ae8b6d48@oracle.com> <56C82B88-0F86-44F8-9E57-4669260257B3@oracle.com> <2540bae2-3ae9-82ee-7a4b-2f52bb6d9a19@oracle.com> <8750AD50-8B8D-4822-AF21-7A5FE89B0462@oracle.com> <786c8570-bc46-c499-e22d-3deca90cba91@oracle.com> <152030BC-93F1-475E-ACC4-C7DADB7BF30D@oracle.com> <0435cd18-8a83-cf1c-2b07-2b5e52563ae8@oracle.com> <0fa3e841-d94d-3a71-c7c1-5bcc2e9f77de@oracle.com> Message-ID: <2A97DD53-E39A-4539-8B8C-1BB7DC4A7272@oracle.com> > On Feb 12, 2018, at 8:04 PM, David Holmes wrote: > > On 13/02/2018 12:51 PM, David Holmes wrote: >> On 13/02/2018 12:39 PM, Paul Sandoz wrote: >>>> On Feb 12, 2018, at 6:24 PM, David Holmes wrote: >>>> >>>> On 13/02/2018 11:45 AM, Paul Sandoz wrote: >>>>>> On Feb 12, 2018, at 1:55 PM, David Holmes wrote: >>>>>>> getNestMembers >>>>>>> ? >>>>>>> "The list of nest members in the classfile is permitted to contain duplicates, or to explicitly include the nest host. It is not required that an implementation of this method removes these duplicates." >>>>>>> The "or to explicitly include the nest host? suggests it might not include the nest host, but a prior statement says it will be present in the zeroth element. >>>>>> >>>>>> The "or" pertains to the list of nest members in the classfile - ie the contents of the NestMembers attribute. The returned array of nestmembers will always contain the nesthost as the zeroeth element, but may also contain it somewhere else if the classfile explicitly listed it in nestmembers. >>>>>> >>>>> I see, it?s easy to misread, well i did :-) Perhaps call out the attribute and provide a link to the JVMS? >>>> >>>> I can add a link to JVMS if that is what we normally do. As for misreading ... the subject of the sentence is "The list of nest members in the classfile". ;-) >>>> >>> >>> I know :-) most developers will not be thinking at the classfile level so some link for those that are interested or care is helpful i think. >> Okay will see what I can reasonably add. > > Added JVMS ref: > > *

The list of nest members in the classfile (JVMS 4.1) is permitted to > * contain duplicates, or to explicitly include the nest host. > Ok, you can also add @jvms, see examples in StackWalker (alas JavaDoc does not support generating links to the specification). Paul. From paul.sandoz at oracle.com Tue Feb 13 16:53:22 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 13 Feb 2018 08:53:22 -0800 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: <0fa3e841-d94d-3a71-c7c1-5bcc2e9f77de@oracle.com> References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> <30772B5A-46C4-43D1-A78D-B598EB011C73@oracle.com> <4824a94b-81dd-8ede-6b2d-dda1ae8b6d48@oracle.com> <56C82B88-0F86-44F8-9E57-4669260257B3@oracle.com> <2540bae2-3ae9-82ee-7a4b-2f52bb6d9a19@oracle.com> <8750AD50-8B8D-4822-AF21-7A5FE89B0462@oracle.com> <786c8570-bc46-c499-e22d-3deca90cba91@oracle.com> <152030BC-93F1-475E-ACC4-C7DADB7BF30D@oracle.com> <0435cd18-8a83-cf1c-2b07-2b5e52563ae8@oracle.com> <0fa3e841-d94d-3a71-c7c1-5bcc2e9f77de@oracle.com> Message-ID: > On Feb 12, 2018, at 6:51 PM, David Holmes wrote: >>> But I, for one, prefer a "user pays" scheme over an "everyone pays" scheme (which is what disallowing duplicates would also be). >> It?s an awkward situation, experience suggests this type of thing bites back later on, so my inclination is to take the hit in the JDK and not return dups. The implementation might be able to cache information on the internal ReflectionData class. > > Please see John's response on this: > > http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2017-December/000465.html > Thanks for the context. You may be surprised to learn that i still don?t agree :-) so i will just state my objection more concisely and move on: I don?t think as specified the method serves developers very well for the common use-case of iterating over the nest mates and performing actions with side-effects. A developer will have to write defensive code or, more likely, write buggy code that will blow up in the rare case a classfile contain duplicates e.g. generated by something other than javac. That rare case may be hard to track down and expensive to fix. Paul. From karen.kinnear at oracle.com Tue Feb 13 23:26:31 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Tue, 13 Feb 2018 18:26:31 -0500 Subject: Valhalla EG minutes Jan 31, 3018 Message-ID: <6E30130E-A5D4-4BDF-8FB0-C4D412CD2F08@oracle.com> attendees: Tobi, Dan Smith, Dan Heidinga, Karen Kinnear AIs: Dan S: Condy SOE circularity issue: - sent http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2018-January/000526.html - additional feedback welcome David Holmes: JVMTI spec updates for nestmates: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2018-February/000541.html - feedback welcome All: please review LWorld Value Types and JVMS proposals note: updated versions from early reviews: http://cr.openjdk.java.net/~acorn/LWorldValueTypesFeb13.pdf http://cr.openjdk.java.net/~fparain/L-world/L-World-JVMS-4b.pdf I. Condy: JVMS in finali stages - any open issues? Dan H - not seeing any major issues Dan S - Alex Buckley is reviewing prior to incorporating into JVMS revising circularity detection - currently flexibility is a bit too nondeterministic clarifying when StackOverflowError happens - backedge of cycle, i.e. static arguments are resolved before SOE Nested BSM invocations on a single thread - the deepest one wins II. Nestmates JVMS review? Dan H: looks reasonable, including IAE change for invokeinterface non-public/non-private method Question about whether we have explicitly shared javac changes for inner/outer classes with Eclipse/IntelliJ - note: javac did not add nestmate changes for multiple top level classes in the same source - feel free to share, or to invite a contact to track the spec-observers, or to contact us directly III. Value Types - LWorld Karen walked through beginning of proposal: http://cr.openjdk.java.net/~acorn/LWorldValueTypesjan31.pdf we got as far as the Nullability and migration Open Design Issues. Summary: 1. java.lang.Object as root 2. value types have no identity, ever, there are no boxes and no implicit conversions ever 3. common bytecodes between object class (ID class?) and value class except for new vs. defaultvalue/withfield 3. migration ok object class -> value class, but not the reverse 4. to maximize ability to support existing code for a) methods and fields that refer to Objects or Object[] which now will also see value classes b) migration of value-based-classes to value types with e.g. caller/callee unchanged - to continue working we are proposing handling nullability by allowing value types to be nullable in general. The proposal is to have a field (or array) declaration declare if a value type is ACC_FLATTENABLE - i.e. if it is non-nullable, then it may be flattenable if the JVM chooses to. This proposal allows removing all Q-descriptors. Note that with a caller/callee/actual class separate compilation/migration challenge - only the actual class knows whether it is a value type or not. Open question for prototyping: can we get the JIT performance optimizations we need? Discussion: Dan S - how hard would it be to add Q?s to descriptors? Frederic - we have proposed a way to handle Q descriptors for fields and methods - we need to deal with signature mismatch between caller/callee (accessor/declarer) and the actual class that is loaded so internally the JVM would need to store both an all-L ?canonical? signature for handling signature searches as well as inheritance/overriding, as well as reflection, jvmti, etc. - and then would need to consult the Q-descriptor to see what was claimed - and then would have to consult the actual loaded class to see whether we actually have a Qtype - and if so far we only have null e.g. as a method argument, we may not yet have loaded the class, ? - note: there are NO conversions (implicit or explicit) - in an actual runtime, Foo is always actually either a value type or an object type - so without the Q descriptor - this is simpler - there is only one namespace Dan S: - what about using Q-signature to mean flattenable/non-nullability? Frederic: yes - if we had a fresh implementation that would work well, with the migration it does not note: no heisenboxes, no accidental identity method call: sig L: do not know L vs. Q until we load the type the intentional model here is that most bytecodes do not care - they work either way except for new and defaultvalue/withfield note: other bytecodes have different behaviors - e.g. putfield can not write to a value type immutable instance field Dan H: What about JIT performance? Frederic: for the hotspot JIT, we have loaded the signature types before we compile Dan H: what about needing different paths for L vs. Q? Frederic: also if you have an Object or Interface, the subtype may be a value class or object class - yes we do need to determine if we can get the JIT performance we need Dan S: does defaultvalue give a null for LType? defaultvalue only works on a value type, and gives a fully initialized instance note: anewarray/multianewarray - preload type of component Dan S: If you want non-nullable/flattenable today: if you have a value -> might flatten, populate with default values objectArray - populate with null references with this proposal: field: reference - initializes to null - which is a general default for a reference (which has no type) explicitly flattenable - pre-load class and initialize to default value for that type Corrections welcome, thanks, Karen From daniel.smith at oracle.com Tue Feb 13 23:55:19 2018 From: daniel.smith at oracle.com (Dan Smith) Date: Tue, 13 Feb 2018 16:55:19 -0700 Subject: Final CONSTANT_Dynamic spec In-Reply-To: References: <50F0A11C-476D-4C2B-8AE2-6BD2E27F891A@oracle.com> <80E6B9AF-AF53-416D-8710-5DCCED3A8EB5@oracle.com> Message-ID: <660ACF01-CBE8-452C-A695-9434F080EE7D@oracle.com> > On Feb 12, 2018, at 10:05 PM, Daniel Heidinga wrote: > > > That said, what side-effects are you concerned about? My claim is that re-resolution in this scenario would not, for example, trigger any class loading or bootstrap invocations. > > The simplest implementation strategy for a resolution that is guaranteed to throw SOF is to re-execute it each time the constant needs to be re-resolved and let the SOF reoccur naturally. This would result in new BSM invocations. Nope, it shouldn't. Example: Constant pool: ... #10 CONSTANT_Dynamic(BSM1, foo:I) #11 CONSTANT_Dynamic(BSM2, bar:D) #12 CONSTANT_Dynamic(BSM3, baz:J) BootstrapMethods: BSM1=invokestatic Bootstraps.m1(#1, #2, #11, #3) BSM2=invokestatic Bootstraps.m2(#4, #12, #5) BSM3=invokestatic Bootstraps.m3(#6, #10) The spec says that, at the point where resolution of #10 is required by the program (e.g., an ldc), the following are resolved in order: #1 #2 #4 #6 Then the recursion reaches #10, and a SOE occurs. Rather than throwing immediately, the implementation may choose to begin resolution of #10 again, which will trigger re-resolution of #1, #2, #4, and #6, but all of those have completed previously so they're just cache lookups. None of the bootstrap methods m1, m2, or m3 would be invoked before the SOE, because we're still working on building an argument list. If #1, #2, #4, or #6 have bootstrap methods of their own, those would be invoked the first time and never again. You might be thinking of a different case: a bootstrap method that triggers another call to itself within its own body. That's not addressed at all by this rule. The expected behavior falls out from the rules for bytecode evaluation: either there will be some control flow changes to break the cycle, or the stack will run out and you'll get a vanilla SOE. ?Dan From karen.kinnear at oracle.com Wed Feb 14 14:33:16 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Wed, 14 Feb 2018 09:33:16 -0500 Subject: Valhalla EG minutes Jan 31, 3018 In-Reply-To: <6E30130E-A5D4-4BDF-8FB0-C4D412CD2F08@oracle.com> References: <6E30130E-A5D4-4BDF-8FB0-C4D412CD2F08@oracle.com> Message-ID: <9FC85163-1660-4C40-8111-70EC1684B731@oracle.com> A clarification below: > On Feb 13, 2018, at 6:26 PM, Karen Kinnear wrote: > > attendees: Tobi, Dan Smith, Dan Heidinga, Karen Kinnear > > AIs: > Dan S: Condy SOE circularity issue: > - sent http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2018-January/000526.html > - additional feedback welcome > David Holmes: JVMTI spec updates for nestmates: > http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2018-February/000541.html > - feedback welcome > All: please review LWorld Value Types and JVMS proposals > note: updated versions from early reviews: http://cr.openjdk.java.net/~acorn/LWorldValueTypesFeb13.pdf > http://cr.openjdk.java.net/~fparain/L-world/L-World-JVMS-4b.pdf > > I. Condy: > JVMS in finali stages - any open issues? > Dan H - not seeing any major issues > Dan S - Alex Buckley is reviewing prior to incorporating into JVMS > revising circularity detection - currently flexibility is a bit too nondeterministic > clarifying when StackOverflowError happens - backedge of cycle, > i.e. static arguments are resolved before SOE > > Nested BSM invocations on a single thread - the deepest one wins > > II. Nestmates > JVMS review? > Dan H: looks reasonable, including IAE change for invokeinterface non-public/non-private method > Question about whether we have explicitly shared javac changes for inner/outer classes with Eclipse/IntelliJ > - note: javac did not add nestmate changes for multiple top level classes in the same source Q: Is there a nestmate relationship between top level classes that share the same source file? A: No. Being defined in the same source file is not sufficient to have a nestmate relationship. > - feel free to share, or to invite a contact to track the spec-observers, or to contact us directly > > > III. Value Types - LWorld > Karen walked through beginning of proposal: http://cr.openjdk.java.net/~acorn/LWorldValueTypesjan31.pdf > we got as far as the Nullability and migration Open Design Issues. > > Summary: > 1. java.lang.Object as root > 2. value types have no identity, ever, there are no boxes and no implicit conversions ever > 3. common bytecodes between object class (ID class?) and value class > except for new vs. defaultvalue/withfield > 3. migration ok object class -> value class, but not the reverse > 4. to maximize ability to support existing code for > a) methods and fields that refer to Objects or Object[] which now will also see value classes > b) migration of value-based-classes to value types with e.g. caller/callee unchanged - to continue working > > we are proposing handling nullability by allowing value types to be nullable in general. > The proposal is to have a field (or array) declaration declare if a value type is ACC_FLATTENABLE - i.e. > if it is non-nullable, then it may be flattenable if the JVM chooses to. > > This proposal allows removing all Q-descriptors. > > Note that with a caller/callee/actual class separate compilation/migration challenge - only the actual > class knows whether it is a value type or not. > > Open question for prototyping: can we get the JIT performance optimizations we need? > > Discussion: > Dan S - how hard would it be to add Q?s to descriptors? > Frederic - we have proposed a way to handle Q descriptors for fields and methods > - we need to deal with signature mismatch between > caller/callee (accessor/declarer) and the actual class that is loaded > so internally the JVM would need to store both an all-L ?canonical? signature for handling signature searches > as well as inheritance/overriding, as well as reflection, jvmti, etc. > - and then would need to consult the Q-descriptor to see what was claimed > - and then would have to consult the actual loaded class to see whether we actually have a Qtype > - and if so far we only have null e.g. as a method argument, we may not yet have loaded the class, ? > - note: there are NO conversions (implicit or explicit) - in an actual runtime, Foo is always actually > either a value type or an object type > - so without the Q descriptor - this is simpler - there is only one namespace > > Dan S: > - what about using Q-signature to mean flattenable/non-nullability? > Frederic: yes - if we had a fresh implementation that would work well, with the migration it does not > note: no heisenboxes, no accidental identity > method call: sig L: do not know L vs. Q until we load the type > > the intentional model here is that most bytecodes do not care - they work either way > except for new and defaultvalue/withfield > note: other bytecodes have different behaviors - e.g. putfield can not write to a value type immutable instance field > > Dan H: What about JIT performance? > Frederic: for the hotspot JIT, we have loaded the signature types before we compile > Dan H: what about needing different paths for L vs. Q? > Frederic: also if you have an Object or Interface, the subtype may be a value class or object class > - yes we do need to determine if we can get the JIT performance we need > > Dan S: > does defaultvalue give a null for LType? > defaultvalue only works on a value type, and gives a fully initialized instance > > note: anewarray/multianewarray - preload type of component > > Dan S: > If you want non-nullable/flattenable > today: if you have a value -> might flatten, populate with default values > objectArray - populate with null references > > with this proposal: > field: reference - initializes to null - which is a general default for a reference (which has no type) > explicitly flattenable - pre-load class and initialize to default value for that type > > Corrections welcome, > thanks, > Karen > > > > From brian.goetz at oracle.com Wed Feb 14 14:41:24 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 14 Feb 2018 06:41:24 -0800 Subject: Valhalla EG minutes Jan 31, 3018 In-Reply-To: <9FC85163-1660-4C40-8111-70EC1684B731@oracle.com> References: <6E30130E-A5D4-4BDF-8FB0-C4D412CD2F08@oracle.com> <9FC85163-1660-4C40-8111-70EC1684B731@oracle.com> Message-ID: <71F1A86F-2173-4699-9BB7-407F34C4F2A5@oracle.com> > Q: Is there a nestmate relationship between top level classes that share the same source file? > A: No. Being defined in the same source file is not sufficient to have a nestmate relationship. More precisely: that?s up to the language compiler. If the rules of your language say that these classes are nest mates, they are nest mates. By default, Java will likely not treat them as nest mates. From karen.kinnear at oracle.com Wed Feb 14 15:16:38 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Wed, 14 Feb 2018 10:16:38 -0500 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> Message-ID: <61716004-3F0B-4223-A4DC-7EDECF338D00@oracle.com> David, Re-reading these I had one suggestion: > - java/lang/invoke/MethodHandles.java > > *

> - * In some cases, access between nested classes is obtained by the Java compiler by creating > - * an wrapper method to access a private method of another class > - * in the same top-level declaration. > + * Since JDK 11 the relationship between nested types can be expressed directly through the > + * {@code NestHost} and {@code NestMembers} attributes. > + * (See the Java Virtual Machine Specification, sections 4.7.28 and 4.7.29.) > + * In that case, the lookup class has direct access to private members of all its nestmates, and > + * that is true of the associated {@code Lookup} object as well. > + * Otherwise, access between nested classes is obtained by the Java compiler creating > + * a wrapper method to access a private method of another class in the same nest. > * For example, a nested class {@code C.D} > Updated the nested classes description to cover legacy approach and new nestmate approach. > > - * {@code C.E} would be unable to those private members. > + * {@code C.E} would be unable to access those private members. > Fixed typo: "access" was missing. > > * Discussion of private access: > * We say that a lookup has private access > * if its {@linkplain #lookupModes lookup modes} > - * include the possibility of accessing {@code private} members. > + * include the possibility of accessing {@code private} members > + * (which includes the private members of nestmates). > * As documented in the relevant methods elsewhere, > * only lookups with private access possess the following capabilities: > *

    > - *
  • access private fields, methods, and constructors of the lookup class > + *
  • access private fields, methods, and constructors of the lookup class and its nestmates > Clarify that private access includes nestmate access. > > - * access all members of the caller's class, all public types in the caller's module, > + * access all members of the caller's class and nestmates, all public types in the caller's module, For the above, I would change this to * access all members of the caller?s class, all private members of nestmates, all types in the caller?s package, all public ? Specifically, we are extended the PRIVATE mode as above to include access to all private members of nestmates and this description is trying to summarize access when all possible bits are set. None of the settings give you access to default (package-private) members that a nestmate inherits from a different package - since the Lookup model is based on the JVMS/bytecode behavior. I added the types in the caller?s package since PACKAGE gives you that but it was missing from the existing list. thanks, Karen > > Thanks, > > David From john.r.rose at oracle.com Wed Feb 14 20:23:22 2018 From: john.r.rose at oracle.com (John Rose) Date: Wed, 14 Feb 2018 12:23:22 -0800 Subject: Final CONSTANT_Dynamic spec In-Reply-To: References: <660ACF01-CBE8-452C-A695-9434F080EE7D@oracle.com> <50F0A11C-476D-4C2B-8AE2-6BD2E27F891A@oracle.com> <80E6B9AF-AF53-416D-8710-5DCCED3A8EB5@oracle.com> Message-ID: On Feb 14, 2018, at 12:06 PM, Daniel Heidinga wrote: > > The stackoverflow in this example occurs in the static argument resolution phase, not the BSM invocation. > In one particular JVM implementation with which I am familiar, an OS stack overflow occurred in the static argument resolution phase of a test case like this, *but* it was not converted into a Java SOE; it turned into a hard crash. This was fixed, in that JVM implementation, by threading the static argument resolution logic through a Java call, which brought the appropriate overflow sensing and conversion logic. ? John From david.holmes at oracle.com Thu Feb 15 01:36:02 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 Feb 2018 11:36:02 +1000 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: <61716004-3F0B-4223-A4DC-7EDECF338D00@oracle.com> References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> <61716004-3F0B-4223-A4DC-7EDECF338D00@oracle.com> Message-ID: <84157714-9c88-9026-37d4-b33ae9a7161f@oracle.com> Hi Karen, Thanks for looking at this. On 15/02/2018 1:16 AM, Karen Kinnear wrote: > David, > > Re-reading these I had one suggestion: > >> - java/lang/invoke/MethodHandles.java >> *

    >> - * In some cases, access between nested classes is obtained by the Java compiler by creating >> - * an wrapper method to access a private method of another class >> - * in the same top-level declaration. >> + * Since JDK 11 the relationship between nested types can be expressed directly through the >> + * {@code NestHost} and {@code NestMembers} attributes. >> + * (See the Java Virtual Machine Specification, sections 4.7.28 and 4.7.29.) >> + * In that case, the lookup class has direct access to private members of all its nestmates, and >> + * that is true of the associated {@code Lookup} object as well. >> + * Otherwise, access between nested classes is obtained by the Java compiler creating >> + * a wrapper method to access a private method of another class in the same nest. >> * For example, a nested class {@code C.D} >> >> Updated the nested classes description to cover legacy approach and >> new nestmate approach. >> >> - * {@code C.E} would be unable to those private members. >> + * {@code C.E} would be unable to access those private members. >> >> Fixed typo: "access" was missing. >> >> * Discussion of private access: >> * We say that a lookup has private access >> * if its {@linkplain #lookupModes lookup modes} >> - * include the possibility of accessing {@code private} members. >> + * include the possibility of accessing {@code private} members >> + * (which includes the private members of nestmates). >> * As documented in the relevant methods elsewhere, >> * only lookups with private access possess the following capabilities: >> *

      >> - *
    • access private fields, methods, and constructors of the lookup class >> + *
    • access private fields, methods, and constructors of the lookup class and its nestmates >> >> Clarify that private access includes nestmate access. >> >> - * access all members of the caller's class, all public types in the caller's module, >> + * access all members of the caller's class and nestmates, all public types in the caller's module, > For the above, I would change this to > * access all members of the caller?s class, all private members of > nestmates, all types in the caller?s package, all public ? > > Specifically, we are extended the PRIVATE mode as above to include > access to all private members of nestmates > and this description is trying to summarize access when all possible > bits are set. > None of the settings give you access to default (package-private) > members that a nestmate inherits from a > different package - since the Lookup model is based on the JVMS/bytecode > behavior. True - you don't get access to a nestmates inherited protected members declared in a different package. I certainly didn't intend to somehow imply that. > I added the types in the caller?s package since PACKAGE gives you that > but it was missing from the existing list. Backing up ... in 8 this method doc simply said: "A freshly-created lookup object on the caller's class has all possible bits set, since the caller class can access all its own members." It doesn't try to say what a class can access, it just makes the obvious statement that it can access all its own members. If that was the current text I would not have needed to make any adjustment for nestmates. But for 9/10 it states: "A freshly-created lookup object on the caller's class has all possible bits set, except UNCONDITIONAL. The lookup can be used to access all members of the caller's class, all public types in the caller's module, and all public types in packages exported by other modules to the caller's module." This is quite a different formulation as it now tries to enumerate the set of accessible things - or at least gives that impression to me! But it is not complete as it doesn't mention non-public types in the current package, nor does it mention package-accessible members of types (whether public or not). With nestmates we have only expanded to include private member access, but the original text doesn't touch on nestmates directly at all. If we were to say only "all private members of nestmates" then we seem to suggest no access to the all the other members (public, directly declared protected, package). But if we say "nestmates" then that may imply more than intended as you point out. If we say nothing then we again imply by omission that there is no nestmate access. This seems to be a bit of an unwravelling thread started by the changes in 9. I would suggest we simply delete this sentence altogether: "The lookup can be used to access all members of the caller's class, all public types in the caller's module, and all public types in packages exported by other modules to the caller's module." as each of the modes, together with the "Access Checking" section of the class docs, define what is accessible under which mode. I don't think lookupModes() needs to try and restate that. What do you think? Thanks, David > thanks, > Karen >> >> >> Thanks, >> >> David >> > From karen.kinnear at oracle.com Thu Feb 15 22:01:52 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Thu, 15 Feb 2018 17:01:52 -0500 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: <84157714-9c88-9026-37d4-b33ae9a7161f@oracle.com> References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> <61716004-3F0B-4223-A4DC-7EDECF338D00@oracle.com> <84157714-9c88-9026-37d4-b33ae9a7161f@oracle.com> Message-ID: <7ABE4496-283C-4629-9A3B-1A4DAE960E07@oracle.com> David, I think that is a much better solution. Let the description of each Lookup mode be precise, and you have already updated PRIVATE mode to include nestmates. I brought this concern up in the EG meeting yesterday and wanted to clarify the difference between handling of inner/outer classes for backward compatibility and general nestmate handling. Assumptions: 1. MethodHandle/VarHandle behavior is modeled on bytecode behavior. 2. Nestmates have the added capability of access to private members of their nestmates. Period. 3. In future we expect to use nestmates for more than inner/outer classes. 4. Inner/outer classes will continue to have the InnerClasses attribute, and starting in JDK11, javac will also generate NestHost and NestMember attributes. 5. With Nestmates, javac will not generate the default (package) trampolines to allow inner/outer classes to access each other?s private members. Note that today this is only done for members that have compile time accesses. (6. With Nestmates, bridges for protected members will still be generated unchanged.) For nestmates in general, the modifications you have made below allow a nestmate to access private members of their nestmates to match the bytecode behavior. Prior to nestmates, there is a special workaround in MethodHandles.Lookup.in() to allow inner/outer classes to access any member of any class that shares its top level class to emulate the generated trampolines. https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/MethodHandles.Lookup.html#lookupModes-- In some cases, access between nested classes is obtained by the Java compiler by creating an wrapper method to access a private method of another class in the same top-level declaration. For example, a nested class C.D can access private members within other related classes such as C, C.D.E, or C.B, but the Java compiler may need to generate wrapper methods in those related classes. In such cases, a Lookup object on C.E would be unable to those private members. A workaround for this limitation is the Lookup.in method, which can transform a lookup on C.E into one on any of those other classes, without special elevation of privilege. This workaround will continue to be supported going forward explicitly for inner/outer classes. A side-effect of this workaround is the ability of the returned Lookup to access not only private methods in the ?related? class, but also protected and inherited members of that class which are defined in other packages. So this workaround will continue to work for inner/outer classes that are also nestmates for backward compatibility. Going forward, for nestmates in general, the goal is to provide access to private members, which can be done via the access check to match bytecode behavior, and does not require special Lookup.in() workarounds. If at some point in the future we decide we want increased access for nestmates, we can widen this. Let?s just say that the complexity there is challenging and that it is better to err on the side of starting out more restrictive. Summary - I agree with David. We can leave the documentation as is, with the explicit changes to access checking modified below for private members accessible to nestmates. thanks David! Karen > On Feb 14, 2018, at 8:36 PM, David Holmes wrote: > > Hi Karen, > > Thanks for looking at this. > > On 15/02/2018 1:16 AM, Karen Kinnear wrote: >> David, >> Re-reading these I had one suggestion: >>> - java/lang/invoke/MethodHandles.java >>> *

      >>> - * In some cases, access between nested classes is obtained by the Java compiler by creating >>> - * an wrapper method to access a private method of another class >>> - * in the same top-level declaration. >>> + * Since JDK 11 the relationship between nested types can be expressed directly through the >>> + * {@code NestHost} and {@code NestMembers} attributes. >>> + * (See the Java Virtual Machine Specification, sections 4.7.28 and 4.7.29.) >>> + * In that case, the lookup class has direct access to private members of all its nestmates, and >>> + * that is true of the associated {@code Lookup} object as well. >>> + * Otherwise, access between nested classes is obtained by the Java compiler creating >>> + * a wrapper method to access a private method of another class in the same nest. >>> * For example, a nested class {@code C.D} >>> >>> Updated the nested classes description to cover legacy approach and new nestmate approach. >>> >>> - * {@code C.E} would be unable to those private members. >>> + * {@code C.E} would be unable to access those private members. >>> >>> Fixed typo: "access" was missing. >>> >>> * Discussion of private access: >>> * We say that a lookup has private access >>> * if its {@linkplain #lookupModes lookup modes} >>> - * include the possibility of accessing {@code private} members. >>> + * include the possibility of accessing {@code private} members >>> + * (which includes the private members of nestmates). >>> * As documented in the relevant methods elsewhere, >>> * only lookups with private access possess the following capabilities: >>> *

        >>> - *
      • access private fields, methods, and constructors of the lookup class >>> + *
      • access private fields, methods, and constructors of the lookup class and its nestmates >>> >>> Clarify that private access includes nestmate access. >>> >>> - * access all members of the caller's class, all public types in the caller's module, >>> + * access all members of the caller's class and nestmates, all public types in the caller's module, >> For the above, I would change this to >> * access all members of the caller?s class, all private members of nestmates, all types in the caller?s package, all public ? >> Specifically, we are extended the PRIVATE mode as above to include access to all private members of nestmates >> and this description is trying to summarize access when all possible bits are set. >> None of the settings give you access to default (package-private) members that a nestmate inherits from a >> different package - since the Lookup model is based on the JVMS/bytecode behavior. > > True - you don't get access to a nestmates inherited protected members declared in a different package. I certainly didn't intend to somehow imply that. > >> I added the types in the caller?s package since PACKAGE gives you that but it was missing from the existing list. > > Backing up ... in 8 this method doc simply said: > > "A freshly-created lookup object on the caller's class has all possible bits set, since the caller class can access all its own members." > > It doesn't try to say what a class can access, it just makes the obvious statement that it can access all its own members. If that was the current text I would not have needed to make any adjustment for nestmates. > > But for 9/10 it states: > > "A freshly-created lookup object on the caller's class has all possible bits set, except UNCONDITIONAL. The lookup can be used to access all members of the caller's class, all public types in the caller's module, and all public types in packages exported by other modules to the caller's module." > > This is quite a different formulation as it now tries to enumerate the set of accessible things - or at least gives that impression to me! But it is not complete as it doesn't mention non-public types in the current package, nor does it mention package-accessible members of types (whether public or not). > > With nestmates we have only expanded to include private member access, but the original text doesn't touch on nestmates directly at all. If we were to say only "all private members of nestmates" then we seem to suggest no access to the all the other members (public, directly declared protected, package). But if we say "nestmates" then that may imply more than intended as you point out. If we say nothing then we again imply by omission that there is no nestmate access. > > This seems to be a bit of an unwravelling thread started by the changes in 9. I would suggest we simply delete this sentence altogether: > > "The lookup can be used to access all members of the caller's class, all public types in the caller's module, and all public types in packages exported by other modules to the caller's module." > > as each of the modes, together with the "Access Checking" section of the class docs, define what is accessible under which mode. I don't think lookupModes() needs to try and restate that. > > What do you think? > > Thanks, > David > >> thanks, >> Karen >>> >>> >>> Thanks, >>> >>> David From david.holmes at oracle.com Fri Feb 16 09:20:18 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 16 Feb 2018 19:20:18 +1000 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: <7ABE4496-283C-4629-9A3B-1A4DAE960E07@oracle.com> References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> <61716004-3F0B-4223-A4DC-7EDECF338D00@oracle.com> <84157714-9c88-9026-37d4-b33ae9a7161f@oracle.com> <7ABE4496-283C-4629-9A3B-1A4DAE960E07@oracle.com> Message-ID: <8dde407e-4d5d-24b0-b06e-cd285a3c4b7f@oracle.com> Hi Karen, On 16/02/2018 8:01 AM, Karen Kinnear wrote: > David, > > I think that is a much better solution. Let the description of each > Lookup mode be precise, and you have already updated PRIVATE mode to > include nestmates. Okay I've deleted that sentence. Unfortunately something has broken specdiff so I can't regenerate the docs at the moment - and I've lost the ones I had generated. > > I brought this concern up in the EG meeting ?yesterday and wanted to > clarify the difference > between handling of inner/outer classes for backward compatibility and > general nestmate handling. Many thanks for that detailed walk through. David ----- > Assumptions: > 1. MethodHandle/VarHandle behavior is modeled on bytecode behavior. > 2. Nestmates have the added capability of access to private members of > their nestmates. Period. > 3. In future we expect to use nestmates for more than inner/outer classes. > 4. Inner/outer classes will continue to have the InnerClasses attribute, > and starting in JDK11, javac > will also generate NestHost and NestMember attributes. > 5. With Nestmates, javac will not generate the default (package) > trampolines to allow inner/outer > classes to access each other?s private members. Note that today this is > only done for members that > have compile time accesses. > (6. With Nestmates, bridges for protected members will still be > generated unchanged.) > > For nestmates in general, the modifications you have made below allow a > nestmate to access private > members of their nestmates to match the bytecode behavior. > > Prior to nestmates, there is a special workaround in > MethodHandles.Lookup.in() to allow inner/outer > classes to access any member of any class that shares its top level > class to emulate the generated trampolines. > > https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/MethodHandles.Lookup.html#lookupModes-- > > In some cases, access between nested classes is obtained by the Java > compiler by creating an wrapper method to access a private method of > another class in the same top-level declaration. For example, a nested > class |C.D|?can access private members within other related classes such > as |C|, |C.D.E|, or |C.B|, but the Java compiler may need to generate > wrapper methods in those related classes. In such cases, a > |Lookup|?object on |C.E|?would be unable to those private members. A > workaround for this limitation is the |Lookup.in| > ?method, > which can transform a lookup on |C.E|?into one on any of those other > classes, without special elevation of privilege. > > This workaround will continue to be supported going forward explicitly > for inner/outer classes. > A side-effect of this workaround is the ability of the returned Lookup > to access not only private methods in > the??related? class, but also protected and inherited members of that > class which are defined in other packages. > So this workaround will continue to work for inner/outer classes that > are also nestmates for backward > compatibility. > > Going?forward, for nestmates in general, the goal is to provide access > to private members, which > can be done via the access check to match bytecode behavior, and does > not require special Lookup.in() workarounds. > > If at some point in the future we decide we want increased access for > nestmates, we can widen > this. Let?s just say that the complexity there is challenging and that > it is better to err on the side of > starting out more restrictive. > > Summary - I agree with David. We can leave the documentation as is, with > the explicit changes to > access checking modified below for private members accessible to nestmates. > > thanks David! > Karen > >> On Feb 14, 2018, at 8:36 PM, David Holmes > > wrote: >> >> Hi Karen, >> >> Thanks for looking at this. >> >> On 15/02/2018 1:16 AM, Karen Kinnear wrote: >>> David, >>> Re-reading these I had one suggestion: >>>> - java/lang/invoke/MethodHandles.java >>>> ??????*

        >>>> - ????* In some cases, access between nested classes is obtained by >>>> the Java compiler by creating >>>> - ????* an wrapper method to access a private method of another class >>>> - ????* in the same top-level declaration. >>>> + ????* Since JDK 11 the relationship between nested types can be >>>> expressed directly through the >>>> + ????* {@code NestHost} and {@code NestMembers} attributes. >>>> + ????* (See the Java Virtual Machine Specification, sections 4.7.28 >>>> and 4.7.29.) >>>> + ????* In that case, the lookup class has direct access to private >>>> members of all its nestmates, and >>>> + ????* that is true of the associated {@code Lookup} object as well. >>>> + ????* Otherwise, access between nested classes is obtained by the >>>> Java compiler creating >>>> + ????* a wrapper method to access a private method of another class >>>> in the same nest. >>>> ??????* For example, a nested class {@code C.D} >>>> >>>> Updated the nested classes description to cover legacy approach and >>>> new nestmate approach. >>>> >>>> - ????* {@code C.E} would be unable to those private members. >>>> + ????* {@code C.E} would be unable to access those private members. >>>> >>>> Fixed typo: "access" was missing. >>>> >>>> ??????* Discussion of private access: >>>> ??????* We say that a lookup has private access >>>> ??????* if its {@linkplain #lookupModes lookup modes} >>>> - ????* include the possibility of accessing {@code private} members. >>>> + ????* include the possibility of accessing {@code private} members >>>> + ????* (which includes the private members of nestmates). >>>> ??????* As documented in the relevant methods elsewhere, >>>> ??????* only lookups with private access possess the following >>>> capabilities: >>>> ??????*

          >>>> - ????*
        • access private fields, methods, and constructors of the >>>> lookup class >>>> + ????*
        • access private fields, methods, and constructors of the >>>> lookup class and its nestmates >>>> >>>> Clarify that private access includes nestmate access. >>>> >>>> - ????* ?access all members of the caller's class, all public types >>>> in the caller's module, >>>> + ????* ?access all members of the caller's class and nestmates, all >>>> public types in the caller's module, >>> For the above, I would change this to >>> * access all members of the caller?s class, all private members of >>> nestmates, all types in the caller?s package, all public ? >>> Specifically, we are extended the PRIVATE mode as above to include >>> access to all private members of nestmates >>> and this description is trying to summarize access when all possible >>> bits are set. >>> None of the settings give you access to default (package-private) >>> members that a nestmate inherits from a >>> different package - since the Lookup model is based on the >>> JVMS/bytecode behavior. >> >> True - you don't get access to a nestmates inherited protected members >> declared in a different package. I certainly didn't intend to somehow >> imply that. >> >>> I added the types in the caller?s package since PACKAGE gives you >>> that but it was missing from the existing list. >> >> Backing up ... in 8 this method doc simply said: >> >> "A freshly-created lookup object on the caller's class has all >> possible bits set, since the caller class can access all its own members." >> >> It doesn't try to say what a class can access, it just makes the >> obvious statement that it can access all its own members. If that was >> the current text I would not have needed to make any adjustment for >> nestmates. >> >> But for 9/10 it states: >> >> "A freshly-created lookup object on the caller's class has all >> possible bits set, except UNCONDITIONAL. The lookup can be used to >> access all members of the caller's class, all public types in the >> caller's module, and all public types in packages exported by other >> modules to the caller's module." >> >> This is quite a different formulation as it now tries to enumerate the >> set of accessible things - or at least gives that impression to me! >> But it is not complete as it doesn't mention non-public types in the >> current package, nor does it mention package-accessible members of >> types (whether public or not). >> >> With nestmates we have only expanded to include private member access, >> but the original text doesn't touch on nestmates directly at all. If >> we were to say only "all private members of nestmates" then we seem to >> suggest no access to the all the other members (public, directly >> declared protected, package). But if we say "nestmates" then that may >> imply more than intended as you point out. If we say nothing then we >> again imply by omission that there is no nestmate access. >> >> This seems to be a bit of an unwravelling thread started by the >> changes in 9. I would suggest we simply delete this sentence altogether: >> >> "The lookup can be used to access all members of the caller's class, >> all public types in the caller's module, and all public types in >> packages exported by other modules to the caller's module." >> >> as each of the modes, together with the "Access Checking" section of >> the class docs, define what is accessible under which mode. I don't >> think lookupModes() needs to try and restate that. >> >> What do you think? >> >> Thanks, >> David >> >>> thanks, >>> Karen >>>> >>>> >>>> Thanks, >>>> >>>> David > From david.holmes at oracle.com Fri Feb 16 23:31:21 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 17 Feb 2018 09:31:21 +1000 Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access In-Reply-To: <8dde407e-4d5d-24b0-b06e-cd285a3c4b7f@oracle.com> References: <9613ad0d-4ac3-bc3f-cefb-15bfd9fae3d1@oracle.com> <61716004-3F0B-4223-A4DC-7EDECF338D00@oracle.com> <84157714-9c88-9026-37d4-b33ae9a7161f@oracle.com> <7ABE4496-283C-4629-9A3B-1A4DAE960E07@oracle.com> <8dde407e-4d5d-24b0-b06e-cd285a3c4b7f@oracle.com> Message-ID: Specdiffs now available again: http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang/java/lang/Class.html http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.invoke/java/lang/invoke/MethodHandle.html http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.invoke/java/lang/invoke/MethodHandles.Lookup.html http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.reflect/java/lang/reflect/AccessibleObject.html http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.reflect/java/lang/reflect/Method.html David On 16/02/2018 7:20 PM, David Holmes wrote: > Hi Karen, > > On 16/02/2018 8:01 AM, Karen Kinnear wrote: >> David, >> >> I think that is a much better solution. Let the description of each >> Lookup mode be precise, and you have already updated PRIVATE mode to >> include nestmates. > > Okay I've deleted that sentence. > > Unfortunately something has broken specdiff so I can't regenerate the > docs at the moment - and I've lost the ones I had generated. > >> >> I brought this concern up in the EG meeting ?yesterday and wanted to >> clarify the difference >> between handling of inner/outer classes for backward compatibility and >> general nestmate handling. > > Many thanks for that detailed walk through. > > David > ----- > > >> Assumptions: >> 1. MethodHandle/VarHandle behavior is modeled on bytecode behavior. >> 2. Nestmates have the added capability of access to private members of >> their nestmates. Period. >> 3. In future we expect to use nestmates for more than inner/outer >> classes. >> 4. Inner/outer classes will continue to have the InnerClasses >> attribute, and starting in JDK11, javac >> will also generate NestHost and NestMember attributes. >> 5. With Nestmates, javac will not generate the default (package) >> trampolines to allow inner/outer >> classes to access each other?s private members. Note that today this >> is only done for members that >> have compile time accesses. >> (6. With Nestmates, bridges for protected members will still be >> generated unchanged.) >> >> For nestmates in general, the modifications you have made below allow >> a nestmate to access private >> members of their nestmates to match the bytecode behavior. >> >> Prior to nestmates, there is a special workaround in >> MethodHandles.Lookup.in() to allow inner/outer >> classes to access any member of any class that shares its top level >> class to emulate the generated trampolines. >> >> https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/MethodHandles.Lookup.html#lookupModes-- >> >> >> In some cases, access between nested classes is obtained by the Java >> compiler by creating an wrapper method to access a private method of >> another class in the same top-level declaration. For example, a nested >> class |C.D|?can access private members within other related classes >> such as |C|, |C.D.E|, or |C.B|, but the Java compiler may need to >> generate wrapper methods in those related classes. In such cases, a >> |Lookup|?object on |C.E|?would be unable to those private members. A >> workaround for this limitation is the |Lookup.in| >> ?method, >> which can transform a lookup on |C.E|?into one on any of those other >> classes, without special elevation of privilege. >> >> This workaround will continue to be supported going forward explicitly >> for inner/outer classes. >> A side-effect of this workaround is the ability of the returned Lookup >> to access not only private methods in >> the??related? class, but also protected and inherited members of that >> class which are defined in other packages. >> So this workaround will continue to work for inner/outer classes that >> are also nestmates for backward >> compatibility. >> >> Going?forward, for nestmates in general, the goal is to provide access >> to private members, which >> can be done via the access check to match bytecode behavior, and does >> not require special Lookup.in() workarounds. >> >> If at some point in the future we decide we want increased access for >> nestmates, we can widen >> this. Let?s just say that the complexity there is challenging and that >> it is better to err on the side of >> starting out more restrictive. >> >> Summary - I agree with David. We can leave the documentation as is, >> with the explicit changes to >> access checking modified below for private members accessible to >> nestmates. >> >> thanks David! >> Karen >> >>> On Feb 14, 2018, at 8:36 PM, David Holmes >> > wrote: >>> >>> Hi Karen, >>> >>> Thanks for looking at this. >>> >>> On 15/02/2018 1:16 AM, Karen Kinnear wrote: >>>> David, >>>> Re-reading these I had one suggestion: >>>>> - java/lang/invoke/MethodHandles.java >>>>> ??????*

          >>>>> - ????* In some cases, access between nested classes is obtained by >>>>> the Java compiler by creating >>>>> - ????* an wrapper method to access a private method of another class >>>>> - ????* in the same top-level declaration. >>>>> + ????* Since JDK 11 the relationship between nested types can be >>>>> expressed directly through the >>>>> + ????* {@code NestHost} and {@code NestMembers} attributes. >>>>> + ????* (See the Java Virtual Machine Specification, sections >>>>> 4.7.28 and 4.7.29.) >>>>> + ????* In that case, the lookup class has direct access to private >>>>> members of all its nestmates, and >>>>> + ????* that is true of the associated {@code Lookup} object as well. >>>>> + ????* Otherwise, access between nested classes is obtained by the >>>>> Java compiler creating >>>>> + ????* a wrapper method to access a private method of another >>>>> class in the same nest. >>>>> ??????* For example, a nested class {@code C.D} >>>>> >>>>> Updated the nested classes description to cover legacy approach and >>>>> new nestmate approach. >>>>> >>>>> - ????* {@code C.E} would be unable to those private members. >>>>> + ????* {@code C.E} would be unable to access those private members. >>>>> >>>>> Fixed typo: "access" was missing. >>>>> >>>>> ??????* Discussion of private access: >>>>> ??????* We say that a lookup has private access >>>>> ??????* if its {@linkplain #lookupModes lookup modes} >>>>> - ????* include the possibility of accessing {@code private} members. >>>>> + ????* include the possibility of accessing {@code private} members >>>>> + ????* (which includes the private members of nestmates). >>>>> ??????* As documented in the relevant methods elsewhere, >>>>> ??????* only lookups with private access possess the following >>>>> capabilities: >>>>> ??????*

            >>>>> - ????*
          • access private fields, methods, and constructors of the >>>>> lookup class >>>>> + ????*
          • access private fields, methods, and constructors of the >>>>> lookup class and its nestmates >>>>> >>>>> Clarify that private access includes nestmate access. >>>>> >>>>> - ????* ?access all members of the caller's class, all public types >>>>> in the caller's module, >>>>> + ????* ?access all members of the caller's class and nestmates, >>>>> all public types in the caller's module, >>>> For the above, I would change this to >>>> * access all members of the caller?s class, all private members of >>>> nestmates, all types in the caller?s package, all public ? >>>> Specifically, we are extended the PRIVATE mode as above to include >>>> access to all private members of nestmates >>>> and this description is trying to summarize access when all possible >>>> bits are set. >>>> None of the settings give you access to default (package-private) >>>> members that a nestmate inherits from a >>>> different package - since the Lookup model is based on the >>>> JVMS/bytecode behavior. >>> >>> True - you don't get access to a nestmates inherited protected >>> members declared in a different package. I certainly didn't intend to >>> somehow imply that. >>> >>>> I added the types in the caller?s package since PACKAGE gives you >>>> that but it was missing from the existing list. >>> >>> Backing up ... in 8 this method doc simply said: >>> >>> "A freshly-created lookup object on the caller's class has all >>> possible bits set, since the caller class can access all its own >>> members." >>> >>> It doesn't try to say what a class can access, it just makes the >>> obvious statement that it can access all its own members. If that was >>> the current text I would not have needed to make any adjustment for >>> nestmates. >>> >>> But for 9/10 it states: >>> >>> "A freshly-created lookup object on the caller's class has all >>> possible bits set, except UNCONDITIONAL. The lookup can be used to >>> access all members of the caller's class, all public types in the >>> caller's module, and all public types in packages exported by other >>> modules to the caller's module." >>> >>> This is quite a different formulation as it now tries to enumerate >>> the set of accessible things - or at least gives that impression to >>> me! But it is not complete as it doesn't mention non-public types in >>> the current package, nor does it mention package-accessible members >>> of types (whether public or not). >>> >>> With nestmates we have only expanded to include private member >>> access, but the original text doesn't touch on nestmates directly at >>> all. If we were to say only "all private members of nestmates" then >>> we seem to suggest no access to the all the other members (public, >>> directly declared protected, package). But if we say "nestmates" then >>> that may imply more than intended as you point out. If we say nothing >>> then we again imply by omission that there is no nestmate access. >>> >>> This seems to be a bit of an unwravelling thread started by the >>> changes in 9. I would suggest we simply delete this sentence altogether: >>> >>> "The lookup can be used to access all members of the caller's class, >>> all public types in the caller's module, and all public types in >>> packages exported by other modules to the caller's module." >>> >>> as each of the modes, together with the "Access Checking" section of >>> the class docs, define what is accessible under which mode. I don't >>> think lookupModes() needs to try and restate that. >>> >>> What do you think? >>> >>> Thanks, >>> David >>> >>>> thanks, >>>> Karen >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> David >> From daniel.smith at oracle.com Fri Feb 16 23:32:17 2018 From: daniel.smith at oracle.com (Dan Smith) Date: Fri, 16 Feb 2018 16:32:17 -0700 Subject: A "RecursiveConstants" attribute In-Reply-To: References: <131B1F8D-F49C-47F8-BE7C-A660CBE1FCB5@oracle.com> <2CEA5EE3-6F51-4806-9CA5-2BAFCF0620E8@oracle.com> <95f19422-86a4-115b-eae7-d274ab57e072@oracle.com> <60BB9395-3B8B-4901-A8AD-99B3C209A005@oracle.com> Message-ID: <29CA13B1-7A8B-446D-8235-98CBE978AB97@oracle.com> > On Jan 31, 2018, at 2:50 PM, Brian Goetz wrote: > > OK, so the goal is clearly to *prohibit* cyclic constants by enforcing a depth constraint on condy-in-condy nesting. I like this goal much better, and I see how the solution addresses it. > > Still, I have to wonder whether its worth the complexity of capturing it explicitly with a new (stackmap-like) attribute, which must be specified, and gives us more chances for the classfile to be out of sync with itself, for the goal of preventing something that can't usefully happen anyway -- as we discussed previously, trying to actually resolve a cyclic constant will likely result in SOE, so such classfiles are pretty much useless and therefore not likely to spread very far. This is a reasonable critique. Another critique is that the cost of tracking down and parsing an extra attribute will undo a lot of the performance benefit of having it, perhaps even causing a net slowdown. As we've discussed the idea a little more, my inclination (raised in the meeting this week) is to give up on trying to add structural restrictions on constant pools designed to facilitate cycle detection. It seems like we can, in fact, detect cycles without extra guidance, and with minimal overhead in most cases. Some details below. Big picture, the most attractive path forward seems to be: - In 11, a CONSTANT_Dynamic with a cycle is an "error", reported via SOE at resolution time - When we introduce lazy condy resolution, we move the error detection to class load time, with a check that we're comfortable isn't disrupting startup time - As other potentially-cyclical constants are introduced, we use the same load time checks - We never require extra information from the compiler, or reject classes that are hard to prove cycle-free One nice thing about that last point is that we don't have to worry about incompatible support for "hard" classes in different releases. I'd love to hear about some experiments with this. We're talking about performance impact, and all discussion to this point has been hypothetical. Would be good to get some concrete feedback. ?Dan ----- Details: We started with a rule for format checking (JVMS 4.8) that says it's illegal for a CONSTANT_Dynamic (or, in the future, other potentially-cyclical constants) to refer, via static arguments, to itself?directly or indirectly. But we quickly decided this sounded expensive, and we didn't want a heavy validation step to mess with startup time. Hence, we explored various ways to prohibit otherwise-valid classes that would be hard to prove cycle-free. Something like the RecursiveConstants attributes seems to be the best option in that space. However, scrutinizing the original assumption, it seems reasonable that we can optimize the check so that it's extremely lightweight when references are "backward" (to previous entries in the constant pool), and only performs more expensive analysis when references are "forward". Most classes should naturally use backward references exclusively, since entries tend to be generated bottom-up. (In theory. If there are counter examples produced by any mainstream bytecode generators, that would be good to know.) So a dumb (worst case O(n^2)) but fast (very low constant cost) check could look something like: void checkCP(int i) { switch (tag(i)) { case CONSTANT_Dynamic: ... for (int j : condyStaticArgs(i)) { if (j >= i && tag(j) == CONSTANT_Dynamic) { assertUnreachable(i, j); // can't reach i from j } } ... ... } } It's already necessary to check that constant pool references are well-typed, so everything up to 'assertUnreachable' here is practically free*. The 'assertUnreachable' call might be somewhat expensive, but it will rarely be called in practice (and even then, we don't expect many structures to be particularly deep). (*In the particular case of CONSTANT_Dynamic, there's an indirection through BootstrapMethods. If a class heavily shares BootstrapMethods entries, then there's a new cost here, iterating over the same static arguments multiple times. Some optimization could be done to speed that up. For example, if there are no forward references on the first pass, there will be no forward references on later passes, when i is a larger number.) A good worst-case test would be a constant pool without cycles, built entirely out of forward references, consisting of lots of deep CONSTANT_Dynamic trees. Needless to say, we don't expect to encounter classes like this in the wild. But, if we do, the big benefit here is that, while they'll be a little slower to load, they'll still work. From forax at univ-mlv.fr Sat Feb 17 11:00:54 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 17 Feb 2018 12:00:54 +0100 (CET) Subject: A "RecursiveConstants" attribute In-Reply-To: <29CA13B1-7A8B-446D-8235-98CBE978AB97@oracle.com> References: <131B1F8D-F49C-47F8-BE7C-A660CBE1FCB5@oracle.com> <2CEA5EE3-6F51-4806-9CA5-2BAFCF0620E8@oracle.com> <95f19422-86a4-115b-eae7-d274ab57e072@oracle.com> <60BB9395-3B8B-4901-A8AD-99B3C209A005@oracle.com> <29CA13B1-7A8B-446D-8235-98CBE978AB97@oracle.com> Message-ID: <521665787.2472518.1518865254851.JavaMail.zimbra@u-pem.fr> I like this approach, this is fine for me. R?mi ----- Mail original ----- > De: "daniel smith" > ?: "valhalla-spec-experts" > Envoy?: Samedi 17 F?vrier 2018 00:32:17 > Objet: Re: A "RecursiveConstants" attribute >> On Jan 31, 2018, at 2:50 PM, Brian Goetz wrote: >> >> OK, so the goal is clearly to *prohibit* cyclic constants by enforcing a depth >> constraint on condy-in-condy nesting. I like this goal much better, and I see >> how the solution addresses it. >> >> Still, I have to wonder whether its worth the complexity of capturing it >> explicitly with a new (stackmap-like) attribute, which must be specified, and >> gives us more chances for the classfile to be out of sync with itself, for the >> goal of preventing something that can't usefully happen anyway -- as we >> discussed previously, trying to actually resolve a cyclic constant will likely >> result in SOE, so such classfiles are pretty much useless and therefore not >> likely to spread very far. > > This is a reasonable critique. Another critique is that the cost of tracking > down and parsing an extra attribute will undo a lot of the performance benefit > of having it, perhaps even causing a net slowdown. > > As we've discussed the idea a little more, my inclination (raised in the meeting > this week) is to give up on trying to add structural restrictions on constant > pools designed to facilitate cycle detection. It seems like we can, in fact, > detect cycles without extra guidance, and with minimal overhead in most cases. > Some details below. > > Big picture, the most attractive path forward seems to be: > - In 11, a CONSTANT_Dynamic with a cycle is an "error", reported via SOE at > resolution time > - When we introduce lazy condy resolution, we move the error detection to class > load time, with a check that we're comfortable isn't disrupting startup time > - As other potentially-cyclical constants are introduced, we use the same load > time checks > - We never require extra information from the compiler, or reject classes that > are hard to prove cycle-free > > One nice thing about that last point is that we don't have to worry about > incompatible support for "hard" classes in different releases. > > I'd love to hear about some experiments with this. We're talking about > performance impact, and all discussion to this point has been hypothetical. > Would be good to get some concrete feedback. > > ?Dan > > ----- > > Details: > > We started with a rule for format checking (JVMS 4.8) that says it's illegal for > a CONSTANT_Dynamic (or, in the future, other potentially-cyclical constants) to > refer, via static arguments, to itself?directly or indirectly. But we quickly > decided this sounded expensive, and we didn't want a heavy validation step to > mess with startup time. > > Hence, we explored various ways to prohibit otherwise-valid classes that would > be hard to prove cycle-free. Something like the RecursiveConstants attributes > seems to be the best option in that space. > > However, scrutinizing the original assumption, it seems reasonable that we can > optimize the check so that it's extremely lightweight when references are > "backward" (to previous entries in the constant pool), and only performs more > expensive analysis when references are "forward". Most classes should naturally > use backward references exclusively, since entries tend to be generated > bottom-up. (In theory. If there are counter examples produced by any mainstream > bytecode generators, that would be good to know.) > > So a dumb (worst case O(n^2)) but fast (very low constant cost) check could look > something like: > > void checkCP(int i) { > switch (tag(i)) { > case CONSTANT_Dynamic: > ... > for (int j : condyStaticArgs(i)) { > if (j >= i && tag(j) == CONSTANT_Dynamic) { > assertUnreachable(i, j); // can't reach i from j > } > } > ... > ... > } > } > > It's already necessary to check that constant pool references are well-typed, so > everything up to 'assertUnreachable' here is practically free*. The > 'assertUnreachable' call might be somewhat expensive, but it will rarely be > called in practice (and even then, we don't expect many structures to be > particularly deep). > > (*In the particular case of CONSTANT_Dynamic, there's an indirection through > BootstrapMethods. If a class heavily shares BootstrapMethods entries, then > there's a new cost here, iterating over the same static arguments multiple > times. Some optimization could be done to speed that up. For example, if there > are no forward references on the first pass, there will be no forward > references on later passes, when i is a larger number.) > > A good worst-case test would be a constant pool without cycles, built entirely > out of forward references, consisting of lots of deep CONSTANT_Dynamic trees. > Needless to say, we don't expect to encounter classes like this in the wild. > But, if we do, the big benefit here is that, while they'll be a little slower > to load, they'll still work. From forax at univ-mlv.fr Sat Feb 17 11:01:43 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 17 Feb 2018 12:01:43 +0100 (CET) Subject: A "RecursiveConstants" attribute In-Reply-To: <29CA13B1-7A8B-446D-8235-98CBE978AB97@oracle.com> References: <131B1F8D-F49C-47F8-BE7C-A660CBE1FCB5@oracle.com> <2CEA5EE3-6F51-4806-9CA5-2BAFCF0620E8@oracle.com> <95f19422-86a4-115b-eae7-d274ab57e072@oracle.com> <60BB9395-3B8B-4901-A8AD-99B3C209A005@oracle.com> <29CA13B1-7A8B-446D-8235-98CBE978AB97@oracle.com> Message-ID: <75563174.2472564.1518865303213.JavaMail.zimbra@u-pem.fr> > Most classes should naturally use backward references exclusively, since entries tend to be generated bottom-up. > (In theory. If there are counter examples produced by any mainstream bytecode generators, that would be good to know.) Currently, javac and ecj tends to generate their constants in the "wrong order". ecj: #15 = Methodref #16.#18 // java/lang/Class.desiredAssertionStatus:()Z #16 = Class #17 // java/lang/Class #17 = Utf8 java/lang/Class #18 = NameAndType #19:#20 // desiredAssertionStatus:()Z #19 = Utf8 desiredAssertionStatus #20 = Utf8 ()Z javac: ... #20 = Methodref #65.#66 // java/lang/Class.desiredAssertionStatus:()Z ... #65 = Class #75 // java/lang/Class #66 = NameAndType #76:#77 // desiredAssertionStatus:()Z ... #75 = Utf8 java/lang/Class #76 = Utf8 desiredAssertionStatus #77 = Utf8 ()Z It means nothing now because we only cares about constants that can introduce cycles so CONSTANT_Dynamic, it may be an issue in the future if by example a NameAndType can reference a constant with cycle, the code of javac and ecj will have to be changed. R?mi ----- Mail original ----- > De: "daniel smith" > ?: "valhalla-spec-experts" > Envoy?: Samedi 17 F?vrier 2018 00:32:17 > Objet: Re: A "RecursiveConstants" attribute >> On Jan 31, 2018, at 2:50 PM, Brian Goetz wrote: >> >> OK, so the goal is clearly to *prohibit* cyclic constants by enforcing a depth >> constraint on condy-in-condy nesting. I like this goal much better, and I see >> how the solution addresses it. >> >> Still, I have to wonder whether its worth the complexity of capturing it >> explicitly with a new (stackmap-like) attribute, which must be specified, and >> gives us more chances for the classfile to be out of sync with itself, for the >> goal of preventing something that can't usefully happen anyway -- as we >> discussed previously, trying to actually resolve a cyclic constant will likely >> result in SOE, so such classfiles are pretty much useless and therefore not >> likely to spread very far. > > This is a reasonable critique. Another critique is that the cost of tracking > down and parsing an extra attribute will undo a lot of the performance benefit > of having it, perhaps even causing a net slowdown. > > As we've discussed the idea a little more, my inclination (raised in the meeting > this week) is to give up on trying to add structural restrictions on constant > pools designed to facilitate cycle detection. It seems like we can, in fact, > detect cycles without extra guidance, and with minimal overhead in most cases. > Some details below. > > Big picture, the most attractive path forward seems to be: > - In 11, a CONSTANT_Dynamic with a cycle is an "error", reported via SOE at > resolution time > - When we introduce lazy condy resolution, we move the error detection to class > load time, with a check that we're comfortable isn't disrupting startup time > - As other potentially-cyclical constants are introduced, we use the same load > time checks > - We never require extra information from the compiler, or reject classes that > are hard to prove cycle-free > > One nice thing about that last point is that we don't have to worry about > incompatible support for "hard" classes in different releases. > > I'd love to hear about some experiments with this. We're talking about > performance impact, and all discussion to this point has been hypothetical. > Would be good to get some concrete feedback. > > ?Dan > > ----- > > Details: > > We started with a rule for format checking (JVMS 4.8) that says it's illegal for > a CONSTANT_Dynamic (or, in the future, other potentially-cyclical constants) to > refer, via static arguments, to itself?directly or indirectly. But we quickly > decided this sounded expensive, and we didn't want a heavy validation step to > mess with startup time. > > Hence, we explored various ways to prohibit otherwise-valid classes that would > be hard to prove cycle-free. Something like the RecursiveConstants attributes > seems to be the best option in that space. > > However, scrutinizing the original assumption, it seems reasonable that we can > optimize the check so that it's extremely lightweight when references are > "backward" (to previous entries in the constant pool), and only performs more > expensive analysis when references are "forward". Most classes should naturally > use backward references exclusively, since entries tend to be generated > bottom-up. (In theory. If there are counter examples produced by any mainstream > bytecode generators, that would be good to know.) > > So a dumb (worst case O(n^2)) but fast (very low constant cost) check could look > something like: > > void checkCP(int i) { > switch (tag(i)) { > case CONSTANT_Dynamic: > ... > for (int j : condyStaticArgs(i)) { > if (j >= i && tag(j) == CONSTANT_Dynamic) { > assertUnreachable(i, j); // can't reach i from j > } > } > ... > ... > } > } > > It's already necessary to check that constant pool references are well-typed, so > everything up to 'assertUnreachable' here is practically free*. The > 'assertUnreachable' call might be somewhat expensive, but it will rarely be > called in practice (and even then, we don't expect many structures to be > particularly deep). > > (*In the particular case of CONSTANT_Dynamic, there's an indirection through > BootstrapMethods. If a class heavily shares BootstrapMethods entries, then > there's a new cost here, iterating over the same static arguments multiple > times. Some optimization could be done to speed that up. For example, if there > are no forward references on the first pass, there will be no forward > references on later passes, when i is a larger number.) > > A good worst-case test would be a constant pool without cycles, built entirely > out of forward references, consisting of lots of deep CONSTANT_Dynamic trees. > Needless to say, we don't expect to encounter classes like this in the wild. > But, if we do, the big benefit here is that, while they'll be a little slower > to load, they'll still work. From karen.kinnear at oracle.com Tue Feb 20 15:52:51 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Tue, 20 Feb 2018 10:52:51 -0500 Subject: Valhalla EG minutes Feb 14, 2018 Message-ID: attendees: Tobi, Mr Simms, Dan H, Dan S, Frederic, Remi, Karen I. Condy 1. Condy reference implementation was pushed last week into JDK 11. 2. StackOverFlow handling/future LDC early cycle detection Dan S walked us through his StackOverFlow JVMS clarification for condy, specifically the ordering of resolution prior to throwing StackOverFlowError for JDK11 initial Condy release http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2018-February/000560.html AI: implementors - check if this clarification matches implementable behavior Dan: also described an incremental ldc early detection circularity proposal - not requiring candy?s to refer to entries earlier in the classfile - not depending on an attribute to keep current during retransformation - assume earlier references are the common case, so that is fastest - still work if not in order - need to do static cycle tracking - so slower question for ASM users - e.g. JRuby, Groovy - as they add Condy support - how often do they need forward references? AI: all - double-check implementation implications Dan S - if you want to ask Charlie Nutter to let us know for JRuby going forward ... post-meeting Update from Dan Smith: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2018-February/000570.html AI: All - check if works for ASM and implementors 3. Planned uses for condy in jdk? - Nothing in imminent plans - expect longer term constant Lambdas to use condy - lightweight - future: still exploring APIs for constants, switch, pattern match, ? Remi: Python, JRuby - all lambdas are constant Remi: wants support in javac behind a flag Dan S: it is in Amber Remi: wants a binary :-) - Dan S will pass on that message II. Nestmates 1. Lookup handling AI: Karen to send email with details - here it is: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2018-February/000567.html Note: javac will not be generating bridges for private members when nestmate support goes into JDK 11 (soon) protected members will still require bridges 2. Spec updates to JVM Ti, JDWP, JDI, java.lang.instrument http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2018-February/000541.html Request to update JVMTI retransformation to describe ability to add private methods. Recognize this is independent of Nestmates, but perhaps overdue if we intend this to be supported behavior. AI: Karen - review with past owners of JVMTI specification changes. III. Value Types Latest LWorld Value Types proposal: http://cr.openjdk.java.net/~acorn/LWorldValueTypesFeb13.pdf Latest rough draft JVMS: http://cr.openjdk.java.net/~fparain/L-world/L-World-JVMS-4b.pdf Feedback/Q&A: 1. creation of a new value type - Remi - why not vnew ? why default/withfield/withfield/withfield? - transformations - e.g. Byteman - easier if arguments are on the stack Frederic: First proposal had a factory bytecode, returning a single fully constructed value type rejected: concern: cost of pushing all arguments, method signature and attribute to how signature maps to fields Dan S: declared fields do not have an inherit ordering, so e.g. attribute to identify order - expected usage: factory method in the value class itself Dan: also want withfield exposed at the language level to allow tweaking one thing Karen: would be helpful to have a single way to create a value type or an object to allow more shared code - model is to move all toward a factory mechanism Frederic: - inside factory - it is not the same bytecodes for value type and object type creation - note: withfield returns a new value type - it does not have the same stack behavior as putfield Dan H: factory proposal is better than defaultvalue/withfield - less throwing away extra created value types for the interpreter Needs more discussion 2. what does verifier / class file format checker need to do in JVMS? from LWorld Value Type pdf Can verifier check bytecode validity? No - we do not want to eagerly load class files, so it doesn?t know if a bytecode is being applied to a value type or an object identity type AI: Karen - make sure the ?what can javac do for you?? static verification is added to static checking, probably ClassFileFormatError 3. withfield handling Remi: why withfield? Frederic: goal is to allow loop iteration with low cost Remi: why restrict to within the value class itself? Karen: concern: this creates a new value type, think of it as CopyOnWrite, it does NOT go through final and update an existing value type. So this is heavyweight Remi: could we have the language decide restrictions on its usage rather than the JVMS? Dan S: future - if we want a general purpose withfield - we may want to put that in with extended field access controls - e.g. separate read vs. write. At that time you could use withfield if the field were accessible. - e.g. with Records - may expose readability, not availability Frederic: concern about confusing people - withfield with an immutable object Dan S: language could make this clearer that this is not an assignment, but is a ?new? Opinions? 4. arrays We need a new bytecode to create a flattenable/non-nullable array existing bytecodes do not create flattenable arrays with the new model of container marking flattenable rather than by type note: if a field is marked as ACC_FLATTENABLE and you load the field and it is not a value type -> ICCE Dan S: initial value could indicate if this is flattenable or not Remi: C++ does this - it is not a good thing Karen: we do not want to have to pre-load the type to determine if it is flattenable, we require the field access flag in order to require pre-loading 5. Arrays and nullability Question: can you pass a VT[] where an Object[] is expected? Yes you can pass the argument, and sub typing works. Frederic: If you have an Object[], if you have non-flattenable values then elements are nullable, if you have flattenable values, then elements are not nullable 5. Generics and nullability Dan S: With generics, value types will work as is. In future, if we were to change a field to be non-nullable, then we could get NullPointerExceptions Karen: if we were to change a field to be non-nullable, then if we wanted to we could support a different layout, and that would require specialization if the field were non-nullable depending on the parameter type. This is a current open challenge - how to handle migration to non-nullable fields and arrays Note that in future we might want non-nullable identity objects as well as value types. To help migration, Brian would like us to find a way so that javac would detect a mismatch in expectations of nullability, so we catch them at compile time. Corrections welcome, thanks, Karen From brian.goetz at oracle.com Wed Feb 21 03:16:45 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 20 Feb 2018 19:16:45 -0800 Subject: Valhalla EG minutes Feb 14, 2018 In-Reply-To: References: Message-ID: <2D5AE75F-D50F-4F8E-A085-99F74DFF2209@oracle.com> > 3. Planned uses for condy in jdk? > - Nothing in imminent plans > - expect longer term constant Lambdas to use condy - lightweight > - future: still exploring APIs for constants, switch, pattern match, ? Using condy for method references and stateless lambdas could happen relatively soon. We?ve just finished a round of perf testing and found that the linkage costs are measurably lower. Probably in the next six months or so. From karen.kinnear at oracle.com Wed Feb 21 17:52:04 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Wed, 21 Feb 2018 12:52:04 -0500 Subject: Valhalla EG minutes Feb 14, 2018 In-Reply-To: References: Message-ID: <3F9421FB-1898-4E0B-8754-A84635F90FD1@oracle.com> JVMTI RedefineClasses spec handling of private methods is being tracked via: https://bugs.openjdk.java.net/browse/JDK-8192936 thanks, Karen > On Feb 20, 2018, at 10:52 AM, Karen Kinnear wrote: > > attendees: Tobi, Mr Simms, Dan H, Dan S, Frederic, Remi, Karen > > I. Condy > > 1. Condy reference implementation was pushed last week into JDK 11. > > 2. StackOverFlow handling/future LDC early cycle detection > Dan S walked us through his StackOverFlow JVMS clarification for condy, specifically the ordering of resolution > prior to throwing StackOverFlowError for JDK11 initial Condy release > > http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2018-February/000560.html > > AI: implementors - check if this clarification matches implementable behavior > > Dan: also described an incremental ldc early detection circularity proposal > - not requiring candy?s to refer to entries earlier in the classfile > - not depending on an attribute to keep current during retransformation > - assume earlier references are the common case, so that is fastest > - still work if not in order - need to do static cycle tracking - so slower > > question for ASM users - e.g. JRuby, Groovy - as they add Condy support - how > often do they need forward references? > > AI: all - double-check implementation implications > Dan S - if you want to ask Charlie Nutter to let us know for JRuby going forward ... > > post-meeting Update from Dan Smith: > http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2018-February/000570.html > > AI: All - check if works for ASM and implementors > > 3. Planned uses for condy in jdk? > - Nothing in imminent plans > - expect longer term constant Lambdas to use condy - lightweight > - future: still exploring APIs for constants, switch, pattern match, ? > > Remi: Python, JRuby - all lambdas are constant > Remi: wants support in javac behind a flag > Dan S: it is in Amber > Remi: wants a binary :-) - Dan S will pass on that message > > > II. Nestmates > > 1. Lookup handling > AI: Karen to send email with details > - here it is: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2018-February/000567.html > > Note: javac will not be generating bridges for private members when nestmate support goes into JDK 11 (soon) > protected members will still require bridges > > 2. Spec updates to JVM Ti, JDWP, JDI, java.lang.instrument > http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2018-February/000541.html > > Request to update JVMTI retransformation to describe ability to add private methods. Recognize this > is independent of Nestmates, but perhaps overdue if we intend this to be supported behavior. > > AI: Karen - review with past owners of JVMTI specification changes. > > III. Value Types > > Latest LWorld Value Types proposal: http://cr.openjdk.java.net/~acorn/LWorldValueTypesFeb13.pdf > Latest rough draft JVMS: http://cr.openjdk.java.net/~fparain/L-world/L-World-JVMS-4b.pdf > > Feedback/Q&A: > > 1. creation of a new value type - Remi > - why not vnew ? why default/withfield/withfield/withfield? > - transformations - e.g. Byteman - easier if arguments are on the stack > > Frederic: First proposal had a factory bytecode, returning a single fully constructed value type > rejected: concern: cost of pushing all arguments, method signature and attribute to how signature maps to fields > > Dan S: declared fields do not have an inherit ordering, so e.g. attribute to identify order > - expected usage: factory method in the value class itself > > Dan: also want withfield exposed at the language level to allow tweaking one thing > > Karen: would be helpful to have a single way to create a value type or an object to allow more shared code > - model is to move all toward a factory mechanism > > Frederic: > - inside factory - it is not the same bytecodes for value type and object type creation > - note: withfield returns a new value type - it does not have the same stack behavior as putfield > > Dan H: factory proposal is better than defaultvalue/withfield > - less throwing away extra created value types for the interpreter > > Needs more discussion > > 2. what does verifier / class file format checker need to do in JVMS? > from LWorld Value Type pdf > > Can verifier check bytecode validity? No - we do not want to eagerly load class files, so it doesn?t know if > a bytecode is being applied to a value type or an object identity type > > AI: Karen - make sure the ?what can javac do for you?? static verification is added to static checking, probably > ClassFileFormatError > > 3. withfield handling > Remi: why withfield? > Frederic: goal is to allow loop iteration with low cost > Remi: why restrict to within the value class itself? > > Karen: concern: this creates a new value type, think of it as CopyOnWrite, it does NOT go through final > and update an existing value type. So this is heavyweight > > Remi: could we have the language decide restrictions on its usage rather than the JVMS? > > Dan S: future - if we want a general purpose withfield - we may want to put that in with extended > field access controls - e.g. separate read vs. write. At that time you could use withfield if the field were > accessible. > - e.g. with Records - may expose readability, not availability > > Frederic: concern about confusing people - withfield with an immutable object > > Dan S: language could make this clearer that this is not an assignment, but is a ?new? > > Opinions? > > 4. arrays > We need a new bytecode to create a flattenable/non-nullable array > existing bytecodes do not create flattenable arrays with the new model of container marking flattenable > rather than by type > > note: if a field is marked as ACC_FLATTENABLE and you load the field and it is not a value type -> ICCE > > Dan S: initial value could indicate if this is flattenable or not > Remi: C++ does this - it is not a good thing > Karen: we do not want to have to pre-load the type to determine if it is flattenable, we require the field access flag > in order to require pre-loading > > 5. Arrays and nullability > > Question: can you pass a VT[] where an Object[] is expected? > Yes you can pass the argument, and sub typing works. > > Frederic: If you have an Object[], if you have non-flattenable values then elements are nullable, if you have flattenable values, then elements are not nullable > > 5. Generics and nullability > > Dan S: With generics, value types will work as is. > In future, if we were to change a field to be non-nullable, then we could get NullPointerExceptions > Karen: if we were to change a field to be non-nullable, then if we wanted to we could support a different layout, > and that would require specialization if the field were non-nullable depending on the parameter type. > > This is a current open challenge - how to handle migration to non-nullable fields and arrays > Note that in future we might want non-nullable identity objects as well as value types. > > To help migration, Brian would like us to find a way so that javac would detect a mismatch in expectations of nullability, > so we catch them at compile time. > > Corrections welcome, > thanks, > Karen > > > > > > > > > From john.r.rose at oracle.com Wed Feb 21 20:53:18 2018 From: john.r.rose at oracle.com (John Rose) Date: Wed, 21 Feb 2018 12:53:18 -0800 Subject: Valhalla EG minutes Feb 14, 2018 In-Reply-To: References: <3F9421FB-1898-4E0B-8754-A84635F90FD1@oracle.com> Message-ID: <0B1A0837-570E-4ADE-AC60-FC99C75CD175@oracle.com> On Feb 21, 2018, at 10:51 AM, Daniel Heidinga wrote: > > By "private methods" does that imply both static and instance methods? I hope not. > > With NestMates, the JVMS has been updated with the (non-normative) text: > --- > Because private methods may now be invoked from a nestmate class, it is no longer recommended to compile their invocation to invokespecial. (invokespecial may only be used for methods declared in the current class or a superclass.) A standard usage of invokevirtual or invokeinterface works just fine instead, with no special discussion necessary. > --- > implies that private instance methods will have vtable slots and adding a new private instance method will be a heavy weight operation due to modifying a vtable(!). vtable slots are an implementation technique not mentioned in the JVMS. Although it's hard to imagine a good JVM without vtables, the JVMS allows a range of approaches to them. Specifically, there's nothing in the JVMS (AFAIK) that requires private instance methods to be supported by vtable slots. The mere fact that we are using the bytecode point named "invokevirtual" says nothing whether it resolves to a vtable slot or uses some other mechanism for linkage, as needed. HotSpot happens to link such instructions in whatever way most directly reflects the access and morphism of the underlying method. All this is to say, what you are saying sounds like a difficultly with one or more implementations, and not with the logic of the spec. Am I missing something? ? John > > Adding static methods, private or not, is less problematic apart from interfaces due to the knock-on effects to iTables. > > We can live with the later (static) though we'd like to avoid the former (instance). > From karen.kinnear at oracle.com Thu Feb 22 00:54:56 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Wed, 21 Feb 2018 19:54:56 -0500 Subject: Valhalla EG minutes Feb 14, 2018 In-Reply-To: References: <3F9421FB-1898-4E0B-8754-A84635F90FD1@oracle.com> Message-ID: Dan, I was going to ask you if there were changes you would find more supportable than others. So the current RI for retransform classes today supports: ADD and DELETE for PRIVATE && (FINAL || STATIC) methods (We are still investigating history) > On Feb 21, 2018, at 1:51 PM, Daniel Heidinga wrote: > > Thanks Karen for the link to the bug. > > By "private methods" does that imply both static and instance methods? I hope not. > > With NestMates, the JVMS has been updated with the (non-normative) text: > --- > Because private methods may now be invoked from a nestmate class, it is no longer recommended to compile their invocation to invokespecial. (invokespecial may only be used for methods declared in the current class or a superclass.) A standard usage of invokevirtual or invokeinterface works just fine instead, with no special discussion necessary. > --- > implies that private instance methods will have vtable slots and adding a new private instance method will be a heavy weight operation due to modifying a vtable(!). I?m glad we are having this conversation. Just to clarify - we have been able to invoke private methods via invokevirtual for years, so that capability is not new. So I do not see how nestmates JVMS changes make any changes to invokevirtual. Totally agree with the goal of not allowing modification of a vtable (and all tables inherited from it), due to heavy operational costs. With the more detailed restrictions above - does that make it more possible to support private instance method addition/deletion - i.e. if the instance methods have to be final and private? > > Adding static methods, private or not, is less problematic apart from interfaces due to the knock-on effects to iTables. Agree that static methods are easier. Not to go into implementation details, but I don?t understand a relationship between static methods and itables since static methods are not inherited. > > We can live with the later (static) though we'd like to avoid the former (instance). thanks, Karen > > --Dan > > ----- Original message ----- > From: Karen Kinnear > Sent by: "valhalla-spec-experts" > To: valhalla-spec-experts > Cc: > Subject: Re: Valhalla EG minutes Feb 14, 2018 > Date: Wed, Feb 21, 2018 12:52 PM > > JVMTI RedefineClasses spec handling of private methods is being tracked via: > https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8192936&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=LBQnmyrHQkEBElM8bAxhzfwLG2HbsYDdzEznFrQoob4&m=KyGIkr3_m7an4VEDrmlaWzy_eUhQvpypda7FucEbf-M&s=PAZDayMHueNWfP6wI6s3I-XfDpiobFf3OPhEerTcI7s&e= > > thanks, > Karen > > > On Feb 20, 2018, at 10:52 AM, Karen Kinnear wrote: > > > > attendees: Tobi, Mr Simms, Dan H, Dan S, Frederic, Remi, Karen > > > > I. Condy > > > > 1. Condy reference implementation was pushed last week into JDK 11. > > > > 2. StackOverFlow handling/future LDC early cycle detection > > Dan S walked us through his StackOverFlow JVMS clarification for condy, specifically the ordering of resolution > > prior to throwing StackOverFlowError for JDK11 initial Condy release > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.openjdk.java.net_pipermail_valhalla-2Dspec-2Dexperts_2018-2DFebruary_000560.html&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=LBQnmyrHQkEBElM8bAxhzfwLG2HbsYDdzEznFrQoob4&m=KyGIkr3_m7an4VEDrmlaWzy_eUhQvpypda7FucEbf-M&s=wjLLp1GHaRDE0Jpm_PQVRmoCst7uwiVJ7luwjBu_E7c&e= > > > > AI: implementors - check if this clarification matches implementable behavior > > > > Dan: also described an incremental ldc early detection circularity proposal > > - not requiring candy?s to refer to entries earlier in the classfile > > - not depending on an attribute to keep current during retransformation > > - assume earlier references are the common case, so that is fastest > > - still work if not in order - need to do static cycle tracking - so slower > > > > question for ASM users - e.g. JRuby, Groovy - as they add Condy support - how > > often do they need forward references? > > > > AI: all - double-check implementation implications > > Dan S - if you want to ask Charlie Nutter to let us know for JRuby going forward ... > > > > post-meeting Update from Dan Smith: > > https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.openjdk.java.net_pipermail_valhalla-2Dspec-2Dexperts_2018-2DFebruary_000570.html&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=LBQnmyrHQkEBElM8bAxhzfwLG2HbsYDdzEznFrQoob4&m=KyGIkr3_m7an4VEDrmlaWzy_eUhQvpypda7FucEbf-M&s=64MFsDQjWyxhx3hjvJwpEU1DNm9KwcaP1QH0InfASu8&e= > > > > AI: All - check if works for ASM and implementors > > > > 3. Planned uses for condy in jdk? > > - Nothing in imminent plans > > - expect longer term constant Lambdas to use condy - lightweight > > - future: still exploring APIs for constants, switch, pattern match, ? > > > > Remi: Python, JRuby - all lambdas are constant > > Remi: wants support in javac behind a flag > > Dan S: it is in Amber > > Remi: wants a binary :-) - Dan S will pass on that message > > > > > > II. Nestmates > > > > 1. Lookup handling > > AI: Karen to send email with details > > - here it is: https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.openjdk.java.net_pipermail_valhalla-2Dspec-2Dexperts_2018-2DFebruary_000567.html&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=LBQnmyrHQkEBElM8bAxhzfwLG2HbsYDdzEznFrQoob4&m=KyGIkr3_m7an4VEDrmlaWzy_eUhQvpypda7FucEbf-M&s=-fdQS3jfPiI_HfNasVUIBYyqRqmyOYMMHwPH89N2DjE&e= > > > > Note: javac will not be generating bridges for private members when nestmate support goes into JDK 11 (soon) > > protected members will still require bridges > > > > 2. Spec updates to JVM Ti, JDWP, JDI, java.lang.instrument > > https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.openjdk.java.net_pipermail_valhalla-2Dspec-2Dexperts_2018-2DFebruary_000541.html&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=LBQnmyrHQkEBElM8bAxhzfwLG2HbsYDdzEznFrQoob4&m=KyGIkr3_m7an4VEDrmlaWzy_eUhQvpypda7FucEbf-M&s=EuwhQi4t5_6WZczp-jStDfWc5jIRMy1mVR-8yQOJWko&e= > > > > Request to update JVMTI retransformation to describe ability to add private methods. Recognize this > > is independent of Nestmates, but perhaps overdue if we intend this to be supported behavior. > > > > AI: Karen - review with past owners of JVMTI specification changes. > > > > III. Value Types > > > > Latest LWorld Value Types proposal: https://urldefense.proofpoint.com/v2/url?u=http-3A__cr.openjdk.java.net_-7Eacorn_LWorldValueTypesFeb13.pdf&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=LBQnmyrHQkEBElM8bAxhzfwLG2HbsYDdzEznFrQoob4&m=KyGIkr3_m7an4VEDrmlaWzy_eUhQvpypda7FucEbf-M&s=wwzmVDbZdp90GAS939tSP6TLrYuRPlO3OrwVvZYOYBg&e= > > Latest rough draft JVMS: https://urldefense.proofpoint.com/v2/url?u=http-3A__cr.openjdk.java.net_-7Efparain_L-2Dworld_L-2DWorld-2DJVMS-2D4b.pdf&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=LBQnmyrHQkEBElM8bAxhzfwLG2HbsYDdzEznFrQoob4&m=KyGIkr3_m7an4VEDrmlaWzy_eUhQvpypda7FucEbf-M&s=98mDp23fCot9T029MVb6vUgw9TsU_Dr42EDx_FnnBrw&e= > > > > Feedback/Q&A: > > > > 1. creation of a new value type - Remi > > - why not vnew ? why default/withfield/withfield/withfield? > > - transformations - e.g. Byteman - easier if arguments are on the stack > > > > Frederic: First proposal had a factory bytecode, returning a single fully constructed value type > > rejected: concern: cost of pushing all arguments, method signature and attribute to how signature maps to fields > > > > Dan S: declared fields do not have an inherit ordering, so e.g. attribute to identify order > > - expected usage: factory method in the value class itself > > > > Dan: also want withfield exposed at the language level to allow tweaking one thing > > > > Karen: would be helpful to have a single way to create a value type or an object to allow more shared code > > - model is to move all toward a factory mechanism > > > > Frederic: > > - inside factory - it is not the same bytecodes for value type and object type creation > > - note: withfield returns a new value type - it does not have the same stack behavior as putfield > > > > Dan H: factory proposal is better than defaultvalue/withfield > > - less throwing away extra created value types for the interpreter > > > > Needs more discussion > > > > 2. what does verifier / class file format checker need to do in JVMS? > > from LWorld Value Type pdf > > > > Can verifier check bytecode validity? No - we do not want to eagerly load class files, so it doesn?t know if > > a bytecode is being applied to a value type or an object identity type > > > > AI: Karen - make sure the ?what can javac do for you?? static verification is added to static checking, probably > > ClassFileFormatError > > > > 3. withfield handling > > Remi: why withfield? > > Frederic: goal is to allow loop iteration with low cost > > Remi: why restrict to within the value class itself? > > > > Karen: concern: this creates a new value type, think of it as CopyOnWrite, it does NOT go through final > > and update an existing value type. So this is heavyweight > > > > Remi: could we have the language decide restrictions on its usage rather than the JVMS? > > > > Dan S: future - if we want a general purpose withfield - we may want to put that in with extended > > field access controls - e.g. separate read vs. write. At that time you could use withfield if the field were > > accessible. > > - e.g. with Records - may expose readability, not availability > > > > Frederic: concern about confusing people - withfield with an immutable object > > > > Dan S: language could make this clearer that this is not an assignment, but is a ?new? > > > > Opinions? > > > > 4. arrays > > We need a new bytecode to create a flattenable/non-nullable array > > existing bytecodes do not create flattenable arrays with the new model of container marking flattenable > > rather than by type > > > > note: if a field is marked as ACC_FLATTENABLE and you load the field and it is not a value type -> ICCE > > > > Dan S: initial value could indicate if this is flattenable or not > > Remi: C++ does this - it is not a good thing > > Karen: we do not want to have to pre-load the type to determine if it is flattenable, we require the field access flag > > in order to require pre-loading > > > > 5. Arrays and nullability > > > > Question: can you pass a VT[] where an Object[] is expected? > > Yes you can pass the argument, and sub typing works. > > > > Frederic: If you have an Object[], if you have non-flattenable values then elements are nullable, if you have flattenable values, then elements are not nullable > > > > 5. Generics and nullability > > > > Dan S: With generics, value types will work as is. > > In future, if we were to change a field to be non-nullable, then we could get NullPointerExceptions > > Karen: if we were to change a field to be non-nullable, then if we wanted to we could support a different layout, > > and that would require specialization if the field were non-nullable depending on the parameter type. > > > > This is a current open challenge - how to handle migration to non-nullable fields and arrays > > Note that in future we might want non-nullable identity objects as well as value types. > > > > To help migration, Brian would like us to find a way so that javac would detect a mismatch in expectations of nullability, > > so we catch them at compile time. > > > > Corrections welcome, > > thanks, > > Karen > > > > > > > > > > > > > > > > > > > > > From daniel.smith at oracle.com Fri Feb 23 23:47:37 2018 From: daniel.smith at oracle.com (Dan Smith) Date: Fri, 23 Feb 2018 16:47:37 -0700 Subject: A "RecursiveConstants" attribute In-Reply-To: <75563174.2472564.1518865303213.JavaMail.zimbra@u-pem.fr> References: <131B1F8D-F49C-47F8-BE7C-A660CBE1FCB5@oracle.com> <2CEA5EE3-6F51-4806-9CA5-2BAFCF0620E8@oracle.com> <95f19422-86a4-115b-eae7-d274ab57e072@oracle.com> <60BB9395-3B8B-4901-A8AD-99B3C209A005@oracle.com> <29CA13B1-7A8B-446D-8235-98CBE978AB97@oracle.com> <75563174.2472564.1518865303213.JavaMail.zimbra@u-pem.fr> Message-ID: <6BDD2D9A-1399-46ED-960C-6816C310FAAF@oracle.com> > On Feb 17, 2018, at 4:01 AM, Remi Forax wrote: > >> Most classes should naturally use backward references exclusively, since entries tend to be generated bottom-up. >> (In theory. If there are counter examples produced by any mainstream bytecode generators, that would be good to know.) > > Currently, javac and ecj tends to generate their constants in the "wrong order". > ecj: > #15 = Methodref #16.#18 // java/lang/Class.desiredAssertionStatus:()Z > #16 = Class #17 // java/lang/Class > #17 = Utf8 java/lang/Class > #18 = NameAndType #19:#20 // desiredAssertionStatus:()Z > #19 = Utf8 desiredAssertionStatus > #20 = Utf8 ()Z > > javac: > ... > #20 = Methodref #65.#66 // java/lang/Class.desiredAssertionStatus:()Z > ... > #65 = Class #75 // java/lang/Class > #66 = NameAndType #76:#77 // desiredAssertionStatus:()Z > ... > #75 = Utf8 java/lang/Class > #76 = Utf8 desiredAssertionStatus > #77 = Utf8 ()Z > > It means nothing now because we only cares about constants that can introduce cycles so CONSTANT_Dynamic, it may be an issue in the future if by example a NameAndType can reference a constant with cycle, the code of javac and ecj will have to be changed. Yep, looks like my depth-first assumption is wrong. Here's a javac constant pool for an empty class: #1 = Methodref #3.#10 // java/lang/Object."":()V #2 = Class #11 // ConstantPool #3 = Class #12 // java/lang/Object #4 = Utf8 #5 = Utf8 ()V #6 = Utf8 Code #7 = Utf8 LineNumberTable #8 = Utf8 SourceFile #9 = Utf8 ConstantPool.java #10 = NameAndType #4:#5 // "":()V #11 = Utf8 ConstantPool #12 = Utf8 java/lang/Object I can kind of squint and see the logic, but there's no simple pattern here. Forward and backward references both occur. I'll ask our code gen experts about what they'd expect from Dynamic, and whether it's easy to assume they always point in one direction or the other. ?Dan From paul.sandoz at oracle.com Mon Feb 26 20:19:27 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 26 Feb 2018 12:19:27 -0800 Subject: Valhalla EG minutes Feb 14, 2018 In-Reply-To: References: Message-ID: <093435E6-FA38-4EAE-852C-20B6D226B788@oracle.com> > On Feb 20, 2018, at 7:52 AM, Karen Kinnear wrote: > > attendees: Tobi, Mr Simms, Dan H, Dan S, Frederic, Remi, Karen > > I. Condy > > 1. Condy reference implementation was pushed last week into JDK 11. > A next round of development will focus on the BootstrapCallInfo API and BSM invocation accepting such an argument. It?s currently in the condy-folding branch of amber. Observation: this can make recursive resolution errors more explicit, resulting in a more ?obvious? stack overflow. Plus constants can be resolved outside of the context of the BSM invocation (if say the BSM returns something that itself operates on the BCI instance passed to it). > > 3. Planned uses for condy in jdk? > - Nothing in imminent plans > - expect longer term constant Lambdas to use condy - lightweight It would be great if javac could at some point also ?intern? duplicate lambda expressions. > - future: still exploring APIs for constants, switch, pattern match, ? > > Remi: Python, JRuby - all lambdas are constant > Remi: wants support in javac behind a flag > Dan S: it is in Amber Vicente recently pushed a patch to the condy-folding branch of amber [1] to remove the flag, this feature is now always on when building JDK source (and was always on by default for stuff outside the JDK). Paul. [1] http://hg.openjdk.java.net/amber/amber/rev/75e0076b3ef0 > Remi: wants a binary :-) - Dan S will pass on that message > From john.r.rose at oracle.com Tue Feb 27 03:59:10 2018 From: john.r.rose at oracle.com (John Rose) Date: Mon, 26 Feb 2018 19:59:10 -0800 Subject: Valhalla EG minutes Feb 14, 2018 In-Reply-To: References: Message-ID: On Feb 20, 2018, at 7:52 AM, Karen Kinnear wrote: > > attendees: Tobi, Mr Simms, Dan H, Dan S, Frederic, Remi, Karen > > ... > III. Value Types > > Latest LWorld Value Types proposal: http://cr.openjdk.java.net/~acorn/LWorldValueTypesFeb13.pdf > Latest rough draft JVMS: http://cr.openjdk.java.net/~fparain/L-world/L-World-JVMS-4b.pdf > > Feedback/Q&A: > > 1. creation of a new value type - Remi > - why not vnew ? why default/withfield/withfield/withfield? > - transformations - e.g. Byteman - easier if arguments are on the stack > > Frederic: First proposal had a factory bytecode, returning a single fully constructed value type > rejected: concern: cost of pushing all arguments, method signature and attribute to how signature maps to fields Yep. This is really a FAQ. I'll take a shot at it. "Why not just do vnew"? Because vnew would be a complicated construct requiring, for each vnew instruction, a detailed list of fields and values which amount to a whole record type. This would be resolved in the constant pool (again, for each vnew instruction). The constant pool would have to support a new variadic constant type, as a primitive constant type to supply the needs of vnew. This is beyond the complexity of any other constant pool type to date. The data required to correctly link is analogous to (but more complex than) the code generation of the enum-switch code generation or strings-in-switch. Like those features, it is best implemented by a metafactory, not a single VM instruction. Also, we have an independent need for one-field update to values (withfield) to express "wither" methods and similar shapes. But vdefault + withfield* covers the same functionality as vnew. And it reuses CONSTANT_Fieldref. Also, Java constructors for value types translate naturally into vdefault + withfield*. They do not translate naturally into vnew. Some Java constructors translate naturally into vnew, but only those trivial ones which perform no logic other than blank final field assignment. Most Java constructors are not so trivial, and we do not intend to limit the expressiveness of value type constructors in such a way. One objection to use of withfield is that it is hard to implement a series of withfield instructions in the interpreter, without creating many intermediate versions of a variable, each in its own buffer. Of course the JIT has no such problem, since it knows the liveness of each intermediate value, and can immediately reuse storage known to be unique to a particular value, to create the next version after a withfield. The problem in the interpreter can also be fixed, e.g., by appropriate use of approximate liveness tracking, such as reference counts. Such a local implementation issue must not be allowed to overturn the more basic design points noted above, which affect all classfiles and all compilers. Specifically: If a value is loaded (using aload or vnew) to TOS, it may well have a reference count of 2 or more. But the first withfield will rebuffer it to unaliased storage. A chain of further withfields can just modify it in place. This pattern (of provably unaliased value buffers) can be detected on the fly by mechanical reference counts, or else by a lightweight pre-pass run at class load time, recoding each subsequent withfield after the first as "patching_withfield". Classfiles would not be allowed to mention patching_withfield, because it is grossly unsafe, but the interpreter would safely run those instructions where the class loader had determined that the operation is safe. There are lots of ways to skin this cat. > Dan S: declared fields do not have an inherit ordering, so e.g. attribute to identify order > - expected usage: factory method in the value class itself > > Dan: also want withfield exposed at the language level to allow tweaking one thing Yes, this is very important. > Karen: would be helpful to have a single way to create a value type or an object to allow more shared code > - model is to move all toward a factory mechanism For object classes this single way is new + + putfield*. Plus reflective versions. For value classes this single way is + vdefault + withfield*. Plus reflective. The two cases are in close correspondence in order to allow constructors to be translated compatibly for both object classes and value classes. If we go to factory mechanisms it will be very frustrating (as with MVT) to pain-gram the constructor translation strategy. Remember that constructors usually intermix field sets with method calls (including on 'this') and control flow. You can't fit that into a factory or metafactory. You have to use a sequence of elemental operations expressed by bytecodes. > Frederic: > - inside factory - it is not the same bytecodes for value type and object type creation > - note: withfield returns a new value type - it does not have the same stack behavior as putfield There are a superficial differences, but the similarities outweigh them. The initial binding of 'this' in an object constructor is the result of the 'new' instruction passed as the receiver argument to invokespecial , and stored in local number zero. The initial binding of 'this' in a value constructor is produced locally, using 'vdefault', and stored (if necessary) in a local which does not correspond to any of the incoming arguments. (In fact it could be stored at the root of the stack, in many but not all cases.) Those two sequences are different but in the rest of the constructor body, 'this' is uniformly available, perhaps in a partially uninitialized state. It's true: The object might have blank final fields containing their default values because they have not been putfield-ed to. The value might have fields containing their default values because they have not be withfield-ed to. There is no difference to the programmer: Getfield in either case will produce the default value. The rules of definite unassignment in the JLS make it hard to see this, of course, but it's there in both cases. (Probably the JLS rules, as written today, are enough to ensure that *no* value type can contort itself to observe an uninitialized field, but this is not a necessary point. The JVM can see a default field value, if it wants, for both objects and values.) After entry, the constructor body runs control flow and method calls, mixed with assignments (one per field along any path, per JLS rules) to the fields of the new instance (either object or value). At the end of the constructor, when it returns normally, the method returns void, leaving the passed-in object, in the required state. The factory method returns the new value, in the required state. The way I see it, these differences are at the surface, not in the basic semantics of constructors for the two kinds of classes. A final observation: putfield and withfield have different stack behaviors, in that putfield doesn't return a result (just keeps hammering the same object over and over) while withfield returns a new version of the value. Again, this is a surface difference, because the translation of a value class constructor must simply pop the new version of the value and store it in the local variable (mentioned above) which was initially populated with a "vdefault". In some cases, a translator may be able to keep the value on the JVM stack through a series of peephole optimizations, but this point doesn't affect the semantic parity between the two classes of constructors. (Another note to us implementors: It would be somewhat reasonable, though uglier, if withfield took its first operand from a local rather than from the stack, and updated that local in place. The iinc instruction is an pre-existing example of this. There would be two benefits to using such an in-place withfield: First, there would be no need in a constructor or wither to push the local containing this on the stack and then pop the new version back off, for a small reduction in bytecode size. Second, there might be less duplication of buffers in an interpreter which used reference counts to track buffer usage. But these small advantages do not seem to me to outweigh the relative cleanliness of the current design of withfield.) > Dan H: factory proposal is better than defaultvalue/withfield > - less throwing away extra created value types for the interpreter I hate to disappoint Dan, but that would be the tail wagging the dog; see above. > > 3. withfield handling > Remi: why withfield? > Frederic: goal is to allow loop iteration with low cost > Remi: why restrict to within the value class itself? > > Karen: concern: this creates a new value type, think of it as CopyOnWrite, it does NOT go through final > and update an existing value type. So this is heavyweight > > Remi: could we have the language decide restrictions on its usage rather than the JVMS? That's the current scheme: We keep withfield private even if the field is public. This allows class writers to decide independently (a) how visible to make fields for *reading*, and (b) how much trust they give clients to *create* new values with arbitrary field settings. If a value type represents a checked capability, it must not be possible for external users to forge arbitrary new capabilities, in an unchecked manner. But public withfield would do this, or else force API designers always to hide fields behind accessors. Same point even if the value type isn't a capability, but just asserts its right to validate and/or normalize field values. Raw withfield would subvert that. A future version of withfield might allow a class to open up "raw" withfield access. But it is more likely that we will create a way for a class to open up a more "cooked" version of such access, such as some hook for "dumping" the state of a value and "reassembling" it from an altered state. In fact, that's just getfield* + constructor, in many cases, so the API points are already there. Crucially, the class writer has control over validation, normalization, permission checks, etc., in the reconstruction of the new value state. > > Dan S: future - if we want a general purpose withfield - we may want to put that in with extended > field access controls - e.g. separate read vs. write. At that time you could use withfield if the field were > accessible. > - e.g. with Records - may expose readability, not availability Yes. This is possible, even with object classes. I like to name this feature "sealed fields", since the sealed field is "usable but not redefinable", where "usable" = readable and "redefinable" = writable. (By analogy with sealed interfaces.) Value fields are sealed by default for reasons given above, but could be unsealed. Object fields are sealed if final, but unsealed if non-final. An intermediate state might make sense. But: When I work out use cases for these intermediate states, I don't see anything promising yet. So I think we can stick with what we have now and make a note to reevaluate later. > Frederic: concern about confusing people - withfield with an immutable object > > Dan S: language could make this clearer that this is not an assignment, but is a ?new? > > Opinions? Yes, we need a new syntax at the source code level to make it clear that (a) an old value instance is being operated on, but (b) a new version of that instance is the result of the operation. It seems promising to me to allow something like a constructor body (with field values in scope under their own names) to pull this off. But I don't know what this looks like, except maybe in the easier case of a "named reconstructor" within a class: __ByValue class Rational { public final long num, den; public Rational(long num, long den) { this.num = num; this.den = den; assert(den != 0); } public __Reconstructor neg() { // constructor rules here, except fields appear mutable num *= -1; // aload L0; dup; getfield num; iconst -1; imul; withfield num; astore L0 return; // aload L0; areturn } } The rule is, inside a constructor you can assign to your fields. That's the rule already, of course, but in a reconstructor you can do the same things, *plus* refer to previous field values. (And perhaps 'this', although there's some doubt about which version should apply: Either the original or the current state.) > > 4. arrays > We need a new bytecode to create a flattenable/non-nullable array > existing bytecodes do not create flattenable arrays with the new model of container marking flattenable > rather than by type Whoa. I haven't yet seen a strong reason to do per-container flattenability. I'd rather not do this unless there is a strong reason. And even if we need this, there's no reason to burn a new bytecode; it can be a reflective call, as java.lang.reflect.Array.newNullableInstance. (Yes, I think the existing bytecode should make the correct flatness.) > ? > 5. Arrays and nullability > > Question: can you pass a VT[] where an Object[] is expected? > Yes you can pass the argument, and sub typing works. > > Frederic: If you have an Object[], if you have non-flattenable values then elements are nullable, if you have flattenable values, then elements are not nullable Yep. We are eating the cost of buffering up flat elements inside aaload, even though that requires a data-dependent check. The costs of this can be reduced in the JIT, using profiling. But if we allow flatness to be a new randomly changing bit on instances, the profiling will be less effective (until we profile that bit, perhaps, or perhaps not). The JVM works very hard to feed type analysis to the JIT. Let's think twice before we make flatness *not* a property of types. (Note to self: This argument applies somewhat to frozen arrays also. But there reusing the same types seems to be a forced move. Non-flat arrays of flat types is *not* a forced move.) > 5. Generics and nullability > > Dan S: With generics, value types will work as is. > In future, if we were to change a field to be non-nullable, then we could get NullPointerExceptions > Karen: if we were to change a field to be non-nullable, then if we wanted to we could support a different layout, > and that would require specialization if the field were non-nullable depending on the parameter type. > > This is a current open challenge - how to handle migration to non-nullable fields and arrays We are working cases on this. It looks like the issue, often, is how tolerant to be about "polluting nulls" in code which is not fully type-correct. (It is not type-correct when old classfiles co-exist with new ones.) The decision points are at places like getfield, putfield, invoke, return, aastore, checkcast. When do we allow a polluting null to pass by, and when do we throw NPE? The basic right move, I think, is to throw NPE as soon as possible, as a service to the JIT, and also to the user who wants to know something is fishy in the code. But legacy classfiles must never throw NPE, since they can't know any better. This implies that a number of our bytecodes must be sensitive to the classfile version, and throw NPE only on recompiled code. This behavioral divergence is not (IMO) enough to warrant a new slew of bytecodes, but it is a tight fit to support both old and new behaviors. Maybe we want a bytecode *prefix* that means "allow polluting nulls", and treat all relevant bytecodes in *old* code as if that prefix were present. > Note that in future we might want non-nullable identity objects as well as value types. Yes. In which case the prefix might mean "invert treatment of polluting nulls". Or maybe the prefix comes with flag bit to explicitly get the NPE behavior vs. pass behavior. We could also handle statically mandated null checks with a new nullcheck bytecodes or even just invokestatic Os.requireNonNull. That doesn't feel right to me, since the right behavior should also look relatively simple in bytecodes, compared to the wrong behavior. > To help migration, Brian would like us to find a way so that javac would detect a mismatch in expectations of nullability, > so we catch them at compile time. Ooh, that's a good idea. When javac resolves use of an API in a JAR that isn't up to Valhalla classfile version, it could check to see if the API signature mentions return types which are statically known to be value types. (And in other covariant positions, maybe.) That's an example of a classfile that could introduce polluting nulls, and merits a warning. Thanks for pushing all this forward! ? John From Tobi_Ajila at ca.ibm.com Tue Feb 27 18:12:07 2018 From: Tobi_Ajila at ca.ibm.com (Tobi Ajila) Date: Tue, 27 Feb 2018 13:12:07 -0500 Subject: Valhalla EG minutes Feb 14, 2018 In-Reply-To: <3F9421FB-1898-4E0B-8754-A84635F90FD1@oracle.com> References: <3F9421FB-1898-4E0B-8754-A84635F90FD1@oracle.com> Message-ID: > 4. arrays > We need a new bytecode to create a flattenable/non-nullable array > existing bytecodes do not create flattenable arrays with the new model of container marking flattenable > rather than by type On a related note, it might be worth it to also consider the implication of flattened arrays and atomic operations. The Varhandle API currently supports a variety of atomic updates on elements of both primitive and reference arrays. Platform support for atomic updates on memory <= 64bits is fairly standard, but supporting anything larger becomes more difficult. This probably falls under the general category of 'tearing and atomicity concerns with writing flattenable values'. I suspect the answer might be to disallow atomic operations on flattenable fields/arrays and to allow tearing for updates, but this is probably worth a discussion. --Tobi From john.r.rose at oracle.com Tue Feb 27 19:45:16 2018 From: john.r.rose at oracle.com (John Rose) Date: Tue, 27 Feb 2018 11:45:16 -0800 Subject: Valhalla EG minutes Feb 14, 2018 In-Reply-To: References: <3F9421FB-1898-4E0B-8754-A84635F90FD1@oracle.com> Message-ID: On Feb 27, 2018, at 10:12 AM, Tobi Ajila wrote: > > > 4. arrays > > We need a new bytecode to create a flattenable/non-nullable array > > existing bytecodes do not create flattenable arrays with the new model of container marking flattenable > > rather than by type > On a related note, it might be worth it to also consider the implication of flattened arrays and atomic operations. The Varhandle API currently supports a variety of atomic updates on elements of both primitive and reference arrays. Platform support for atomic updates on memory <= 64bits is fairly standard, but supporting anything larger becomes more difficult. Definitely. I expect that an attempt to use a VarHandle atomic on a multi-word array element must fail with an exception rather than produce an indeterminate result. > This probably falls under the general category of 'tearing and atomicity concerns with writing flattenable values'. I suspect the answer might be to disallow atomic operations on flattenable fields/arrays and to allow tearing for updates, but this is probably worth a discussion. Yes, that's probably true. In our current prototyping, we are leaving non-tearing on the shelf for later, except in the important case of volatile fields, which we'll hack for now as separately buffered values. But this doesn't rescue flat arrays. I would advise a library author to use Object[] arrays with atomics, rather than flat arrays. I *do* want to add multi-word value-type atomics, but this is an optimization for later. Actually, there is something we can (and should) do sooner rather than later: Allow an __Atomic modifier (paint bikeshed later) on __ByValue classes which will force the JVM to ensure atomicity of all instances of the class, regardless of use-site annotation (with or without volatile). *This* would force the affected value-type arrays to be non-flat (though they would still reject nulls). ? John From Daniel_Heidinga at ca.ibm.com Wed Feb 28 16:49:37 2018 From: Daniel_Heidinga at ca.ibm.com (Daniel Heidinga) Date: Wed, 28 Feb 2018 16:49:37 +0000 Subject: Valhalla EG minutes Feb 14, 2018 In-Reply-To: <0B1A0837-570E-4ADE-AC60-FC99C75CD175@oracle.com> References: <0B1A0837-570E-4ADE-AC60-FC99C75CD175@oracle.com>, <3F9421FB-1898-4E0B-8754-A84635F90FD1@oracle.com> Message-ID: >All this is to say, what you are saying sounds like a difficultly >with one >or more implementations, and not with the logic of the spec. Am I >missing >something? I haven't seen a rationale for the proposed spec change yet (maybe I missed it?) which only leaves implementation costs as a discussion point. Either the cost to bring the RI into compliance with the current spec or to update other implementations to bless the spec divergence. If you can share rationale beyond "this is what Hotspot has chosen to do", then we can a spec discussion on that basis. Regards, --Dan > >? John >> >> Adding static methods, private or not, is less problematic apart >from interfaces due to the knock-on effects to iTables. >> >> We can live with the later (static) though we'd like to avoid the >former (instance). >> > >