From karen.kinnear at oracle.com Tue Oct 3 20:09:39 2017 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Tue, 3 Oct 2017 16:09:39 -0400 Subject: Valhalla EG meeting Sept 27 - and Sept 13 minutes In-Reply-To: <6457F745-0CB9-46E5-898D-3F5D49DDA3F6@oracle.com> References: <6457F745-0CB9-46E5-898D-3F5D49DDA3F6@oracle.com> Message-ID: <2D14B9C2-B130-443A-82E6-2AEA6D081821@oracle.com> For which we are grateful Paul - agree with your priorities give the timing goal of the condy changes. thanks, Karen > On Sep 27, 2017, at 12:38 PM, Paul Sandoz wrote: > > >> On 27 Sep 2017, at 06:50, Karen Kinnear > wrote: >> >> Reminder - meeting TODAY, Sept 27 9am PT/noon ET/ >> dial-in: https://oracle.zoom.us/j/5249803466 >> >> AI: Karen send Dan H example in which a BSM can be used for both indy and condy >> AI: Dan Smith version of Condy JVMS changes (not an appendix >> AI: nestmates JVMS comments: http://cr.openjdk.java.net/~dlsmith/private-access.html >> Remi proposal: rename attribute to ?NestHost" >> AI: Karen: write-up preparation relative to selection >> AI: Karen - ask Paul progress on VarHandles and atomicity relative to Value Types >> > > Alas very little :-( Most effort has been focused on bashing condy (minus BSCI) into shape for integration. > > Paul. From karen.kinnear at oracle.com Tue Oct 3 20:53:37 2017 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Tue, 3 Oct 2017 16:53:37 -0400 Subject: Valhalla EG minutes Sept 27 In-Reply-To: References: Message-ID: <0961B07D-5261-4E05-9383-208D84567C85@oracle.com> Next Meeting Wednesday October 11 9am PT/ noon ET/? AI: Dan Smith - update of Condy JVMS changes - without BootstrapCallInfo AI: John - email of what is coming - including constant bits, constantGroup (and why more than BsCI provides) AI: All - nestmate JVMS feedback for Dan S attendees: Remi, John, Bjorn, Dan H, Frederic, Lois, Mr Simms, Karen I. MVT: reiterate that we are targeting an Early Access Binary release this allows us to be more responsive for a smaller set of targeted customers when value types goes into a real release we will go through the JCP J1 MVT talk was rejected. noted concern: consolidation repository is slow MVT implementation: field with value type Frederic: if you generate a classfile today we always flatten fields which are value types optional flattening should be coming soon (ed. note: Tobias pushed Sept 29) II. Condy: Goal is to phase this into releases, with phase 1 in 18.3. JVMS will remove the BootstrapCallInfo for phase 1, so no support for lazy resolution. Internally we have prototyped it (and use internally for some edge cases) Changes that will be visible: Today package spec: says as if by MethdHandle.invoke if short arg list, or MethodHandle.invokewithargs otherwise To support > 251 arguments, java.lang.invoke.MethodHandle.invokewithargs will pass a trailing arg list and not check size [ed. note - please correct if the above was inaccurate] Concerns with lazy resolution (phase 2): 1. Lois - with Jigsaw you can now call add-reads or add-exports which could change the resolution of static args. John: that is ok. Longer term plan is to provide bootstrap methods that use the BsCI to 1) eagerly or lazily resolve any number of static arguments 2) lazily resolve ?if present? - i.e. if already successfully resolved, but do not trigger resolution 3) support symbolic reference APIs (see Constable) - get to original symbolic reference - even if resolved successfully or failed resolution use case: might want to add an asType to a symbolic reference that might have failed Remi: dynamic runtimes - often have part loaded, part not yet created John: original concept of invoke dynamic - if you were to get a NoSuchMethodError, might some day be able to dynamically create a bridge using asType (might require patching methodHandles into vtables) 2. Dan H - no intention that the argindex maps to e.g. a javap classfile index? John: NO - not with condy. No promise that this is an constant pool index for instance Dan H: intend to replace sun.reflect.ConstantPool? John: hope so 3. Remi - propose that the BsCI with a constant call info could replace the ConstantGroup requirement John: will send an email of anticipated use cases expect need for condy lazy resolution, constant bits, constantGroup (compressed set of CP entries to get around CP size limits) III. Class file Version handling John described a new proposal in progress - should be out for community review soon - modeled after TCP port numbers - e.g. ranges for private use - propose private CFV #s for experimental use - that are never standardized - simplest: upper half of the 16 bit range - allows us to work together - project would register for a major number - within a project could use capability bits - e.g. valhalla could get a number and nestmates, MVT could use different capability bits so you could combine them DanH: what about an extra attribute? Remi: capabilities would be sufficient John: attribute reading is too late in the class file processing Dan H: how does this relate to JCP experimental? Would still need an experimental feature if in JVMS appendix AI: John: check Brian?s earlier email IV. J1 plans/ new openjdk projects J1 attendees: Brian, John, Dan H - openJ9 talk and panel openjdk projects: Metropolis - java on java initiative - start with Graal + AOT - if interested in being on initial committers list - let John know (Remi: yes please, also suggested Jan Rogers at Google who worked on Jikes VM) Dan H: JVMCI - get VM metadata - seems tied to Graal/hotspot - please work with us to have it less tied to our implementation Project Loom - Fibers/continuations - Ron Pressler who did Quasar, works for Oracle now Dan H - similar to MLVM coroutines? John: goal - skip native stack copying allocate Continuations in heap and not copy onto the stack (most of it) e.g. Parrot VM keeps most stack frames in heap Dan H: Smalltalk heapifies stack Remi: MVT experimenting with parts of value type buffer outside of stack - should consider common groundwork John: will start with mechanism used by the deoptimization framework and expand also - stack walking API - goal is to reflect the entire stack including continuations Project Loom will need a spec expert group V. Nest mates - JVMS feedback 1. DanH - will the tests be in openjdk? Yes - they will be in jtreg soon in the nestmates repository - consolidation upgrade in progress Karen walked through the document she had sent about Preparation and Selection. Please send feedback on this document, or even more importantly any Nestmates JVMS changes you believe are needed. editor?s note: The only JVMS changes identified so far: 1) lazy loading of nest top - e.g. remove from verifier 2) NestHost - naming change from MemberOfNest ? from Karen?s review of the sections related to selection, preparation, overriding - they all look good, no changes needed. In fact they are improved - thanks Dan S! thanks, Karen From john.r.rose at oracle.com Wed Oct 4 20:49:04 2017 From: john.r.rose at oracle.com (John Rose) Date: Wed, 4 Oct 2017 13:49:04 -0700 Subject: [BSM-RETURNS-MH] allow invokedynamic BSM to return MethodHandle instead of CallSite References: <160B0BE2-30DD-4964-ABCA-A07E74112B1D@oracle.com> Message-ID: <650AF025-A44B-4403-AE85-9D5D1DD9B9DC@oracle.com> http://mail.openjdk.java.net/pipermail/mlvm-dev/2017-October/006782.html Begin forwarded message: As we use BSMs more and more, we will want to dispense with the CallSite infrastructure, in many cases. I filed an RFE to track this. If we can agree on specification, it will be fairly easy to implement. (I am CC-ing this to valhalla-spec-experts, for visibility to non-OpenJDK people.) ? John https://bugs.openjdk.java.net/browse/JDK-8188790 allow invokedynamic BSM to return MethodHandle instead of CallSite Invokedynamic is defined to return a CallSite in order to allow a CallSite object to support per-call-site mutability, which enables certain advanced optimizations based on argument type speculation (aka. inline caching). Also, for the same reason, invokedynamic instructions are individually linked, unlike other instructions which refer to the constant pool. This allows each invokedynamic to operate on a separate "history" of type speculations, when the call site is mutable. However, most uses of invokedynamic do not use these advanced degrees of freedom; most uses return a ConstantCallSite, and return the same call site (or an equivalent one) for each distinct occurrence of an invokedynamic. This simpler use case can be referred to as "same CCS every time". By contrast, the fully general case may be referred to as "possibly a different MCS each time". We can reduce link overheads and footprint, as well as classfile complexity, if we support this simpler use case direct, instead of as an under-the-covers optimization of the full use case. We can compatibly extend the JVM Specification, specifically as documented in the "package-info.java" javadoc for java.lang.invoke, by allowing a new behavior from a bootstrap method (BSM) which links an invokedynamic instruction. This new behavior is simply to return a reference to a MethodHandle instead of a CallSite. This is currently an illegal response, so it would be a compatible extension. Specifically, if a MethodHandle is returned from a BSM call in response to linking an invokedynamic instruction, then the linkage of that dynamic call site would directly use the method handle. In terms of the current specification, it would be as if the BSM had returned a ConstantCallSite wrapping the given method handle. (Today's JVM can easily perform this optimization under the covers, also, discarding the CCS and just using the MH that it wraps.) Also, if a BSM for a given CONSTANT_InvokeDynamic constant ever returns a MethodHandle (instead of a CallSite), then, as a new behavior, the JVM records the method handle permanently as the result of linking that constant pool constant. Future linkage events that refer to the same constant pool entry (of type CONSTANT_InvokeDynamic) must then return the identical method handle, with no further calls to the BSM associated with that constant. (Corner case: In the case of race conditions, the present specification says that the JVM is obligated to pick one race winner and publish it as the overriding answer to all racing BSM invocations. With respect to the proposed change, a race condition may propose a mix of MHs and CSs as potential winners. The JVM must choose a winner, as before, and the new semantics of permanent linkage apply if and only if the winner is an MH. In such a case any competing CS results would be discarded; if a CS wins any competing MH results would be ignored. Better luck next time. If a given BSM always returns an MH, then races might occur but only one winner will apply for all call sites; this will be the common case.) A side benefit of this extension would be bringing "indy" and "condy" (dynamic constants) closer together, in their linkage behavior. It would be easier to share BSMs between the two instructions (invokedynamic and ldc). Perhaps it would make it easier to converge the two facilities further in the future. _______________________________________________ mlvm-dev mailing list mlvm-dev at openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev From daniel.smith at oracle.com Fri Oct 6 18:00:06 2017 From: daniel.smith at oracle.com (Dan Smith) Date: Fri, 6 Oct 2017 12:00:06 -0600 Subject: Updated nestmates spec Message-ID: <53E88DAF-BF4F-4D54-A746-DD6E82F19759@oracle.com> I've posted an update to the nestmates spec based on discussions of the last few months. http://cr.openjdk.java.net/~dlsmith/nestmates.html Significant changes: - Renamed MemberOfNest --> NestHost, per some requests; it seems more consistent with names of other attributes - Eliminated checking of the NestHost attribute during verification. This means the nest of a class is computed lazily when access is requested, and we have to account for live classes whose NestHost attribute is invalid I considered and rejected a proposal to guard 'invokeinterface' behavior changes on the class file version number. Consensus was that behavioral changes that replace errors with successful execution are acceptable when running old class files. Feedback is always welcome. ?Dan From brian.goetz at oracle.com Wed Oct 11 18:13:30 2017 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 11 Oct 2017 14:13:30 -0400 Subject: Updated nestmates spec In-Reply-To: <53E88DAF-BF4F-4D54-A746-DD6E82F19759@oracle.com> References: <53E88DAF-BF4F-4D54-A746-DD6E82F19759@oracle.com> Message-ID: <294a24bc-1c84-e915-6b23-ff7797239e35@oracle.com> One thing I didn't see in the spec: the access check on classes.? For example, given: ??? class C { ??????? private class D { ??????????? private int x; ??????? } ??? } D.x is accessible to C because (class check) D and C are in the same package, and (member check) D and C are in the same nest. We erase the "private" modifier when we write out the classfile for D.? But, with nestmates, we shouldn't have to do that; we should be able to mark D as ACC_PRIVATE, and amend 5.4.4 to have a fourth bullet for "class or interface C is accessible to a class or interface D" section, which says "members of the same nest." This should affect all access to D members, and also affect "C implements C.D", where C.D is private, scenarios. On 10/6/2017 2:00 PM, Dan Smith wrote: > I've posted an update to the nestmates spec based on discussions of the last few months. > > http://cr.openjdk.java.net/~dlsmith/nestmates.html > > Significant changes: > - Renamed MemberOfNest --> NestHost, per some requests; it seems more consistent with names of other attributes > - Eliminated checking of the NestHost attribute during verification. This means the nest of a class is computed lazily when access is requested, and we have to account for live classes whose NestHost attribute is invalid > > I considered and rejected a proposal to guard 'invokeinterface' behavior changes on the class file version number. Consensus was that behavioral changes that replace errors with successful execution are acceptable when running old class files. > > Feedback is always welcome. > > ?Dan From daniel.smith at oracle.com Wed Oct 11 21:57:44 2017 From: daniel.smith at oracle.com (Dan Smith) Date: Wed, 11 Oct 2017 15:57:44 -0600 Subject: Updated nestmates spec In-Reply-To: <294a24bc-1c84-e915-6b23-ff7797239e35@oracle.com> References: <53E88DAF-BF4F-4D54-A746-DD6E82F19759@oracle.com> <294a24bc-1c84-e915-6b23-ff7797239e35@oracle.com> Message-ID: <21EB8190-7B4A-496A-BF03-698989059F86@oracle.com> Yes, while not part of the proposed feature set, it would make sense to have ACC_PRIVATE classes in the JVM, with a requirement that a private class can only be accessed from a nestmate (or itself). That change would be a self-contained feature that could be squeezed in wherever it's convenient. ACC_PACKAGE still doesn't make sense for classes in the JVM, because there is no concept of type membership or inheritance. Package-access classes will still have to be compiled as ACC_PUBLIC. ?Dan > On Oct 11, 2017, at 12:13 PM, Brian Goetz wrote: > > One thing I didn't see in the spec: the access check on classes. For example, given: > > class C { > private class D { > private int x; > } > } > > D.x is accessible to C because (class check) D and C are in the same package, and (member check) D and C are in the same nest. > > We erase the "private" modifier when we write out the classfile for D. But, with nestmates, we shouldn't have to do that; we should be able to mark D as ACC_PRIVATE, and amend 5.4.4 to have a fourth bullet for "class or interface C is accessible to a class or interface D" section, which says "members of the same nest." > > This should affect all access to D members, and also affect "C implements C.D", where C.D is private, scenarios. > > On 10/6/2017 2:00 PM, Dan Smith wrote: >> I've posted an update to the nestmates spec based on discussions of the last few months. >> >> http://cr.openjdk.java.net/~dlsmith/nestmates.html >> >> Significant changes: >> - Renamed MemberOfNest --> NestHost, per some requests; it seems more consistent with names of other attributes >> - Eliminated checking of the NestHost attribute during verification. This means the nest of a class is computed lazily when access is requested, and we have to account for live classes whose NestHost attribute is invalid >> >> I considered and rejected a proposal to guard 'invokeinterface' behavior changes on the class file version number. Consensus was that behavioral changes that replace errors with successful execution are acceptable when running old class files. >> >> Feedback is always welcome. >> >> ?Dan > From karen.kinnear at oracle.com Fri Oct 13 10:26:02 2017 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Fri, 13 Oct 2017 12:26:02 +0200 Subject: Minutes Valhalla EG Oct 11 2017 Message-ID: <457FAED4-4501-431C-8742-ADB12C92CBEE@oracle.com> AI: David Holmes - Proposal for reflection APIs for nestmates ( thanks David for volunteering) AI: Dan Smith - updated Condy JVMS AI: Karen Kinnear - for nestmates - transitive overriding example email Attendees: Bjorn, David Simms, Dan H, Frederic, Lois, John, Karen I. Nestmates JVMS comments: (Thanks to David Holmes) 1. IAE due to issues with nestmates such as - NCDFE or not listed in nest host member list or other resolution error We all agreed we want to report more information than just an IAE possible approaches: - add a cause - additional information in the message - possibly subclass of IAE for nestmates? (ed. note - while access checking is likely to be the place where lazy nest host resolution occurs, it might also happen due to a reflection request) Responses welcome. 2. InvokeInterface change to support local private method invocation - proposal is to make that independent of class file version. Change in behavior: old ICCE will now succeed. No issues with mix-n-match class file versions. All will support local and old versions won?t have nestmates. Dan H pointed out that we have reserved the right to go from an error to a success case. Summary: EG was good with change. This will be a strict subset of the new behavior for old classifies. 3. Reflection Need a specific API proposal for when we get nest host resolution errors or there is a mismatch in nest membership. For each API - do we return e.g. null or a failure or a subset of members? If I ask for all my nest mates - do we need to eagerly resolve them? No to returning string names of nest mates John Rose: what if we were able to return the upcoming ClassRef, i.e. an unresolved reference which could then later be resolved which could get an appropriate resolution error at that time? ? if this is going to ship in the same timeframe as support for Constable - maybe this is a better approach going forward. 4. Preparation vs. Selection Dan H said that Karen?s proposal made sense to him (yay! and many thanks) and clarified some of the intended loader constraint handling. So we are ok with the preparation changes to the JVMS if Dan S agrees with Karen?s description as well - i.e. we think they match :-) Karen owes brief summary of transitive overriding including some examples - to double-check the overriding modifications. No other nest mate JVMS issues were raised. === II. Release timing: Intention is that 18.3 contain ConstantDynamic with class file version update. No other class file changes are anticipated. Nestmates are targeted for early in the 18.9 cycle so it gets bake time. Other language changes in 18.3 - LVTI other? === III. Condy JVMS Dan Smith is working through some improvements - in internal review. Should be available soon. 1. 5.4.3 Clarification of pre-condy expected behavior: Indy - if resolution fails with a LinkageException for s given BCI, we need to record the exception and rethrow. Successes for a given BCI also need to be cached and return the same result. (Oracle is currently fixing a bug with that). If a VM error pass through unchanged, else wrap in BSME. 2. InvokeWithArguments Handling for megaarguments - e.g. removing the limits and turning the tail into varargs. This matches what we do i source code. goal: scale BSM (8 bit limitation today) treat BSM as if method descriptor from constant pool - all Object except for boxing note: no method descriptor in constant pool call matches > 256 args on stack - and we do not want to change the JVMS for that. So - looking to find a short simple way to allow this in the specification without having to precisely restrict implementation. In future we plan to clean this up with the BsCI mechanism. 3. Renaming Constant_Dynamic Constant_DynamicCallSite (yes - renaming indy as well) 4. Implementation clarification For non-ConstantCallSite - if setTarget is called - what is the expected behavior? If SetTarget is called - the JVM MUST notice this and - replace any caching, deopt/recompile etc. - it is not valid to setTarget to null - it must be a non-null MethodHandle and the proper type The Callsite returned must be the same for the same BCI for indy, but the target itself is changeable. 5. ldc_w Just ensuring we all are expecting that ldc_w condy works for double and wide 6. Consider using a MethdHandle for ldc rather than a field descriptor? John: intention condy is to indy as get static is to invoke static indy uses a name&type for a method type and condy uses a name&type for a field type building on the existing JVM language split between methods and fields in future - may use MethodTypeRef and ClassRef goal is to benefits from combinators, and languages that want very untyped BSM args. Concern is a bootstrap attribute index which is shared between indy and condy. It has to be an indy BSM - super type of both is java.lang.Object today. thanks, Karen From david.holmes at oracle.com Mon Oct 16 03:36:07 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 16 Oct 2017 13:36:07 +1000 Subject: Minutes Valhalla EG Oct 11 2017 In-Reply-To: <457FAED4-4501-431C-8742-ADB12C92CBEE@oracle.com> References: <457FAED4-4501-431C-8742-ADB12C92CBEE@oracle.com> Message-ID: <2ba184f9-6b3b-ecd4-7666-e852eb848dfa@oracle.com> On 13/10/2017 8:26 PM, Karen Kinnear wrote: > AI: David Holmes - Proposal for reflection APIs for nestmates ( thanks David for volunteering) > AI: Dan Smith - updated Condy JVMS > AI: Karen Kinnear - for nestmates - transitive overriding example email > > > Attendees: Bjorn, David Simms, Dan H, Frederic, Lois, John, Karen > > I. Nestmates JVMS comments: (Thanks to David Holmes) > > 1. IAE due to issues with nestmates > such as - NCDFE or not listed in nest host member list or other resolution error > > We all agreed we want to report more information than just an IAE > possible approaches: > - add a cause > - additional information in the message > - possibly subclass of IAE for nestmates? > > (ed. note - while access checking is likely to be the place where lazy nest host resolution occurs, > it might also happen due to a reflection request) > > Responses welcome. This is tracked as: https://bugs.openjdk.java.net/browse/JDK-8187768 I propose to set any exception that occurs when trying to perform the access check, and the access checks fails, as the cause of the IAE that is thrown. It is worth noting that pending exceptions are not checked in this code at present so it is possible to proceed through different access checks and eventually determine access is permissible, even if one or more checks triggered an exception. > 2. InvokeInterface change to support local private method invocation - proposal is to make that > independent of class file version. > Change in behavior: old ICCE will now succeed. > No issues with mix-n-match class file versions. All will support local and old versions won?t have > nestmates. > Dan H pointed out that we have reserved the right to go from an error to a success case. > > Summary: EG was good with change. This will be a strict subset of the new behavior for old classifies. Tracked as: https://bugs.openjdk.java.net/browse/JDK-8189163 > 3. Reflection > Need a specific API proposal for when we get nest host resolution errors or there is a mismatch > in nest membership. For each API - do we return e.g. null or a failure or a subset of members? > If I ask for all my nest mates - do we need to eagerly resolve them? > > No to returning string names of nest mates > John Rose: what if we were able to return the upcoming ClassRef, i.e. an unresolved reference which > could then later be resolved which could get an appropriate resolution error at that time? > > ? if this is going to ship in the same timeframe as support for Constable - maybe this is a better > approach going forward. This is tracked as: https://bugs.openjdk.java.net/browse/JDK-8188075 The ClassRef idea is intriguing - but it would need to be ready to ship before mid-December if the nestmates work is to be pushed on schedule. I find it hard to see how to define a Class based API now that would be able to evolve to a ClassRef based API in the future. > 4. Preparation vs. Selection > Dan H said that Karen?s proposal made sense to him (yay! and many thanks) and clarified some > of the intended loader constraint handling. So we are ok with the preparation changes to the JVMS > if Dan S agrees with Karen?s description as well - i.e. we think they match :-) > > Karen owes brief summary of transitive overriding including some examples - to double-check the > overriding modifications. I propose this is also tracked by: https://bugs.openjdk.java.net/browse/JDK-8189163 We may need additional tests here. Thanks, David ----- > > No other nest mate JVMS issues were raised. > > === > II. Release timing: > > Intention is that 18.3 contain ConstantDynamic with class file version update. No other class file changes > are anticipated. > > Nestmates are targeted for early in the 18.9 cycle so it gets bake time. > > Other language changes in 18.3 - LVTI > other? > > === > III. Condy JVMS > > Dan Smith is working through some improvements - in internal review. Should be available soon. > > 1. 5.4.3 Clarification of pre-condy expected behavior: > > Indy - if resolution fails with a LinkageException for s given BCI, we need to record the exception and rethrow. > Successes for a given BCI also need to be cached and return the same result. > (Oracle is currently fixing a bug with that). > > If a VM error pass through unchanged, else wrap in BSME. > > 2. InvokeWithArguments > Handling for megaarguments - e.g. removing the limits and turning the tail into varargs. > This matches what we do i source code. > goal: scale BSM (8 bit limitation today) > treat BSM as if method descriptor from constant pool > - all Object except for boxing > > note: no method descriptor in constant pool call matches > 256 args on stack - and we do not want to change the JVMS > for that. > > So - looking to find a short simple way to allow this in the specification without having to precisely restrict > implementation. In future we plan to clean this up with the BsCI mechanism. > > 3. Renaming > Constant_Dynamic > Constant_DynamicCallSite (yes - renaming indy as well) > > 4. Implementation clarification > For non-ConstantCallSite - if setTarget is called - what is the expected behavior? > If SetTarget is called - the JVM MUST notice this and > - replace any caching, deopt/recompile etc. > - it is not valid to setTarget to null - it must be a non-null MethodHandle and the proper type > > The Callsite returned must be the same for the same BCI for indy, but the target itself is changeable. > > 5. ldc_w > Just ensuring we all are expecting that ldc_w condy works for double and wide > > 6. Consider using a MethdHandle for ldc rather than a field descriptor? > > John: intention > condy is to indy as get static is to invoke static > > indy uses a name&type for a method type and condy uses a name&type for a field type > building on the existing JVM language split between methods and fields > in future - may use MethodTypeRef and ClassRef > > goal is to benefits from combinators, and languages that want very untyped BSM args. > > Concern is a bootstrap attribute index which is shared between indy and condy. It has to be an indy BSM - > super type of both is java.lang.Object today. > > thanks, > Karen > > > > > From daniel.smith at oracle.com Tue Oct 17 21:40:06 2017 From: daniel.smith at oracle.com (Dan Smith) Date: Tue, 17 Oct 2017 15:40:06 -0600 Subject: Updated nestmates spec In-Reply-To: <53E88DAF-BF4F-4D54-A746-DD6E82F19759@oracle.com> References: <53E88DAF-BF4F-4D54-A746-DD6E82F19759@oracle.com> Message-ID: <592C5DEA-9A3F-48BD-BF36-E1182B49BFDD@oracle.com> I've made another update. http://cr.openjdk.java.net/~dlsmith/nestmates.html It seemed inappropriate to be ignoring errors arising during resolution of the NestHost class (other than indirectly prompting IllegalAccessErrors). And, in the worst case, some errors like OutOfMemoryError unquestionably need to be passed on to users, not swallowed by access checking. So I've rewritten 5.4.4 to stop trying to define "is accessible" and instead describe an "access checking" process that performs some error checks. This is more verbose but also more precise about exactly when resolution and its associated errors are allowed to happen. It's possible that APIs and tools outside of the JVM spec will want a definition of the total functions "is accessible" or "nest host". This change shifts responsibility of defining those functions (and behavior in the presence of errors) to the outside APIs/tools. If that's problematic, feedback on this point is welcome. ?Dan > On Oct 6, 2017, at 12:00 PM, Dan Smith wrote: > > I've posted an update to the nestmates spec based on discussions of the last few months. > > http://cr.openjdk.java.net/~dlsmith/nestmates.html > > Significant changes: > - Renamed MemberOfNest --> NestHost, per some requests; it seems more consistent with names of other attributes > - Eliminated checking of the NestHost attribute during verification. This means the nest of a class is computed lazily when access is requested, and we have to account for live classes whose NestHost attribute is invalid > > I considered and rejected a proposal to guard 'invokeinterface' behavior changes on the class file version number. Consensus was that behavioral changes that replace errors with successful execution are acceptable when running old class files. > > Feedback is always welcome. > > ?Dan From karen.kinnear at oracle.com Wed Oct 25 14:26:57 2017 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Wed, 25 Oct 2017 10:26:57 -0400 Subject: nestmates and transitive overriding Message-ID: To sanity check the JVMS changes relative to ?transitive? overriding in the JVMS update for JEP 181: nestmates: http://cr.openjdk.java.net/~dlsmith/nestmates.html The first goal here is to ensure that we are all in agreement on the intended behavior of invoke virtual relative to transitive overriding. Once we are sure about that - then we can double-check that the new wording of 5.4.4. Overriding works for all of us. While I agree that the transitive overriding only applies in cases where there is a package private method in the hierarchy, it is not the case that it only applies if the resolved method is package private. I went back to the pre-default method changes in JDK7 when the transitive overriding was added to find the reasoning, examples, and in particular to find the test cases which demonstrated a change in behavior in invoke virtual due to the JVMS change from JVMS 2nd edition to JVMS 3rd edition based on the JVMS Clarifications and Addenda to 2nd edition. I have appended first my notes at the time, then the explicit list of tests that had behavioral changes due to the JVMS change for us. In practice, we create a virtual method table, or vtable, at preparation time, as an invoke virtual selection cache. The conceptual model is that each potential resolved class creates a new index in the vtable which will be overridden for each potential receiver to contain the selected method based on the search and overriding rules. To save space we actually don?t always create a new index - if A.m is public and B extends A has public m as well, they can share an index. We are extremely careful that any package private method does create a new index because even if it overrides a superclass?s public method, a subclass may not override it. So the theory is that we are testing overriding on the root of the index. In practice we are testing overriding of the latest overrider. I think that is where we have a translation from the JVMS and need to make sure we are all saying the same thing. The set of tests that changed behavior when we added the transitive overriding all look like: class A public or protected m() class B extends A package private m() class C extends B public or protected or package private m() call: invoke virtual A.m object ref C The old result was B.m or AME (if B.m is abstract), the new result was C.m (unless IAE due to caller access of A.m protected) I am appending my notes from the change at the time. Below that is the test matrix reporting the tests that changed and the results are listed as : 1.6 results/1.7 expected results. We are internally in the process of cleaning up the tests and making them open jtreg tests. They currently use a very old ASM. However you can find them in JDK-8163974. Give a shout if you need help figuring out how to run them. If you run them on our jvm - you might need to run -Xint -Xverify:none right now. thanks, Karen Here are my notes from the change at the time: Changes in Invoke-Virtual behavior due to specification changes With classfile version 51, which will be introduced in JDK7, the invokevirtual behavior will be fixed to match the clarifications in the JVMS which were made to handle the following test case: Sample test case from: Why the Definition of Method Override was Revised package P1; class A { void m(){System.out.println("A.m");} public void callM(){ m();} } public class B extends A { protected void m() { System.out.println("B.m");} } package P2; class C extends P1.B { protected void m(){ System.out.println("C.m");} public static void main(String[] args) { C c = new C(); c.callM(); } Given the original definition of override, we find that the method m() declared in C overrides the method m() declared in B(), since it has the same name and signature and is accessible from class C. However, we a lso find that the method m() declared in C does not override the method m() declared in A(), since it is pr ivate to package P1 and therefore not accessible from class C. In this case, invoking callM() on an instance of C causes a call of m() where the target of the method invocation is an instance of C as well. However, this call is in the class P1.A, and is attempting to invoke a package private method of A. Since this method is not overridden by C, the code will not print "C.m" as expected. Instead, it prints "B.m". This is quite unintuitive, and also incompatible with the behavior of the classic VM. We believe that this interpretation, while literally accurate, is undesirable. It prevents a package private method from ever being overridden outside its package, even if a subclass within the package has increased the method's accessibility. * Invokevirtual modifications: * Invokevirtual, JVMS Clarifications and Addenda to 2nd edition Let C be the class of objectref. The actual method to be invoked is selected by the following lookup procedure: If C contains a declaration for an instance method M with the same name and descriptor as the resolved method, and M overrides the resolved method, then M is the method to be invoked, and the lookup procedure terminates. Otherwise, if C has a superclass, this same lookup procedure is performed recursively using the direct superclass of C ; the method to be invoked is the result of the recursive invocation of this lookup procedure. Otherwise, an AbstractMethodError? is raised. JVMS 2nd edition invokevirtual: Let C be the class of objectref. The actual method to be invoked is selected by the following lookup procedure: If C contains a declaration for an instance method with the same name and descriptor as the resolved method, and the resolved method is accessible from C, then this is the method to be invoked, and the lookup procedure terminates. Otherwise, if C has a superclass, this same lookup procedure is performed recursively using the direct superclass of C ; the method to be invoked is the result of the recursive invocation of this lookup procedure. Otherwise, an AbstractMethodError? is raised. Note: the key difference here is the phrase: "the resolved method is accessible from C" which in the Clarifications and Addenda has been replaced with "M overrides the resolved method? Inheritance rules: JLS 3rd edition, 8.4.8 Inheritance, Overriding, and Hiding A class C inherits from its direct superclass and direct superinterfaces all non-private methods (whether abstract or not) of the superclass and superinterfaces that are public, protected or declared with default access in the same package as C and are neither overridden ('8.4.8.1) nor hidden ('8.4.8.2) by a declaration in the class. JLS 3rd edition 8.4.8.1 Overriding (by Instance Methods) An instance method m1 declared in a class C overrides another instance method, m2, declared in class A iff all of the following are true: 1. C is a subclass of A. 2. The signature of m1 is a subsignature (?8.4.2) of the signature of m2. 3. Either o m2 is public, protected or declared with default access in the same package as C, or o m1 overrides a method m3, m3 distinct from m1, m3 distinct from m2, such that m3 overrides m2. Moreover, if m1 is not abstract, then m1 is said to implement any and all declarations of abstract methods that it overrides. Note that based on the JVMS 3rd edition transitive overriding rules, we need to do an override check first for the direct superclass and if the current class does not override the direct superclass, recursively for the superclass' superclass. Thanks to Alex Buckley and Vladimir V. Ivanov for help understanding this. ======== Invokevirtual behavior differences between JDK 1.6 and upcoming 1.7. Changes in behavior due to clarification of overriding behavior as transitive, clarified Invokevirtual JVMS 3rd edition and JLS 3rd edition: 8.4.8.1 Overriding (by Instance Methods) This test represents a run of a 1.6 JDK relative to 1.7 expected results. The 1.6 results are listed/expected results are listed. In all of these examples, B.m was the old expected result (sometimes resulting in AME), but with the clarification of overriding behavior as transitive, C.m is now the expected result. Method access modifiers Call site location Status # A.m() B.m() C.m() A pkgA B pkgB C pkgC ------------------------------------------------------------------------------------------------- 841| A PUB a.B PP b.C PUB | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m | FAILED 842| ! A PUB a.B PP b.C PUB | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m | FAILED 843| A PUB ! a.B PP b.C PUB | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m | FAILED 844| ! A PUB ! a.B PP b.C PUB | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m | FAILED 845| A PUB a.B PP b.C PROT | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m | FAILED 846| ! A PUB a.B PP b.C PROT | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m | FAILED 847| A PUB ! a.B PP b.C PROT | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m | FAILED 848| ! A PUB ! a.B PP b.C PROT | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m | FAILED 849| A PUB a.B PP b.C PP | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m | FAILED 850| ! A PUB a.B PP b.C PP | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m | FAILED 851| A PUB ! a.B PP b.C PP | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m | FAILED 852| ! A PUB ! a.B PP b.C PP | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m | FAILED 941| A PROT a.B PP b.C PUB | B.m/C.m B.m/C.m B.m/C.m IAE B.m/C.m IAE | FAILED 942| ! A PROT a.B PP b.C PUB | B.m/C.m B.m/C.m B.m/C.m IAE B.m/C.m IAE | FAILED 943| A PROT ! a.B PP b.C PUB | AME/C.m AME/C.m AME/C.m IAE AME/C.m IAE | FAILED 944| ! A PROT ! a.B PP b.C PUB | AME/C.m AME/C.m AME/C.m IAE AME/C.m IAE | FAILED 945| A PROT a.B PP b.C PROT | B.m/C.m B.m/C.m B.m/C.m IAE B.m/C.m IAE | FAILED 946| ! A PROT a.B PP b.C PROT | B.m/C.m B.m/C.m B.m/C.m IAE B.m/C.m IAE | FAILED 947| A PROT ! a.B PP b.C PROT | AME/C.m AME/C.m AME/C.m IAE AME/C.m IAE | FAILED 948| ! A PROT ! a.B PP b.C PROT | AME/C.m AME/C.m AME/C.m IAE AME/C.m IAE | FAILED 949| A PROT a.B PP b.C PP | B.m/C.m B.m/C.m B.m/C.m IAE B.m/C.m IAE | FAILED 950| ! A PROT a.B PP b.C PP | B.m/C.m B.m/C.m B.m/C.m IAE B.m/C.m IAE | FAILED 951| A PROT ! a.B PP b.C PP | AME/C.m AME/C.m AME/C.m IAE AME/C.m IAE | FAILED 952| ! A PROT ! a.B PP b.C PP | AME/C.m AME/C.m AME/C.m IAE AME/C.m IAE | FAILED 1641| a.A PUB a.B PP b.C PUB | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m | FAILED 1642| ! a.A PUB a.B PP b.C PUB | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m | FAILED 1643| a.A PUB ! a.B PP b.C PUB | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m | FAILED 1644| ! a.A PUB ! a.B PP b.C PUB | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m | FAILED 1645| a.A PUB a.B PP b.C PROT | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m | FAILED 1646| ! a.A PUB a.B PP b.C PROT | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m | FAILED 1647| a.A PUB ! a.B PP b.C PROT | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m | FAILED 1648| ! a.A PUB ! a.B PP b.C PROT | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m | FAILED 1649| a.A PUB a.B PP b.C PP | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m | FAILED 1650| ! a.A PUB a.B PP b.C PP | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m | FAILED 1651| a.A PUB ! a.B PP b.C PP | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m | FAILED 1652| ! a.A PUB ! a.B PP b.C PP | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m | FAILED 1741| a.A PROT a.B PP b.C PUB | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m IAE | FAILED 1742| ! a.A PROT a.B PP b.C PUB | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m IAE | FAILED 1743| a.A PROT ! a.B PP b.C PUB | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m IAE | FAILED 1744| ! a.A PROT ! a.B PP b.C PUB | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m IAE | FAILED 1745| a.A PROT a.B PP b.C PROT | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m IAE | FAILED 1746| ! a.A PROT a.B PP b.C PROT | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m IAE | FAILED 1747| a.A PROT ! a.B PP b.C PROT | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m IAE | FAILED 1748| ! a.A PROT ! a.B PP b.C PROT | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m IAE | FAILED 1749| a.A PROT a.B PP b.C PP | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m IAE | FAILED 1750| ! a.A PROT a.B PP b.C PP | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m IAE | FAILED 1751| a.A PROT ! a.B PP b.C PP | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m IAE | FAILED 1752| ! a.A PROT ! a.B PP b.C PP | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m IAE | FAILED 2041| a.A PUB b.B PP a.C PUB | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m | FAILED 2042| ! a.A PUB b.B PP a.C PUB | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m | FAILED 2043| a.A PUB ! b.B PP a.C PUB | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m | FAILED 2044| ! a.A PUB ! b.B PP a.C PUB | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m | FAILED 2045| a.A PUB b.B PP a.C PROT | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m | FAILED 2046| ! a.A PUB b.B PP a.C PROT | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m | FAILED 2047| a.A PUB ! b.B PP a.C PROT | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m | FAILED 2048| ! a.A PUB ! b.B PP a.C PROT | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m | FAILED 2049| a.A PUB b.B PP a.C PP | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m | FAILED 2050| ! a.A PUB b.B PP a.C PP | B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m B.m/C.m | FAILED 2051| a.A PUB ! b.B PP a.C PP | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m | FAILED 2052| ! a.A PUB ! b.B PP a.C PP | AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m AME/C.m | FAILED 2141| a.A PROT b.B PP a.C PUB | B.m/C.m B.m/C.m B.m/C.m IAE B.m/C.m B.m/C.m | FAILED 2142| ! a.A PROT b.B PP a.C PUB | B.m/C.m B.m/C.m B.m/C.m IAE B.m/C.m B.m/C.m | FAILED 2143| a.A PROT ! b.B PP a.C PUB | AME/C.m AME/C.m AME/C.m IAE AME/C.m AME/C.m | FAILED 2144| ! a.A PROT ! b.B PP a.C PUB | AME/C.m AME/C.m AME/C.m IAE AME/C.m AME/C.m | FAILED 2145| a.A PROT b.B PP a.C PROT | B.m/C.m B.m/C.m B.m/C.m IAE B.m/C.m B.m/C.m | FAILED 2146| ! a.A PROT b.B PP a.C PROT | B.m/C.m B.m/C.m B.m/C.m IAE B.m/C.m B.m/C.m | FAILED 2147| a.A PROT ! b.B PP a.C PROT | AME/C.m AME/C.m AME/C.m IAE AME/C.m AME/C.m | FAILED 2148| ! a.A PROT ! b.B PP a.C PROT | AME/C.m AME/C.m AME/C.m IAE AME/C.m AME/C.m | FAILED 2149| a.A PROT b.B PP a.C PP | B.m/C.m B.m/C.m B.m/C.m IAE B.m/C.m B.m/C.m | FAILED 2150| ! a.A PROT b.B PP a.C PP | B.m/C.m B.m/C.m B.m/C.m IAE B.m/C.m B.m/C.m | FAILED 2151| a.A PROT ! b.B PP a.C PP | AME/C.m AME/C.m AME/C.m IAE AME/C.m AME/C.m | FAILED 2152| ! a.A PROT ! b.B PP a.C PP | AME/C.m AME/C.m AME/C.m IAE AME/C.m AME/C.m | FAILED From karen.kinnear at oracle.com Wed Oct 25 14:45:51 2017 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Wed, 25 Oct 2017 10:45:51 -0400 Subject: nestmates spec open issues Message-ID: <1402428F-A0D1-46B3-B640-8D5817606C78@oracle.com> David Holmes (many thanks) has filed bugs to track most of the open issues: 1. access checking failing due to lazy host resolution: https://bugs.openjdk.java.net/browse/JDK-8187768 Dan has updated the specification. In our prototyping we are still investigating cases in which resolution errors are not expected during access checking (I know IllegalAccessError is a LinkageError). 2. Core reflection APIs https://bugs.openjdk.java.net/browse/JDK-8188075 John Rose proposed: a) Class.getnestHost() - defaults to itself if there is a resolution error b) Class.getNestMembers() - returns full nest, fallback of self if any resolution errors including it lists a nestHost that does not list it. [editor notes: - full statically defined nest from classfile attribute? As distinct from full dynamically currently loaded nest - right? - looks like John is proposing resolving the names, not just returning strings, and skipping any resolution failures 3. transitive overriding - see other email. thanks, Karen From forax at univ-mlv.fr Wed Oct 25 15:33:32 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 25 Oct 2017 17:33:32 +0200 (CEST) Subject: nestmates spec open issues In-Reply-To: <1402428F-A0D1-46B3-B640-8D5817606C78@oracle.com> References: <1402428F-A0D1-46B3-B640-8D5817606C78@oracle.com> Message-ID: <672740258.1078464.1508945612650.JavaMail.zimbra@u-pem.fr> > De: "Karen Kinnear" > ?: "valhalla-spec-experts" > Envoy?: Mercredi 25 Octobre 2017 16:45:51 > Objet: nestmates spec open issues > ... > 2. Core reflection APIs [ https://bugs.openjdk.java.net/browse/JDK-8188075 | > https://bugs.openjdk.java.net/browse/JDK-8188075 ] > John Rose proposed: > a) Class.getnestHost() - defaults to itself if there is a resolution error > b) Class.getNestMembers() - returns full nest, fallback of self if any > resolution errors including it lists a nestHost that > does not list it. > [editor notes: > - full statically defined nest from classfile attribute? As distinct from full > dynamically > currently loaded nest - right? > - looks like John is proposing resolving the names, not just returning strings, > and skipping any resolution failures I do not like swallowing exceptions, it always make things harder to debug, so counter proposition: Class Class.getNestHost() if there is no attribute, return this, throw aTypeNotPresentException if either the NestHost can not be loaded or the NestHost doesn't list this as its member. Class[] Class.getNestMembers() scrub all members that are either not found or or not valid (not the same NestHost) from the array and always add this to the array. ... > thanks, > Karen cheers, R?mi From brian.goetz at oracle.com Wed Oct 25 15:39:31 2017 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 25 Oct 2017 11:39:31 -0400 Subject: nestmates spec open issues In-Reply-To: <1402428F-A0D1-46B3-B640-8D5817606C78@oracle.com> References: <1402428F-A0D1-46B3-B640-8D5817606C78@oracle.com> Message-ID: <555bc857-0b31-c9c6-ca28-dc407a520cfe@oracle.com> John Rose proposed: > a) Class.getnestHost() - defaults to itself if there is a resolution error (or if there is no Nest attribute) > b) Class.getNestMembers() - ?returns full nest, fallback of self if > any resolution errors including it lists a nestHost that > ? ? does not list it. For consistency with the classfile representation, this should probably omit the host class. > [editor notes: > - ?full statically defined nest from classfile attribute? As distinct > from full dynamically > currently loaded nest - right? I would prefer that this return only the static members, which is consistent with the design center for reflection -- reflection over classfiles. From john.r.rose at oracle.com Wed Oct 25 16:51:59 2017 From: john.r.rose at oracle.com (John Rose) Date: Wed, 25 Oct 2017 09:51:59 -0700 Subject: nestmates spec open issues In-Reply-To: <555bc857-0b31-c9c6-ca28-dc407a520cfe@oracle.com> References: <1402428F-A0D1-46B3-B640-8D5817606C78@oracle.com> <555bc857-0b31-c9c6-ca28-dc407a520cfe@oracle.com> Message-ID: On Oct 25, 2017, at 8:39 AM, Brian Goetz wrote: > > John Rose proposed: >> a) Class.getnestHost() - defaults to itself if there is a resolution error > > (or if there is no Nest attribute) (yes) >> b) Class.getNestMembers() - returns full nest, fallback of self if any resolution errors including it lists a nestHost that >> does not list it. > > For consistency with the classfile representation, this should probably omit the host class. (yes, that's my preference, although the other way is not too terrible) > >> [editor notes: >> - full statically defined nest from classfile attribute? As distinct from full dynamically >> currently loaded nest - right? > > I would prefer that this return only the static members, which is consistent with the design center for reflection -- reflection over classfiles. yes yes yes; the reflection should reflect what's in the class-file (or a resolvable subset), not everything in the VM's knowledge I like Remi's idea of scrubbing the list (of reflected nestmates) of bad actors rather than clearing the list if there are *any* bad actors. Reminder: bad actors are an edge case, not a normal case. Question: what do we do in the exactly parallel case for getInnerClasses? Do we scrub bad actors? Nullify the result? Throw? (Probably throw.) ? John From david.holmes at oracle.com Wed Oct 25 21:52:31 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 26 Oct 2017 07:52:31 +1000 Subject: nestmates spec open issues In-Reply-To: References: <1402428F-A0D1-46B3-B640-8D5817606C78@oracle.com> <555bc857-0b31-c9c6-ca28-dc407a520cfe@oracle.com> Message-ID: <21ad17b6-60f1-962b-13ea-2cc2113c05e6@oracle.com> On 26/10/2017 2:51 AM, John Rose wrote: > On Oct 25, 2017, at 8:39 AM, Brian Goetz wrote: >> >> John Rose proposed: >>> a) Class.getnestHost() - defaults to itself if there is a resolution error >> >> (or if there is no Nest attribute) > > (yes) > >>> b) Class.getNestMembers() - returns full nest, fallback of self if any resolution errors including it lists a nestHost that >>> does not list it. >> >> For consistency with the classfile representation, this should probably omit the host class. > > (yes, that's my preference, although the other way is not too terrible) > >> >>> [editor notes: >>> - full statically defined nest from classfile attribute? As distinct from full dynamically >>> currently loaded nest - right? >> >> I would prefer that this return only the static members, which is consistent with the design center for reflection -- reflection over classfiles. > > yes yes yes; the reflection should reflect what's in the class-file (or a resolvable subset), not everything in the VM's knowledge > > I like Remi's idea of scrubbing the list (of reflected nestmates) of bad actors rather than clearing the list if there are *any* bad actors. > > Reminder: bad actors are an edge case, not a normal case. > Question: what do we do in the exactly parallel case for getInnerClasses? Do we scrub bad actors? Nullify the result? Throw? (Probably throw.) Throw. David > ? John > From forax at univ-mlv.fr Wed Oct 25 22:33:45 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 26 Oct 2017 00:33:45 +0200 (CEST) Subject: nestmates spec open issues In-Reply-To: <21ad17b6-60f1-962b-13ea-2cc2113c05e6@oracle.com> References: <1402428F-A0D1-46B3-B640-8D5817606C78@oracle.com> <555bc857-0b31-c9c6-ca28-dc407a520cfe@oracle.com> <21ad17b6-60f1-962b-13ea-2cc2113c05e6@oracle.com> Message-ID: <1489940844.1161854.1508970825203.JavaMail.zimbra@u-pem.fr> getClasses() throws an exception but getAnnotations() skips unavailable annotations. that said, i'm not against throwing in this case. R?mi ----- Mail original ----- > De: "David Holmes" > ?: "Valhalla Expert Group Observers" , "John Rose" , > "Brian Goetz" > Cc: "valhalla-spec-experts" > Envoy?: Mercredi 25 Octobre 2017 23:52:31 > Objet: Re: nestmates spec open issues > On 26/10/2017 2:51 AM, John Rose wrote: >> On Oct 25, 2017, at 8:39 AM, Brian Goetz wrote: >>> >>> John Rose proposed: >>>> a) Class.getnestHost() - defaults to itself if there is a resolution error >>> >>> (or if there is no Nest attribute) >> >> (yes) >> >>>> b) Class.getNestMembers() - returns full nest, fallback of self if any >>>> resolution errors including it lists a nestHost that >>>> does not list it. >>> >>> For consistency with the classfile representation, this should probably omit the >>> host class. >> >> (yes, that's my preference, although the other way is not too terrible) >> >>> >>>> [editor notes: >>>> - full statically defined nest from classfile attribute? As distinct from full >>>> dynamically >>>> currently loaded nest - right? >>> >>> I would prefer that this return only the static members, which is consistent >>> with the design center for reflection -- reflection over classfiles. >> >> yes yes yes; the reflection should reflect what's in the class-file (or a >> resolvable subset), not everything in the VM's knowledge >> >> I like Remi's idea of scrubbing the list (of reflected nestmates) of bad actors >> rather than clearing the list if there are *any* bad actors. >> >> Reminder: bad actors are an edge case, not a normal case. >> Question: what do we do in the exactly parallel case for getInnerClasses? Do >> we scrub bad actors? Nullify the result? Throw? (Probably throw.) > > Throw. > > David > >> ? John From john.r.rose at oracle.com Thu Oct 26 03:07:25 2017 From: john.r.rose at oracle.com (John Rose) Date: Wed, 25 Oct 2017 20:07:25 -0700 Subject: nestmates spec open issues In-Reply-To: <1489940844.1161854.1508970825203.JavaMail.zimbra@u-pem.fr> References: <1402428F-A0D1-46B3-B640-8D5817606C78@oracle.com> <555bc857-0b31-c9c6-ca28-dc407a520cfe@oracle.com> <21ad17b6-60f1-962b-13ea-2cc2113c05e6@oracle.com> <1489940844.1161854.1508970825203.JavaMail.zimbra@u-pem.fr> Message-ID: <654111C5-C286-419F-98AB-1CC741B07DE9@oracle.com> On Oct 25, 2017, at 3:33 PM, Remi Forax wrote: > > getClasses() throws an exception but getAnnotations() skips unavailable annotations. > > that said, i'm not against throwing in this case. I'm not against throwing either, but I think scrubbing (like annotations) is a little better, because it is more robust about retaining partial results. Partial results are important if you are just asking about one or two classes and don't care about their other nestmates. From john.r.rose at oracle.com Thu Oct 26 03:11:40 2017 From: john.r.rose at oracle.com (John Rose) Date: Wed, 25 Oct 2017 20:11:40 -0700 Subject: nestmates spec open issues In-Reply-To: <654111C5-C286-419F-98AB-1CC741B07DE9@oracle.com> References: <1402428F-A0D1-46B3-B640-8D5817606C78@oracle.com> <555bc857-0b31-c9c6-ca28-dc407a520cfe@oracle.com> <21ad17b6-60f1-962b-13ea-2cc2113c05e6@oracle.com> <1489940844.1161854.1508970825203.JavaMail.zimbra@u-pem.fr> <654111C5-C286-419F-98AB-1CC741B07DE9@oracle.com> Message-ID: On Oct 25, 2017, at 8:07 PM, John Rose wrote: > > On Oct 25, 2017, at 3:33 PM, Remi Forax > wrote: >> >> getClasses() throws an exception but getAnnotations() skips unavailable annotations. >> >> that said, i'm not against throwing in this case. > > I'm not against throwing either, but I think scrubbing (like annotations) > is a little better, because it is more robust about retaining partial results. > Partial results are important if you are just asking about one or two > classes and don't care about their other nestmates. And even if H.getNestMembers() throws because of some missing N1, we have to be careful to allow N2.getNestHost() to return H, if we can validate that N2 points to H and vice versa. We don't want the validated relation between N2 and H to be collateral damage to a failure of N1. (If H goes missing there is nothing we can do with N1 and N2, except to assign each to its own 1-nest. Which is OK with me.) From david.holmes at oracle.com Thu Oct 26 03:54:50 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 26 Oct 2017 13:54:50 +1000 Subject: nestmates spec open issues In-Reply-To: References: <1402428F-A0D1-46B3-B640-8D5817606C78@oracle.com> <555bc857-0b31-c9c6-ca28-dc407a520cfe@oracle.com> <21ad17b6-60f1-962b-13ea-2cc2113c05e6@oracle.com> <1489940844.1161854.1508970825203.JavaMail.zimbra@u-pem.fr> <654111C5-C286-419F-98AB-1CC741B07DE9@oracle.com> Message-ID: <47a7177d-9f92-8141-6e11-522d3de5da8a@oracle.com> On 26/10/2017 1:11 PM, John Rose wrote: > On Oct 25, 2017, at 8:07 PM, John Rose wrote: >> >> On Oct 25, 2017, at 3:33 PM, Remi Forax > wrote: >>> >>> getClasses() throws an exception but getAnnotations() skips unavailable annotations. >>> >>> that said, i'm not against throwing in this case. >> >> I'm not against throwing either, but I think scrubbing (like annotations) >> is a little better, because it is more robust about retaining partial results. Aren't annotations a different situation because annotations are well known to potentially not be available at runtime? A missing annotation does not necessarily reflect an error in the runtime environment of an application. Also AFAIK see from the jdk/java/lang/annotation/Missing/ test the throw/no-throw behaviour depends on exactly how the missing annotation is used. >> Partial results are important if you are just asking about one or two >> classes and don't care about their other nestmates. How do you know to only ask about "one or two classes"? Sorry I'm not understanding the potential usecase here.If you've already identified specific classes of interest then you can call getNestHost on them. > And even if H.getNestMembers() throws because of some missing N1, > we have to be careful to allow N2.getNestHost() to return H, if we can > validate that N2 points to H and vice versa. We don't want the validated > relation between N2 and H to be collateral damage to a failure of N1. Absolutely not. The two queries would be completely unrelated in that regard. > (If H goes missing there is nothing we can do with N1 and N2, except > to assign each to its own 1-nest. Which is OK with me.) Then presumably we should do the same at the VM level instead of allowing resolution related exceptions to propagate as currently proposed? We could just throw IAE as we do if a nest-host validation check fails. Though a case can still be made to allow VME's to pass through. Though I still feel uncomfortable lying about the nest-host. I don't see what usecases would be served by doing that. Other than informational uses, I can't see a reason to actually identify the nest-host which would not be impacted by lying about that identity. The only truly useful query is for nestmate access and that doesn't need to expose the identity of the nest-host and conservatively rejects access if anything goes wrong. Bottom line for my personal preferences: - hasNestMateAccess: never throws, always returns true or false - getNestHost(): throw if the host isn't there or else a membership validation check fails - getNestMembers(): throw if any nest member isn't there or a membership validation check fails Cheers, David From john.r.rose at oracle.com Thu Oct 26 04:03:46 2017 From: john.r.rose at oracle.com (John Rose) Date: Wed, 25 Oct 2017 21:03:46 -0700 Subject: nestmates spec open issues In-Reply-To: <47a7177d-9f92-8141-6e11-522d3de5da8a@oracle.com> References: <1402428F-A0D1-46B3-B640-8D5817606C78@oracle.com> <555bc857-0b31-c9c6-ca28-dc407a520cfe@oracle.com> <21ad17b6-60f1-962b-13ea-2cc2113c05e6@oracle.com> <1489940844.1161854.1508970825203.JavaMail.zimbra@u-pem.fr> <654111C5-C286-419F-98AB-1CC741B07DE9@oracle.com> <47a7177d-9f92-8141-6e11-522d3de5da8a@oracle.com> Message-ID: <7B55CBF7-C16D-4405-B5D3-A9E57AACF5B3@oracle.com> On Oct 25, 2017, at 8:54 PM, David Holmes wrote: > > On 26/10/2017 1:11 PM, John Rose wrote: >> On Oct 25, 2017, at 8:07 PM, John Rose wrote: >>> >>> On Oct 25, 2017, at 3:33 PM, Remi Forax > wrote: >>>> >>>> getClasses() throws an exception but getAnnotations() skips unavailable annotations. >>>> >>>> that said, i'm not against throwing in this case. >>> >>> I'm not against throwing either, but I think scrubbing (like annotations) >>> is a little better, because it is more robust about retaining partial results. > > Aren't annotations a different situation because annotations are well known to potentially not be available at runtime? A missing annotation does not necessarily reflect an error in the runtime environment of an application. Also AFAIK see from the jdk/java/lang/annotation/Missing/ test the throw/no-throw behaviour depends on exactly how the missing annotation is used. I think the most likely cause of a missing nestmate will be JAR minimization. As long as the nest-host isn't deleted, a dropped nestmate is of no concern, in that scenario. This is obviously different from annotations, but is enough like them to make me think favorably on the scrubbing of lists. Drop an annotation, drop a nestmate; both seem to benefit from a fail-over tactic that ignores the dropped thing, when asking about neighboring things. > >>> Partial results are important if you are just asking about one or two >>> classes and don't care about their other nestmates. > > How do you know to only ask about "one or two classes"? Sorry I'm not understanding the potential usecase here.If you've already identified specific classes of interest then you can call getNestHost on them. I'm thinking about code that emulates hasNestMateAccess by scanning the getNestMembers list. Perhaps if we have a correctly functioning hasNestMateAccess, my concern goes away. > >> And even if H.getNestMembers() throws because of some missing N1, >> we have to be careful to allow N2.getNestHost() to return H, if we can >> validate that N2 points to H and vice versa. We don't want the validated >> relation between N2 and H to be collateral damage to a failure of N1. > > Absolutely not. The two queries would be completely unrelated in that regard. Yep, good. > >> (If H goes missing there is nothing we can do with N1 and N2, except >> to assign each to its own 1-nest. Which is OK with me.) > > Then presumably we should do the same at the VM level instead of allowing resolution related exceptions to propagate as currently proposed? We could just throw IAE as we do if a nest-host validation check fails. Though a case can still be made to allow VME's to pass through. Please don't throw IAE that case. I don't want to bikeshed exception types, but I think we have a strong precedent for using ICCE when encountering a questionable classfile configuration (one that shouldn't have come out of javac). Note also the ICCE <: VME, so it just gets passed through. > Though I still feel uncomfortable lying about the nest-host. I don't see what usecases would be served by doing that. Other than informational uses, I can't see a reason to actually identify the nest-host which would not be impacted by lying about that identity. The only truly useful query is for nestmate access and that doesn't need to expose the identity of the nest-host and conservatively rejects access if anything goes wrong. I'm sympathetic to this, but that implies that getNestHost should return null rather than this for a non-nesty class. I think it's more valuable that getNestHost return 'this' for non-nesty classes, and therefore also for broken-nest classes. > Bottom line for my personal preferences: > > - hasNestMateAccess: never throws, always returns true or false Yes! > - getNestHost(): throw if the host isn't there or else a membership validation check fails See above. > - getNestMembers(): throw if any nest member isn't there or a membership validation check fails As I said before, I don't mind this choice. Especially if we have hasNestMateAccess. > Cheers, > David From david.holmes at oracle.com Thu Oct 26 06:11:23 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 26 Oct 2017 16:11:23 +1000 Subject: nestmates spec open issues In-Reply-To: <7B55CBF7-C16D-4405-B5D3-A9E57AACF5B3@oracle.com> References: <1402428F-A0D1-46B3-B640-8D5817606C78@oracle.com> <555bc857-0b31-c9c6-ca28-dc407a520cfe@oracle.com> <21ad17b6-60f1-962b-13ea-2cc2113c05e6@oracle.com> <1489940844.1161854.1508970825203.JavaMail.zimbra@u-pem.fr> <654111C5-C286-419F-98AB-1CC741B07DE9@oracle.com> <47a7177d-9f92-8141-6e11-522d3de5da8a@oracle.com> <7B55CBF7-C16D-4405-B5D3-A9E57AACF5B3@oracle.com> Message-ID: <4e19fa48-3c7e-98c2-0e69-c8dde0322d46@oracle.com> One immediate follow-up ... On 26/10/2017 2:03 PM, John Rose wrote: > On Oct 25, 2017, at 8:54 PM, David Holmes wrote: >> >> On 26/10/2017 1:11 PM, John Rose wrote: >>> On Oct 25, 2017, at 8:07 PM, John Rose wrote: >>>> >>>> On Oct 25, 2017, at 3:33 PM, Remi Forax > wrote: >>>>> >>>>> getClasses() throws an exception but getAnnotations() skips unavailable annotations. >>>>> >>>>> that said, i'm not against throwing in this case. >>>> >>>> I'm not against throwing either, but I think scrubbing (like annotations) >>>> is a little better, because it is more robust about retaining partial results. >> >> Aren't annotations a different situation because annotations are well known to potentially not be available at runtime? A missing annotation does not necessarily reflect an error in the runtime environment of an application. Also AFAIK see from the jdk/java/lang/annotation/Missing/ test the throw/no-throw behaviour depends on exactly how the missing annotation is used. > > I think the most likely cause of a missing nestmate will be JAR minimization. > As long as the nest-host isn't deleted, a dropped nestmate is of no concern, > in that scenario. > > This is obviously different from annotations, but is enough like them to > make me think favorably on the scrubbing of lists. Drop an annotation, > drop a nestmate; both seem to benefit from a fail-over tactic that ignores > the dropped thing, when asking about neighboring things. > >> >>>> Partial results are important if you are just asking about one or two >>>> classes and don't care about their other nestmates. >> >> How do you know to only ask about "one or two classes"? Sorry I'm not understanding the potential usecase here.If you've already identified specific classes of interest then you can call getNestHost on them. > > I'm thinking about code that emulates hasNestMateAccess by > scanning the getNestMembers list. Perhaps if we have a correctly > functioning hasNestMateAccess, my concern goes away. > > >> >>> And even if H.getNestMembers() throws because of some missing N1, >>> we have to be careful to allow N2.getNestHost() to return H, if we can >>> validate that N2 points to H and vice versa. We don't want the validated >>> relation between N2 and H to be collateral damage to a failure of N1. >> >> Absolutely not. The two queries would be completely unrelated in that regard. > > Yep, good. > >> >>> (If H goes missing there is nothing we can do with N1 and N2, except >>> to assign each to its own 1-nest. Which is OK with me.) >> >> Then presumably we should do the same at the VM level instead of allowing resolution related exceptions to propagate as currently proposed? We could just throw IAE as we do if a nest-host validation check fails. Though a case can still be made to allow VME's to pass through. > > Please don't throw IAE that case. I don't want to bikeshed exception types, > but I think we have a strong precedent for using ICCE when encountering > a questionable classfile configuration (one that shouldn't have come out of > javac). Note also the ICCE <: VME, so it just gets passed through. You must have missed Dan's update regarding access checks. He's already proposed to throw IAE - and I've implemented it. :) http://cr.openjdk.java.net/~dlsmith/nestmates.html David ------ >> Though I still feel uncomfortable lying about the nest-host. I don't see what usecases would be served by doing that. Other than informational uses, I can't see a reason to actually identify the nest-host which would not be impacted by lying about that identity. The only truly useful query is for nestmate access and that doesn't need to expose the identity of the nest-host and conservatively rejects access if anything goes wrong. > > I'm sympathetic to this, but that implies that getNestHost should return null > rather than this for a non-nesty class. I think it's more valuable that getNestHost > return 'this' for non-nesty classes, and therefore also for broken-nest classes. > >> Bottom line for my personal preferences: >> >> - hasNestMateAccess: never throws, always returns true or false > Yes! > >> - getNestHost(): throw if the host isn't there or else a membership validation check fails > See above. > >> - getNestMembers(): throw if any nest member isn't there or a membership validation check fails > As I said before, I don't mind this choice. Especially if we have hasNestMateAccess. > >> Cheers, >> David > From john.r.rose at oracle.com Fri Oct 27 02:02:39 2017 From: john.r.rose at oracle.com (John Rose) Date: Thu, 26 Oct 2017 19:02:39 -0700 Subject: nestmates spec open issues In-Reply-To: <4e19fa48-3c7e-98c2-0e69-c8dde0322d46@oracle.com> References: <1402428F-A0D1-46B3-B640-8D5817606C78@oracle.com> <555bc857-0b31-c9c6-ca28-dc407a520cfe@oracle.com> <21ad17b6-60f1-962b-13ea-2cc2113c05e6@oracle.com> <1489940844.1161854.1508970825203.JavaMail.zimbra@u-pem.fr> <654111C5-C286-419F-98AB-1CC741B07DE9@oracle.com> <47a7177d-9f92-8141-6e11-522d3de5da8a@oracle.com> <7B55CBF7-C16D-4405-B5D3-A9E57AACF5B3@oracle.com> <4e19fa48-3c7e-98c2-0e69-c8dde0322d46@oracle.com> Message-ID: <36CFA8BE-C009-4725-81DB-60BD98AE6E21@oracle.com> On Oct 25, 2017, at 11:11 PM, David Holmes wrote: > >> Please don't throw IAE that case. I don't want to bikeshed exception types, >> but I think we have a strong precedent for using ICCE when encountering >> a questionable classfile configuration (one that shouldn't have come out of >> javac). Note also the ICCE <: VME, so it just gets passed through. > > You must have missed Dan's update regarding access checks. He's already proposed to throw IAE - and I've implemented it. :) > > http://cr.openjdk.java.net/~dlsmith/nestmates.html > Actually, IllegalAccessError is not too terrible, although I'd prefer ICCE for reasons stated. I read IAE as "IllegalArgumentException", which explains the warmth of my response. Why did I think you meant IllegalArgumentException? *That* is unexplainable. ? John From john.r.rose at oracle.com Fri Oct 27 02:11:45 2017 From: john.r.rose at oracle.com (John Rose) Date: Thu, 26 Oct 2017 19:11:45 -0700 Subject: Minutes Valhalla EG Oct 11 2017 In-Reply-To: <457FAED4-4501-431C-8742-ADB12C92CBEE@oracle.com> References: <457FAED4-4501-431C-8742-ADB12C92CBEE@oracle.com> Message-ID: <9490F401-162A-437F-B7AD-4D2A790EC8F9@oracle.com> FTR, Dan's draft is posted at: http://cr.openjdk.java.net/~dlsmith/constant-dynamic.html One thing I like very much about Dan's work is the refactoring of information that used to be co-located with invokedynamic-specific sections of the spec, into their own sections, where indy and condy can refer to them in common. I did some more clarification and refactoring here: http://cr.openjdk.java.net/~jrose/jvm/constant-dynamic-jrose.html Format should be self-explanatory. My new text is in yellow, stuff of Dan's I deleted is in crossed-out green. The clarifications in my draft are about errors thrown around edge cases, such as circular dependencies, and about permitted implementation options (the "as if" rules). The new processing step *after* the BSM invocation collects a nice bundle of information about edge cases pertaining to the result of the invocation, IMO. The refactoring is around the treatment of arguments and return values to the BSM. By shifting the BSM call specification to use invokeWithArguments, non-essential arity restrictions are incidentally lifted, which was difficult to do in the previous treatment. Also, an intermediate fictitious descriptor (which implies a fictitious MethodType) is no longer extracted from descriptor fragments collected from the BSM argument constants; this IMO is a helpful simplification. Cheers, ? John On Oct 13, 2017, at 3:26 AM, Karen Kinnear wrote: > > III. Condy JVMS > > Dan Smith is working through some improvements - in internal review. Should be available soon. > > 1. 5.4.3 Clarification of pre-condy expected behavior: > > Indy - if resolution fails with a LinkageException for s given BCI, we need to record the exception and rethrow. > Successes for a given BCI also need to be cached and return the same result. > (Oracle is currently fixing a bug with that). > > If a VM error pass through unchanged, else wrap in BSME. > > 2. InvokeWithArguments > Handling for megaarguments - e.g. removing the limits and turning the tail into varargs. > This matches what we do i source code. > goal: scale BSM (8 bit limitation today) > treat BSM as if method descriptor from constant pool > - all Object except for boxing > > note: no method descriptor in constant pool call matches > 256 args on stack - and we do not want to change the JVMS > for that. > > So - looking to find a short simple way to allow this in the specification without having to precisely restrict > implementation. In future we plan to clean this up with the BsCI mechanism. > > 3. Renaming > Constant_Dynamic > Constant_DynamicCallSite (yes - renaming indy as well) > > 4. Implementation clarification > For non-ConstantCallSite - if setTarget is called - what is the expected behavior? > If SetTarget is called - the JVM MUST notice this and > - replace any caching, deopt/recompile etc. > - it is not valid to setTarget to null - it must be a non-null MethodHandle and the proper type > > The Callsite returned must be the same for the same BCI for indy, but the target itself is changeable. > > 5. ldc_w > Just ensuring we all are expecting that ldc_w condy works for double and wide > > 6. Consider using a MethdHandle for ldc rather than a field descriptor? > > John: intention > condy is to indy as get static is to invoke static > > indy uses a name&type for a method type and condy uses a name&type for a field type > building on the existing JVM language split between methods and fields > in future - may use MethodTypeRef and ClassRef > > goal is to benefits from combinators, and languages that want very untyped BSM args. > > Concern is a bootstrap attribute index which is shared between indy and condy. It has to be an indy BSM - > super type of both is java.lang.Object today. From david.holmes at oracle.com Fri Oct 27 03:35:06 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 27 Oct 2017 13:35:06 +1000 Subject: nestmates spec open issues In-Reply-To: <36CFA8BE-C009-4725-81DB-60BD98AE6E21@oracle.com> References: <1402428F-A0D1-46B3-B640-8D5817606C78@oracle.com> <555bc857-0b31-c9c6-ca28-dc407a520cfe@oracle.com> <21ad17b6-60f1-962b-13ea-2cc2113c05e6@oracle.com> <1489940844.1161854.1508970825203.JavaMail.zimbra@u-pem.fr> <654111C5-C286-419F-98AB-1CC741B07DE9@oracle.com> <47a7177d-9f92-8141-6e11-522d3de5da8a@oracle.com> <7B55CBF7-C16D-4405-B5D3-A9E57AACF5B3@oracle.com> <4e19fa48-3c7e-98c2-0e69-c8dde0322d46@oracle.com> <36CFA8BE-C009-4725-81DB-60BD98AE6E21@oracle.com> Message-ID: <2b2a7124-b74b-4370-4d33-14e8ac6920df@oracle.com> On 27/10/2017 12:02 PM, John Rose wrote: > On Oct 25, 2017, at 11:11 PM, David Holmes > wrote: >> >>> Please don't throw IAE that case. ?I don't want to bikeshed exception >>> types, >>> but I think we have a strong precedent for using ICCE when encountering >>> a questionable classfile configuration (one that shouldn't have come >>> out of >>> javac). ?Note also the ICCE <: VME, so it just gets passed through. >> >> You must have missed Dan's update regarding access checks. He's >> already proposed to throw IAE - and I've implemented it. :) >> >> http://cr.openjdk.java.net/~dlsmith/nestmates.html >> > > Actually, IllegalAccessError is not too terrible, although > I'd prefer ICCE for reasons stated. > > I read IAE as "IllegalArgumentException", which explains > the warmth of my response. ?Why did I think you meant > IllegalArgumentException? ?*That* is unexplainable. Sorry - it never occurred to me that IAE was an overloaded acronym. :) David > > ? John From martijnverburg at gmail.com Fri Oct 27 14:57:37 2017 From: martijnverburg at gmail.com (Martijn Verburg) Date: Fri, 27 Oct 2017 15:57:37 +0100 Subject: Minutes Valhalla EG Oct 11 2017 In-Reply-To: <9490F401-162A-437F-B7AD-4D2A790EC8F9@oracle.com> References: <457FAED4-4501-431C-8742-ADB12C92CBEE@oracle.com> <9490F401-162A-437F-B7AD-4D2A790EC8F9@oracle.com> Message-ID: Hi John, Just for my reading benefit :-), did you mean: "Format should be self-explanatory. My new text is in yellow, stuff of Dan's *is green*, *stuff* I deleted is in crossed-out *red*."? Cheers, Martijn On 27 October 2017 at 03:11, John Rose wrote: > FTR, Dan's draft is posted at: > http://cr.openjdk.java.net/~dlsmith/constant-dynamic.html > > One thing I like very much about Dan's work is the refactoring of > information that used to be co-located with invokedynamic-specific > sections of the spec, into their own sections, where indy and condy > can refer to them in common. > > I did some more clarification and refactoring here: > http://cr.openjdk.java.net/~jrose/jvm/constant-dynamic-jrose.html > > Format should be self-explanatory. My new text is in yellow, stuff of > Dan's I deleted is in crossed-out green. > > The clarifications in my draft are about errors thrown around edge cases, > such as circular dependencies, and about permitted implementation options > (the "as if" rules). The new processing step *after* the BSM invocation > collects a nice bundle of information about edge cases pertaining > to the result of the invocation, IMO. > > The refactoring is around the treatment of arguments and return values to > the > BSM. By shifting the BSM call specification to use invokeWithArguments, > non-essential arity restrictions are incidentally lifted, which was > difficult to > do in the previous treatment. Also, an intermediate fictitious descriptor > (which implies a fictitious MethodType) is no longer extracted from > descriptor > fragments collected from the BSM argument constants; this IMO is a helpful > simplification. > > Cheers, > ? John > > On Oct 13, 2017, at 3:26 AM, Karen Kinnear > wrote: > > > > III. Condy JVMS > > > > Dan Smith is working through some improvements - in internal review. > Should be available soon. > > > > 1. 5.4.3 Clarification of pre-condy expected behavior: > > > > Indy - if resolution fails with a LinkageException for s given BCI, we > need to record the exception and rethrow. > > Successes for a given BCI also need to be cached and return the same > result. > > (Oracle is currently fixing a bug with that). > > > > If a VM error pass through unchanged, else wrap in BSME. > > > > 2. InvokeWithArguments > > Handling for megaarguments - e.g. removing the limits and turning the > tail into varargs. > > This matches what we do i source code. > > goal: scale BSM (8 bit limitation today) > > treat BSM as if method descriptor from constant pool > > - all Object except for boxing > > > > note: no method descriptor in constant pool call matches > 256 args on > stack - and we do not want to change the JVMS > > for that. > > > > So - looking to find a short simple way to allow this in the > specification without having to precisely restrict > > implementation. In future we plan to clean this up with the BsCI > mechanism. > > > > 3. Renaming > > Constant_Dynamic > > Constant_DynamicCallSite (yes - renaming indy as well) > > > > 4. Implementation clarification > > For non-ConstantCallSite - if setTarget is called - what is the expected > behavior? > > If SetTarget is called - the JVM MUST notice this and > > - replace any caching, deopt/recompile etc. > > - it is not valid to setTarget to null - it must be a non-null > MethodHandle and the proper type > > > > The Callsite returned must be the same for the same BCI for indy, but > the target itself is changeable. > > > > 5. ldc_w > > Just ensuring we all are expecting that ldc_w condy works for double and > wide > > > > 6. Consider using a MethdHandle for ldc rather than a field descriptor? > > > > John: intention > > condy is to indy as get static is to invoke static > > > > indy uses a name&type for a method type and condy uses a name&type for > a field type > > building on the existing JVM language split between methods and fields > > in future - may use MethodTypeRef and ClassRef > > > > goal is to benefits from combinators, and languages that want very > untyped BSM args. > > > > Concern is a bootstrap attribute index which is shared between indy and > condy. It has to be an indy BSM - > > super type of both is java.lang.Object today. > > From john.r.rose at oracle.com Fri Oct 27 18:14:40 2017 From: john.r.rose at oracle.com (John Rose) Date: Fri, 27 Oct 2017 11:14:40 -0700 Subject: Minutes Valhalla EG Oct 11 2017 In-Reply-To: References: <457FAED4-4501-431C-8742-ADB12C92CBEE@oracle.com> <9490F401-162A-437F-B7AD-4D2A790EC8F9@oracle.com> Message-ID: <06931CB3-E19A-40B1-91DB-20A14B3B1533@oracle.com> On Oct 27, 2017, at 7:57 AM, Martijn Verburg wrote: > > Just for my reading benefit :-), did you mean: > > "Format should be self-explanatory. My new text is in yellow, stuff of > Dan's *is green*, *stuff* I deleted is in crossed-out *red*."? Heh. Here's the secret decoder key: white - unchanged green - first draft (Dan), added to original red s/o - first draft, removed from original yellow - second draft (John), added to first draft green s/o - second draft, removed from first draft (second draft didn't remove anything from the original) To derive the proposed final, ignore all strike-outs, and change yellow/green to white. From martijnverburg at gmail.com Mon Oct 30 10:15:16 2017 From: martijnverburg at gmail.com (Martijn Verburg) Date: Mon, 30 Oct 2017 10:15:16 +0000 Subject: Minutes Valhalla EG Oct 11 2017 In-Reply-To: <06931CB3-E19A-40B1-91DB-20A14B3B1533@oracle.com> References: <457FAED4-4501-431C-8742-ADB12C92CBEE@oracle.com> <9490F401-162A-437F-B7AD-4D2A790EC8F9@oracle.com> <06931CB3-E19A-40B1-91DB-20A14B3B1533@oracle.com> Message-ID: Perfect, thank you - I'll dig in now (I doubt I'll have much to offer, but it's super interesting!). Cheers, Martijn On 27 October 2017 at 19:14, John Rose wrote: > On Oct 27, 2017, at 7:57 AM, Martijn Verburg > wrote: > > > > Just for my reading benefit :-), did you mean: > > > > "Format should be self-explanatory. My new text is in yellow, stuff of > > Dan's *is green*, *stuff* I deleted is in crossed-out *red*."? > > Heh. Here's the secret decoder key: > > white - unchanged > green - first draft (Dan), added to original > red s/o - first draft, removed from original > yellow - second draft (John), added to first draft > green s/o - second draft, removed from first draft > (second draft didn't remove anything from the original) > > To derive the proposed final, ignore all strike-outs, > and change yellow/green to white. > > From david.holmes at oracle.com Tue Oct 31 01:45:39 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 31 Oct 2017 11:45:39 +1000 Subject: nestmates spec open issues In-Reply-To: <7B55CBF7-C16D-4405-B5D3-A9E57AACF5B3@oracle.com> References: <1402428F-A0D1-46B3-B640-8D5817606C78@oracle.com> <555bc857-0b31-c9c6-ca28-dc407a520cfe@oracle.com> <21ad17b6-60f1-962b-13ea-2cc2113c05e6@oracle.com> <1489940844.1161854.1508970825203.JavaMail.zimbra@u-pem.fr> <654111C5-C286-419F-98AB-1CC741B07DE9@oracle.com> <47a7177d-9f92-8141-6e11-522d3de5da8a@oracle.com> <7B55CBF7-C16D-4405-B5D3-A9E57AACF5B3@oracle.com> Message-ID: <0425b428-a5aa-f6b9-bd14-066249e0bc21@oracle.com> Hi John, On 26/10/2017 2:03 PM, John Rose wrote: > On Oct 25, 2017, at 8:54 PM, David Holmes wrote: >> Though I still feel uncomfortable lying about the nest-host. I don't see what usecases would be served by doing that. Other than informational uses, I can't see a reason to actually identify the nest-host which would not be impacted by lying about that identity. The only truly useful query is for nestmate access and that doesn't need to expose the identity of the nest-host and conservatively rejects access if anything goes wrong. > > I'm sympathetic to this, but that implies that getNestHost should return null > rather than this for a non-nesty class. I think it's more valuable that getNestHost > return 'this' for non-nesty classes, and therefore also for broken-nest classes. Ok. The JVMS is proposed to state: "A class or interface without a NestHost attribute belongs to the nest hosted by itself." That covers top-level classes and all types compiled with an earlier version of javac. If we consider a class with an unresolvable, or otherwise invalid, NestHost attribute, as-if it had no NestHost attribute, then returning "this" is the natural response for that case. >> Bottom line for my personal preferences: >> >> - hasNestMateAccess: never throws, always returns true or false > Yes! Okay. >> - getNestHost(): throw if the host isn't there or else a membership validation check fails > See above. getNestHost() won't throw, as above. Suggestion: also add getValidatedNestHost() that can throw? >> - getNestMembers(): throw if any nest member isn't there or a membership validation check fails > As I said before, I don't mind this choice. Especially if we have hasNestMateAccess. Okay. getNestMembers() will throw. Now the question is what to throw? Dan's spec change for the access checking allows resolution/linkage errors (and VME) to pass through, and specifies IllegalAccessError for an invalid nest membership claim. The IAE makes sense in the context of an access check - though could also be ICCE. But in the context of the reflection API, getNestMembers or getValidatedNestHost, IllegalAccessError doesn't really make sense - we're not trying to access anything. ICCE seems more fitting there. Thanks, David >> Cheers, >> David > From john.r.rose at oracle.com Tue Oct 31 02:44:59 2017 From: john.r.rose at oracle.com (John Rose) Date: Mon, 30 Oct 2017 19:44:59 -0700 Subject: nestmates spec open issues In-Reply-To: <0425b428-a5aa-f6b9-bd14-066249e0bc21@oracle.com> References: <1402428F-A0D1-46B3-B640-8D5817606C78@oracle.com> <555bc857-0b31-c9c6-ca28-dc407a520cfe@oracle.com> <21ad17b6-60f1-962b-13ea-2cc2113c05e6@oracle.com> <1489940844.1161854.1508970825203.JavaMail.zimbra@u-pem.fr> <654111C5-C286-419F-98AB-1CC741B07DE9@oracle.com> <47a7177d-9f92-8141-6e11-522d3de5da8a@oracle.com> <7B55CBF7-C16D-4405-B5D3-A9E57AACF5B3@oracle.com> <0425b428-a5aa-f6b9-bd14-066249e0bc21@oracle.com> Message-ID: <7114A027-58EA-4059-826B-30AB81C042B9@oracle.com> On Oct 30, 2017, at 6:45 PM, David Holmes wrote: > > Hi John, > > On 26/10/2017 2:03 PM, John Rose wrote: >> On Oct 25, 2017, at 8:54 PM, David Holmes wrote: >>> Though I still feel uncomfortable lying about the nest-host. I don't see what usecases would be served by doing that. Other than informational uses, I can't see a reason to actually identify the nest-host which would not be impacted by lying about that identity. The only truly useful query is for nestmate access and that doesn't need to expose the identity of the nest-host and conservatively rejects access if anything goes wrong. >> I'm sympathetic to this, but that implies that getNestHost should return null >> rather than this for a non-nesty class. I think it's more valuable that getNestHost >> return 'this' for non-nesty classes, and therefore also for broken-nest classes. > > Ok. The JVMS is proposed to state: > > "A class or interface without a NestHost attribute belongs to the nest hosted by itself." > > That covers top-level classes and all types compiled with an earlier version of javac. If we consider a class with an unresolvable, or otherwise invalid, NestHost attribute, as-if it had no NestHost attribute, then returning "this" is the natural response for that case. Excellent. > >>> Bottom line for my personal preferences: >>> >>> - hasNestMateAccess: never throws, always returns true or false >> Yes! > > Okay. > >>> - getNestHost(): throw if the host isn't there or else a membership validation check fails >> See above. > > getNestHost() won't throw, as above. > > Suggestion: also add getValidatedNestHost() that can throw? It's a lot of API for a corner case that doesn't matter much. Counter-suggestion: void validateNesting() which has no effect if the class validates properly w.r.t. nesting requirements, and throws ICCE otherwise. If you want to validate the whole nest you iterate over getNestMembers. OTOH, if we make getNestMs validate all that stuff up front then we don't need validateNesting. So, how about make getNestMs enforce complete bi-di referential integrity and throw ICCE if that fails. That give a "hook" for forcing validation. > >>> - getNestMembers(): throw if any nest member isn't there or a membership validation check fails >> As I said before, I don't mind this choice. Especially if we have hasNestMateAccess. > > Okay. getNestMembers() will throw. Good; see above also. > > Now the question is what to throw? Dan's spec change for the access checking allows resolution/linkage errors (and VME) to pass through, and specifies IllegalAccessError for an invalid nest membership claim. The IAE makes sense in the context of an access check - though could also be ICCE. > > But in the context of the reflection API, getNestMembers or getValidatedNestHost, IllegalAccessError doesn't really make sense - we're not trying to access anything. ICCE seems more fitting there. ICCE stands for "somebody broke the classfiles, either intentionally or not". That's why I think it is a good result from invalid classfile data, including the present case, which is invalid nestmate relations. You can only "see" the full story if someone is asking for the whole nest, but that's exactly what getNestMs does. ? John From forax at univ-mlv.fr Tue Oct 31 08:47:50 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 31 Oct 2017 09:47:50 +0100 (CET) Subject: Fix Parameter Runtime*ParameterAnnotations spec Message-ID: <803691474.399683.1509439670003.JavaMail.zimbra@u-pem.fr> [I've trouble to find what is the right list for the JVMS spec change describe below, so i apologize in advance for the spam] Hi all, the spec of the Runtime*ParameterAnnotations attribute [1], allow the number of parameter annotations to be different from the number of parameter from the method descriptor but it fails to provide a way to retrieve/compute the mapping between a parameter and a parameter annotation. So people try to guess and fail, here is by example the ASM bug when we tried to provide such mapping to our user [2]. We (the ASM team) believe the only way to fix that is to require that if the number of parameters from the descriptor and the number of parameter annotations doesn't match then compilers should also emit a Parameter attribute which already indicate if a parameter is synthetic or not. The Parameter attribute is not emitted by default by compilers because it will make the class files too fat, here we are proposing to only emit the Parameter attribute in the case where there are annotations on parameters and the method has synthetic parameters, so it should not be a problem. regards, R?mi [1] https://docs.oracle.com/javase/specs/jvms/se9/html/jvms-4.html#jvms-4.7.18 [2] https://gitlab.ow2.org/asm/asm/issues/317788 From alex.buckley at oracle.com Tue Oct 31 19:11:26 2017 From: alex.buckley at oracle.com (Alex Buckley) Date: Tue, 31 Oct 2017 12:11:26 -0700 Subject: Fix Parameter Runtime*ParameterAnnotations spec In-Reply-To: <803691474.399683.1509439670003.JavaMail.zimbra@u-pem.fr> References: <803691474.399683.1509439670003.JavaMail.zimbra@u-pem.fr> Message-ID: <59F8CADE.9040106@oracle.com> On 10/31/2017 1:47 AM, Remi Forax wrote: > Hi all, the spec of the Runtime*ParameterAnnotations attribute [1], > allow the number of parameter annotations to be different from the > number of parameter from the method descriptor but it fails to > provide a way to retrieve/compute the mapping between a parameter and > a parameter annotation. > > So people try to guess and fail, here is by example the ASM bug when > we tried to provide such mapping to our user [2]. > > We (the ASM team) believe the only way to fix that is to require that > if the number of parameters from the descriptor and the number of > parameter annotations doesn't match then compilers should also emit a > Parameter attribute which already indicate if a parameter is > synthetic or not. I recognize that constructor parameters are a painful cause of discrepancy between the method descriptor and various attributes -- not just Runtime*ParameterAnnotations but also Signature (see the note "A method signature encoded by ..." in https://docs.oracle.com/javase/specs/jvms/se9/html/jvms-4.html#jvms-4.7.9.1). However, the decision in JDK-8067975 was to loosen the JVMS' description so that it admitted the class files emitted by javac and ecj. If you want javac and ecj to emit something different than they do today, then your best bet is to write up a "State of the Parameters" page that shows the "interesting" programs, and what javac and ecj emit. Only then can suggestions like "Emit a MethodParameters attribute if ..." be evaluated. Alex