From adam.sotona at oracle.com Wed Mar 1 09:24:43 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Wed, 1 Mar 2023 09:24:43 +0000 Subject: Classfile API jdktypes package rename and Signatures move proposal Message-ID: Hi, Based on PR feedback I would like to propose two possible API changes: 1. Rename jdk.internal.classfile.jdktypes package to jdk.internal.classfile.constant to reflect the fact that PackageDesc and ModuleDesc are complementing java.lang.constant package content. 2. Move Signature, MethodSignature and ClassSignature from jdk.internal.classfile package to the same package as #1, as the signature models are similar to descriptor models and serve the same purpose. Please let me know if you agree or disagree individually with #1 and #2. Thank you, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From heidinga at redhat.com Wed Mar 1 14:08:54 2023 From: heidinga at redhat.com (Dan Heidinga) Date: Wed, 1 Mar 2023 09:08:54 -0500 Subject: Classfile API jdktypes package rename and Signatures move proposal In-Reply-To: References: Message-ID: As a minor consumer of the API, both changes seem reasonable to me though I have yet to need to use either set of classes. Given "jdktypes" isn't a particularly descriptive package name, drawing the parallel for PackageDesc/ModuleDesc to java.lang.constant makes sense. --Dan On Wed, Mar 1, 2023 at 4:25 AM Adam Sotona wrote: > Hi, > > Based on PR feedback I would like to propose two possible API changes: > > 1. Rename jdk.internal.classfile.jdktypes package to > jdk.internal.classfile.constant to reflect the fact that PackageDesc > and ModuleDesc are complementing java.lang.constant package content. > 2. Move Signature, MethodSignature and ClassSignature from > jdk.internal.classfile package to the same package as #1, as the > signature models are similar to descriptor models and serve the same > purpose. > > > > Please let me know if you agree or disagree individually with #1 and #2. > > > > Thank you, > > Adam > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Wed Mar 1 15:56:25 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 1 Mar 2023 10:56:25 -0500 Subject: Classfile API jdktypes package rename and Signatures move proposal In-Reply-To: References: Message-ID: <1cd884e0-bb41-473f-ea84-62a26382e27e@oracle.com> Regarding #1, I actually would have thought we'd have integrated the "jdk types" into the JDK as RFEs already and they'd be removed by the time we integrated the bytecode API, so hadn't given a lot of thought to package naming.? If this won't be the case, then we should probably let the package name reflect the final resting place, so something like jdk.i.c.java.lang.constant? Assuming that the package will go after upstreaming {Module,Package}Desc, I think what you're suggesting is that we will eventually put the Signature stuff somewhere outside the bytecode library as well?? I think this is a bigger question, which is: for types related to quantities in classfiles that are not tied to the bytecode library, where should they live?? These include things like descriptor parsing and validation (which are duplicated in a dozen places in the JDK), signature parsing, conversion between internal and external names, etc. On 3/1/2023 4:24 AM, Adam Sotona wrote: > > Hi, > > Based on PR feedback I would like to propose two possible API changes: > > 1. Rename jdk.internal.classfile.jdktypespackage to > jdk.internal.classfile.constantto reflect the fact that > PackageDescand ModuleDescare complementing > java.lang.constantpackage content. > 2. Move Signature, MethodSignatureand ClassSignaturefrom > jdk.internal.classfilepackage to the same package as #1, as the > signature models are similar to descriptor models and serve the > same purpose. > > Please let me know if you agree or disagree individually with #1 and #2. > > Thank you, > > Adam > -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Wed Mar 1 21:02:54 2023 From: liangchenblue at gmail.com (-) Date: Wed, 1 Mar 2023 15:02:54 -0600 Subject: Signature model issues Message-ID: Hi, While reviewing the Signature move proposal, I found a few issue with this model: 1. Signature.TypeArg specifically represents wildcards, which is one of the type arguments (the other being reference types represented by Signature.RefTypeSig). Thus, the current model allows passing in a wildcard as a bound of another wildcard, which is illegal in both the Java language and JVM. I recommend renaming Signature.TypeArg to Signature.WildcardSig and make it directly extend Signature instead (or add another common superinterface for wildcard and ref type signatures). 2. Another issue with the model is ClassTypeSig does not validate its input, so it's currently possible to pass a base type or void signature as a type argument, which is illegal as well, and it won't cause an error. If we have the common superinterface recommended above, we can control such malformed inputs. From adam.sotona at oracle.com Fri Mar 3 09:07:40 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Fri, 3 Mar 2023 09:07:40 +0000 Subject: Classfile API jdktypes package rename and Signatures move proposal In-Reply-To: <1cd884e0-bb41-473f-ea84-62a26382e27e@oracle.com> References: <1cd884e0-bb41-473f-ea84-62a26382e27e@oracle.com> Message-ID: From: Brian Goetz Date: Wednesday, 1 March 2023 16:56 To: Adam Sotona , classfile-api-dev at openjdk.org Subject: Re: Classfile API jdktypes package rename and Signatures move proposal Regarding #1, I actually would have thought we'd have integrated the "jdk types" into the JDK as RFEs already and they'd be removed by the time we integrated the bytecode API, so hadn't given a lot of thought to package naming. If this won't be the case, then we should probably let the package name reflect the final resting place, so something like jdk.i.c.java.lang.constant? Yes, we should integrate ?jdk types? before opening Classfile API as preview, because any further package change will affect much larger audience. However JDK-internal Classfile API implementation use and follow-up integrations work should not be blocked by waiting on these. Isn?t jdk.internal.classfile.java.lang.constant a little bit too long? Assuming that the package will go after upstreaming {Module,Package}Desc, I think what you're suggesting is that we will eventually put the Signature stuff somewhere outside the bytecode library as well? I think this is a bigger question, which is: for types related to quantities in classfiles that are not tied to the bytecode library, where should they live? These include things like descriptor parsing and validation (which are duplicated in a dozen places in the JDK), signature parsing, conversion between internal and external names, etc. Yes, one of the optional plans include putting signatures outside the Classfile library, or maybe to merge it and extend functionality of ClassDesc and MethodTypeDesc. However we are not there yet, so I take it as ?don?t move signatures yet?. Thanks, Adam On 3/1/2023 4:24 AM, Adam Sotona wrote: Hi, Based on PR feedback I would like to propose two possible API changes: 1. Rename jdk.internal.classfile.jdktypes package to jdk.internal.classfile.constant to reflect the fact that PackageDesc and ModuleDesc are complementing java.lang.constant package content. 2. Move Signature, MethodSignature and ClassSignature from jdk.internal.classfile package to the same package as #1, as the signature models are similar to descriptor models and serve the same purpose. Please let me know if you agree or disagree individually with #1 and #2. Thank you, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Fri Mar 3 09:37:56 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Fri, 3 Mar 2023 09:37:56 +0000 Subject: Signature model issues In-Reply-To: References: Message-ID: From: classfile-api-dev on behalf of liangchenblue at gmail.com Date: Wednesday, 1 March 2023 22:03 To: classfile-api-dev at openjdk.org Subject: Signature model issues Hi, While reviewing the Signature move proposal, I found a few issue with this model: 1. Signature.TypeArg specifically represents wildcards, which is one of the type arguments (the other being reference types represented by Signature.RefTypeSig). Thus, the current model allows passing in a wildcard as a bound of another wildcard, which is illegal in both the Java language and JVM. I see, the wildcards model has been probably simplified and squashed too much. I recommend renaming Signature.TypeArg to Signature.WildcardSig and make it directly extend Signature instead (or add another common superinterface for wildcard and ref type signatures). Thanks for the proposal, I?ll double-check with all the use cases. My first impression when looking at Signature.TypeArg now is that it probably should not extend Signature.RefTypeSig nor Signature. It does not represent a signature according to the spec. 2. Another issue with the model is ClassTypeSig does not validate its input, so it's currently possible to pass a base type or void signature as a type argument, which is illegal as well, and it won't cause an error. If we have the common superinterface recommended above, we can control such malformed inputs. Right, validation is not there yet. Thanks for pointing it out. -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Tue Mar 7 10:09:48 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Tue, 7 Mar 2023 10:09:48 +0000 Subject: Classfile API CodeBuilder::transforming clarification Message-ID: Hi, During the Classfile API reviews there have been raised concerns about `CodeBuilder::transforming` method, for details see: https://github.com/openjdk/jdk/pull/10982/files/074dd30401a68638a24c157595caeb96b3511614#r1123858513 I would like to (re-)open this discussion here to find the best suitable form of the following method: /** * Apply a transform to the code built by a handler, directing results to this builder. * * @param transform the transform to apply to the code built by the handler * @param handler the handler that receives a {@linkplain CodeBuilder} to * build the code. * @return this builder */ default CodeBuilder transforming(CodeTransform transform, Consumer handler) My proposal is to align it more with `CodeBuilder::block` method and emphasize more the bytecode block than the transformation itself. I propose to change the method name and arguments to: /** * Add a transformed lexical block to the method being built. *

* Within this block, the {@link #startLabel()} and {@link #endLabel()} correspond * to the start and end of the block, and the {@link BlockCodeBuilder#breakLabel()} * also corresponds to the end of the block. * * @param handler handler that receives a {@linkplain BlockCodeBuilder} to * generate the body of the lexical block. * @param transform the transform to apply to the lexical block generated by handler * @return this builder */ default CodeBuilder transformedBlock(Consumer handler, CodeTransform transform) Or alternatively: default CodeBuilder transformBlock(Consumer handler, CodeTransform transform) or: default CodeBuilder transformingBlock(Consumer handler, CodeTransform transform) or just simple: default CodeBuilder block(Consumer handler, CodeTransform transform) Please let me know which version do you prefer or propose alternatives. Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Tue Mar 7 14:40:41 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Tue, 7 Mar 2023 14:40:41 +0000 Subject: Classfile API tracking RFEs Message-ID: Hi, Some of the proposed RFEs fall beyond the initial Classfile API pull request and we should probably start filling them to JBS or find another way to track them. Here are listed citations (for temporary tracking purpose) what I?ve collected from the feedback: 1. switch seems the one instruction for which an high-level variant (like trying) could be useful, as generating code for that can be quite painful. 2. I noticed that this pattern of start/endScope is here, but also in ExceptionCatch, LocalVariable and LocalVariableType. Consider using a common interface for "instructions that are associated with a range". 3. Should we add a helper method on CodeBuilder that does the new + dup + invoke special dance? 4. Another issue with the model is ClassTypeSig does not validate its input. Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Tue Mar 7 14:42:57 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 7 Mar 2023 09:42:57 -0500 Subject: Classfile API CodeBuilder::transforming clarification In-Reply-To: References: Message-ID: <18587a63-49d2-31bd-c956-6400d208bba4@oracle.com> I'd like some time to think about this; this was one of the trickiest corners of the API to design.? Its quite possible there's an inconsistency or vestige, but also possible something subtle is going on. Can we put this on the RFE list as well? On 3/7/2023 5:09 AM, Adam Sotona wrote: > > Hi, > > During the Classfile API reviews there have been raised concerns about > `CodeBuilder::transforming` method, for details see: > > https://github.com/openjdk/jdk/pull/10982/files/074dd30401a68638a24c157595caeb96b3511614#r1123858513 > > I would like to (re-)open this discussion here to find the best > suitable form of the following method: > > /** > ???? * *Apply**a**transform**to**the**code**built**by**a**handler, > directing**results**to**this**builder.* > ???? * > ???? * *@param*transform thetransformtoapplytothecodebuiltbythehandler > ???? * *@param*handler thehandlerthatreceivesa > {*@linkplain*CodeBuilder} to > ???? * buildthecode. > ???? * *@return*thisbuilder > */ > defaultCodeBuilder *transforming*(CodeTransform transform, > Consumer handler) > > My proposal is to align it more with `CodeBuilder::block` method and > emphasize more the bytecode block than the transformation itself. > > I propose to change the method name and arguments to: > > /** > > ???? * > *Add**a**transformed**lexical**block**to**the**method**being**built.* > > ???? * *

* > > ???? * Withinthisblock, the {*@link*#startLabel()} and > {*@link*#endLabel()} correspond > > ???? * tothestartandendoftheblock, andthe > {*@link*BlockCodeBuilder#breakLabel()} > > ???? * alsocorrespondstotheendoftheblock. > > ???? * > > ???? * *@param*handler handlerthatreceivesa > {*@linkplain*BlockCodeBuilder} to > > ???? * generatethebodyofthelexicalblock. > > ???? * *@param*transform > thetransformtoapplytothelexicalblockgeneratedbyhandler > > ???? * *@return*thisbuilder > > */ > > defaultCodeBuilder *transformedBlock*(/Consumer/ > handler, /CodeTransform/ transform) > > Or alternatively: > > defaultCodeBuilder *transformBlock*(/Consumer/ > handler, /CodeTransform/ transform) > > or: > > defaultCodeBuilder > *transform**ing**Block*(/Consumer/ handler, > /CodeTransform/ transform) > > or just simple: > > defaultCodeBuilder *b**lock*(/Consumer/ handler, > /CodeTransform/ transform) > > Please let me know which version do you prefer or propose alternatives. > > Thanks, > > Adam > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Tue Mar 7 15:36:49 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Tue, 7 Mar 2023 15:36:49 +0000 Subject: Classfile API CodeBuilder::transforming clarification In-Reply-To: <18587a63-49d2-31bd-c956-6400d208bba4@oracle.com> References: <18587a63-49d2-31bd-c956-6400d208bba4@oracle.com> Message-ID: Sure no, problem :) From: Brian Goetz Date: Tuesday, 7 March 2023 15:42 To: Adam Sotona , classfile-api-dev at openjdk.org , Paul Sandoz Subject: Re: Classfile API CodeBuilder::transforming clarification I'd like some time to think about this; this was one of the trickiest corners of the API to design. Its quite possible there's an inconsistency or vestige, but also possible something subtle is going on. Can we put this on the RFE list as well? On 3/7/2023 5:09 AM, Adam Sotona wrote: Hi, During the Classfile API reviews there have been raised concerns about `CodeBuilder::transforming` method, for details see: https://github.com/openjdk/jdk/pull/10982/files/074dd30401a68638a24c157595caeb96b3511614#r1123858513 I would like to (re-)open this discussion here to find the best suitable form of the following method: /** * Apply a transform to the code built by a handler, directing results to this builder. * * @param transform the transform to apply to the code built by the handler * @param handler the handler that receives a {@linkplain CodeBuilder} to * build the code. * @return this builder */ default CodeBuilder transforming(CodeTransform transform, Consumer handler) My proposal is to align it more with `CodeBuilder::block` method and emphasize more the bytecode block than the transformation itself. I propose to change the method name and arguments to: /** * Add a transformed lexical block to the method being built. *

* Within this block, the {@link #startLabel()} and {@link #endLabel()} correspond * to the start and end of the block, and the {@link BlockCodeBuilder#breakLabel()} * also corresponds to the end of the block. * * @param handler handler that receives a {@linkplain BlockCodeBuilder} to * generate the body of the lexical block. * @param transform the transform to apply to the lexical block generated by handler * @return this builder */ default CodeBuilder transformedBlock(Consumer handler, CodeTransform transform) Or alternatively: default CodeBuilder transformBlock(Consumer handler, CodeTransform transform) or: default CodeBuilder transformingBlock(Consumer handler, CodeTransform transform) or just simple: default CodeBuilder block(Consumer handler, CodeTransform transform) Please let me know which version do you prefer or propose alternatives. Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Tue Mar 7 15:39:58 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Tue, 7 Mar 2023 15:39:58 +0000 Subject: Classfile API tracking RFEs In-Reply-To: References: Message-ID: 5. Classfile API CodeBuilder::transforming clarification From: classfile-api-dev on behalf of Adam Sotona Date: Tuesday, 7 March 2023 15:40 To: classfile-api-dev at openjdk.org Subject: Classfile API tracking RFEs Hi, Some of the proposed RFEs fall beyond the initial Classfile API pull request and we should probably start filling them to JBS or find another way to track them. Here are listed citations (for temporary tracking purpose) what I?ve collected from the feedback: 1. switch seems the one instruction for which an high-level variant (like trying) could be useful, as generating code for that can be quite painful. 2. I noticed that this pattern of start/endScope is here, but also in ExceptionCatch, LocalVariable and LocalVariableType. Consider using a common interface for "instructions that are associated with a range". 3. Should we add a helper method on CodeBuilder that does the new + dup + invoke special dance? 4. Another issue with the model is ClassTypeSig does not validate its input. Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.van.acken at gmail.com Tue Mar 7 15:41:46 2023 From: michael.van.acken at gmail.com (Michael van Acken) Date: Tue, 7 Mar 2023 16:41:46 +0100 Subject: Classfile API tracking RFEs In-Reply-To: References: Message-ID: Below some minor comments based on my experience from last summer. Am Di., 7. M?rz 2023 um 15:40 Uhr schrieb Adam Sotona < adam.sotona at oracle.com>: > Hi, > > Some of the proposed RFEs fall beyond the initial Classfile API pull > request and we should probably start filling them to JBS or find another > way to track them. > > > > Here are listed citations (for temporary tracking purpose) what I?ve > collected from the feedback: > > > > 1. switch seems the one instruction for which an high-level variant > (like trying) could be useful, as generating code for that can be > quite painful. > > I'm not a user of trying and I probably would not use switch as well, but a method wrapping tableSwitchInstruction() and lookupSwitchInstruction() would be useful to me. As Brian pointed out, one is optimizing for byte code size when choosing between the two. Being effectively isolated from byte codes and their size by the Classfile API, I would appreciate if there was a way to delegate this decision to the API. (This is not a vote against a trying-like higher level switch variant. My last major code generation bug was the bytecode equivalent of omitting the breaks from a switch, i.e. forgetting the GOTO at the end of paths that did not end with a RETURN.) In a similar vein and along the lines of constantInstruction(), a method moving newarray(), anewarray(), and multianewarray() under a single umbrella would be helpful. Oh, and constantInstruction() properly handling -0.0d and -0.0f would be nice. This is the one patch I have been carrying around since the very beginning that has not been resolved. -- mva -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.sandoz at oracle.com Tue Mar 7 17:05:32 2023 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 7 Mar 2023 17:05:32 +0000 Subject: Classfile API CodeBuilder::transforming clarification In-Reply-To: <18587a63-49d2-31bd-c956-6400d208bba4@oracle.com> References: <18587a63-49d2-31bd-c956-6400d208bba4@oracle.com> Message-ID: <14CF3275-7A3C-41E0-A54C-AD5C57A07FBA@oracle.com> When reviewing I found transforming to be the trickiest area and also the most impactful, and the jewel in the overall design. Hence a modest suggestion about the naming of transform vs. transforming, the difference of which is too subtle and confusing IMO. I did state in summarizing the review that this area likely needs some more soak time after integration. Paul. > On Mar 7, 2023, at 6:42 AM, Brian Goetz wrote: > > I'd like some time to think about this; this was one of the trickiest corners of the API to design. Its quite possible there's an inconsistency or vestige, but also possible something subtle is going on. > > Can we put this on the RFE list as well? > > On 3/7/2023 5:09 AM, Adam Sotona wrote: >> Hi, >> During the Classfile API reviews there have been raised concerns about `CodeBuilder::transforming` method, for details see: >> https://github.com/openjdk/jdk/pull/10982/files/074dd30401a68638a24c157595caeb96b3511614#r1123858513 >> >> I would like to (re-)open this discussion here to find the best suitable form of the following method: >> /** >> * Apply a transform to the code built by a handler, directing results to this builder. >> * >> * @param transform the transform to apply to the code built by the handler >> * @param handler the handler that receives a {@linkplain CodeBuilder} to >> * build the code. >> * @return this builder >> */ >> default CodeBuilder transforming(CodeTransform transform, Consumer handler) >> >> My proposal is to align it more with `CodeBuilder::block` method and emphasize more the bytecode block than the transformation itself. >> I propose to change the method name and arguments to: >> /** >> * Add a transformed lexical block to the method being built. >> *

>> * Within this block, the {@link #startLabel()} and {@link #endLabel()} correspond >> * to the start and end of the block, and the {@link BlockCodeBuilder#breakLabel()} >> * also corresponds to the end of the block. >> * >> * @param handler handler that receives a {@linkplain BlockCodeBuilder} to >> * generate the body of the lexical block. >> * @param transform the transform to apply to the lexical block generated by handler >> * @return this builder >> */ >> default CodeBuilder transformedBlock(Consumer handler, CodeTransform transform) >> >> Or alternatively: >> default CodeBuilder transformBlock(Consumer handler, CodeTransform transform) >> >> or: >> default CodeBuilder transformingBlock(Consumer handler, CodeTransform transform) >> >> or just simple: >> default CodeBuilder block(Consumer handler, CodeTransform transform) >> >> >> Please let me know which version do you prefer or propose alternatives. >> >> Thanks, >> Adam From adam.sotona at oracle.com Wed Mar 8 07:05:10 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Wed, 8 Mar 2023 07:05:10 +0000 Subject: [External] : Re: Classfile API tracking RFEs In-Reply-To: References: Message-ID: Oh, and constantInstruction() properly handling -0.0d and -0.0f would be nice. This is the one patch I have been carrying around since the very beginning that has not been resolved. -- mva Hi Michael, Do you know some cases of -0.0d and -0.0f wrong handling? Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.van.acken at gmail.com Wed Mar 8 07:23:14 2023 From: michael.van.acken at gmail.com (Michael van Acken) Date: Wed, 8 Mar 2023 08:23:14 +0100 Subject: [External] : Re: Classfile API tracking RFEs In-Reply-To: References: Message-ID: Am Mi., 8. M?rz 2023 um 08:05 Uhr schrieb Adam Sotona < adam.sotona at oracle.com>: > > > Oh, and constantInstruction() properly handling -0.0d and -0.0f would be > nice. This is the > > one patch I have been carrying around since the very beginning that has > not been resolved. > > > > -- mva > > > > Hi Michael, > > Do you know some cases of -0.0d and -0.0f wrong handling? > I don't quite understand the question. Negative zero compares as ==-equal to positive zero under IEEE-754, but the two are distinct bit patterns. Both javac and jshell treat them as distinct, as do equals() and compareTo(). jshell> 0.0f == -0.0f $3 ==> true jshell> Float.floatToRawIntBits(0.0f) $4 ==> 0 jshell> Float.floatToRawIntBits(-0.0f) $5 ==> -2147483648 The current constantInstruction would lead to a different output for the last line, by squashing the value to 0.0f. -- mva -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Wed Mar 8 08:19:00 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Wed, 8 Mar 2023 08:19:00 +0000 Subject: [External] : Re: Classfile API tracking RFEs In-Reply-To: References: Message-ID: On 08.03.2023 8:23, "Michael van Acken" wrote: Am Mi., 8. M?rz 2023 um 08:05 Uhr schrieb Adam Sotona >: Oh, and constantInstruction() properly handling -0.0d and -0.0f would be nice. This is the one patch I have been carrying around since the very beginning that has not been resolved. -- mva Hi Michael, Do you know some cases of -0.0d and -0.0f wrong handling? I don't quite understand the question. Negative zero compares as ==-equal to positive zero under IEEE-754, but the two are distinct bit patterns. Both javac and jshell treat them as distinct, as do equals() and compareTo(). jshell> 0.0f == -0.0f $3 ==> true jshell> Float.floatToRawIntBits(0.0f) $4 ==> 0 jshell> Float.floatToRawIntBits(-0.0f) $5 ==> -2147483648 The current constantInstruction would lead to a different output for the last line, by squashing the value to 0.0f. -- mva Thanks for pointing it out, I?ve added test and fixed it, see: https://github.com/openjdk/jdk/pull/10982/commits/65a69135f6623e0f78aeaf8cd7b2ebdc8e1bbb27 -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Thu Mar 9 15:55:47 2023 From: liangchenblue at gmail.com (-) Date: Thu, 9 Mar 2023 09:55:47 -0600 Subject: Status of Classfile API JEP Message-ID: Hello, Congratulations! The Classfile API implementation was just integrated into JDK mainline. However, the associated JEP at https://openjdk.org/jeps/8280389 is still a draft. Is the Classfile API to be made an incubating module in JDK 21, or will it stay within jdk.internal for the time being? From brian.goetz at oracle.com Thu Mar 9 16:46:57 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 9 Mar 2023 11:46:57 -0500 Subject: Status of Classfile API JEP In-Reply-To: References: Message-ID: <61ae7729-97d7-49ab-ebd3-3b3b35b09fe3@oracle.com> The current integration treats the library as internal, and it is but one step in a long chain.? Next steps will include updating the JEP, and once the JEP integrates, exposing the library as a _preview API feature_. On 3/9/2023 10:55 AM, - wrote: > Hello, > Congratulations! The Classfile API implementation was just integrated > into JDK mainline. However, the associated JEP at > https://openjdk.org/jeps/8280389 is still a draft. Is the Classfile > API to be made an incubating module in JDK 21, or will it stay within > jdk.internal for the time being? -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Thu Mar 9 18:07:10 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Thu, 9 Mar 2023 18:07:10 +0000 Subject: Classfile API implementation merged to JDK Message-ID: Hi, As you?ve probably noticed Classfile API implementation has been merged into JDK master as commit 4655b79. Implications on our work are following: Documentation Online API documentation will be updated at the existing URL until further notice. JDK Sandbox branches JDK Sandbox branches https://github.com/openjdk/jdk-sandbox/tree/classfile-api-branch and https://github.com/openjdk/jdk-sandbox/tree/classfile-api-dev-branch will no longer serve for primary development. JDK Integration development work JDK integrations are tracked under https://bugs.openjdk.org/browse/JDK-8294957 ?Consolidate JDK class files parsing, generating, and transforming (umbrella)? as individual subtasks and individual pull requests. Sources Classfile Processing API sources are now an integral part of java.base JDK module sources. Building For JDK build instructions please see online documentation. Testing Classfile Processing API tests are a part of JDK tests. Test can be selectivelly executed as: make test TEST=jdk/classfile Benchmarking Classfile Processing API benchmarks are a part of JDK Microbenchmark Suite. Benchmarks can be selectively executed as: make test TEST=micro:org.openjdk.bench.jdk.classfile.+ See JEP 230: Microbenchmark Suite for more information about JDK benchmarks. Thank you, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Thu Mar 9 18:18:10 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 9 Mar 2023 13:18:10 -0500 Subject: Classfile API implementation merged to JDK In-Reply-To: References: Message-ID: <7bd3ba10-1772-f016-498d-46f4a69ee4f3@oracle.com> Great milestone! More to come.... On 3/9/2023 1:07 PM, Adam Sotona wrote: > > Hi, > > As you?ve probably noticed Classfile API implementation has been > merged into JDK master as commit4655b79 > . > Implications on our work are following: > > > Documentation > > Online API documentation > ?will > be updated at the existing URL until further notice. > > > JDK Sandbox branches > > JDK Sandbox branches > https://github.com/openjdk/jdk-sandbox/tree/classfile-api-branch and > https://github.com/openjdk/jdk-sandbox/tree/classfile-api-dev-branch > will no longer serve for primary development. > > > JDK Integration development work > > JDK integrations are tracked under > https://bugs.openjdk.org/browse/JDK-8294957 ?Consolidate JDK class > files parsing, generating, and transforming (umbrella)? as individual > subtasks and individual pull requests. > > > Sources > > Classfile Processing API sources are now an integral part of java.base > JDK module sources. > > > Building > > For JDK build instructions please seeonline documentation > . > > > Testing > > Classfile Processing API tests are a part of JDK tests. > > Test can be selectivelly executed as: > > |make test TEST=jdk/classfile| > > > Benchmarking > > Classfile Processing API benchmarks are a part of JDK Microbenchmark > Suite. > > Benchmarks can be selectively executed as: > > |make test TEST=micro:org.openjdk.bench.jdk.classfile.+| > > SeeJEP 230: Microbenchmark Suite > for more information > about JDK benchmarks. > > Thank you, > > Adam > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Thu Mar 9 19:30:42 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 9 Mar 2023 14:30:42 -0500 Subject: [External] : Re: Reflective access to bytes[] of loaded class In-Reply-To: References: <578390b7-b8f7-633c-fef7-dbe0722d4625@oracle.com> Message-ID: Doing `javap -p -c ` assumes file-system access to the entire application.? Indeed, defending against "has write access to the application files" is more difficult.? But malicious libraries may be invoked by application code, and while they may not have access to the file system, they have access to reflection.? So leaking this information through reflection makes it available to more attackers than simply having it on the file system. On 2/10/2023 8:54 AM, Gary Frost wrote: > Brian > > I am not sure how? having a reflective method to access bytecode opens > up any more security issues than having access to? javap -p -c > ? > > Unless the password itself (in your example) was injected by a > transformation of the code by ClassLoader/JVMTI/Java agent. > > Maybe I am missing something. > ------------------------------------------------------------------------ > *From:* Brian Goetz > *Sent:* Thursday, February 9, 2023 6:52 PM > *To:* Dan Heidinga ; Gary Frost > > *Cc:* classfile-api-dev at openjdk.org > *Subject:* Re: [External] : Re: Reflective access to bytes[] of loaded > class > One concern that we have is that unrestricted access to bytecode of > loaded classes may constitute an attack vector for malicious code. > > While we don't recommend people program like this: > > ??? boolean checkPassword(String pw) { > ??????? return "s00perSeekrit".equals(pw); > ??? } > > arbitrary access to bytecode will compromise the "security" of this > code in ways that the author might not have reasonably forseen. > > > > On 2/9/2023 1:24 PM, Dan Heidinga wrote: >> Thanks for confirming, Gary. >> >> On Thu, Feb 9, 2023 at 11:59 AM Gary Frost > > wrote: >> >> Dan >> >> I have indeed used JVMTI (and Java) agents for this. In the case >> of JVMTI by keeping a std::map, and providing >> a JNI call to get the bytes. >> >> It works well for cases where we know the set of classes we want >> bytes for at JVM 'launch time' , or there is something in the >> class (name match? constant pool entry?) that we can trigger to >> prune the size of the map.??? Otherwise we are forced to retain a >> map for all classes, just in case. >> >> And of course who knew whether our JVMTI agent was the last >> 'actor' in the chain of possible mutators to play with the bytes. >> >> >> Can you expand on the use case for getting the current bytes from the >> runtime?? As I said in the previous email, they aren't a good >> candidate for feeding back into the runtime due to the existing JVMTI >> agent process.? Is this mostly for testing purposes? >> >> --Dan >> >> >> >> Hence the desire for runtime help >> >> >> ------------------------------------------------------------------------ >> *From:* Dan Heidinga > > >> *Sent:* Thursday, February 9, 2023 4:35 PM >> *To:* Gary Frost > > >> *Cc:* classfile-api-dev at openjdk.org >> >> > > >> *Subject:* [External] : Re: Reflective access to bytes[] of >> loaded class >> >> >> On Thu, Feb 9, 2023 at 11:03 AM Gary Frost > > wrote: >> >> I would like to make a case for adding a reflection API for >> getting the bytes for class loaded by the VM as part of this >> Classfile API. >> >> Something akin to >> >> byte[] Class.getClassfileBytes(); >> >> >> I've often wanted the same kind of API when doing one off tests >> and minor modifications so I'm very sympathetic to the request.? >> Typically when doing more "serious" class modification, I've used >> the Instrumentation::retransform API [0] or written a native >> JVMTI agent [1] and then such an api to get the current classfile >> bytes isn't required.? Does this match your typical use cases as >> well? >> >> The JVM does some interesting handling of classfile bytes in >> these cases - it allows non-retransform-capable agents to do a 1 >> time modification of the original classfile bytes and then saves >> those bytes away before passing them on to the >> retransform-capable agents.? One subsequent retransform events, >> those saved bytes are reused and only the retransform-capable >> agents get a chance to modify the bytes. >> >> The reason this matters is that existing retransform-capable >> agents aren't expecting to see the modifications they've made >> present in the classfile bytes they get passed.? Providing an >> easy api to get the currently executing bytecodes will mean that >> these agents will now "see" their own modifications which may >> result in incompatibilities. >> >> I'd be interested in feedback from existing major agent providers >> to see how much of a problem such a change would cause them >> before pursuing it unless there's a more common pattern of use >> I'm unaware of. >> >> [0] >> https://cr.openjdk.java.net/~iris/se/17/latestSpec/api/java.instrument/java/lang/instrument/Instrumentation.html#retransformClasses(java.lang.Class.. >> .) >> >> [1] >> https://cr.openjdk.java.net/~iris/se/17/latestSpec/specs/jvmti.html#RetransformClasses >> >> >> >> >> >> At present developers usually resort to something like >> >> clazz.getClassLoader().getResourceAsStream( >> ??? clazz.getName().replace(".", "/") + ".class") >> >> Ignoring the fact that we may have just forced an expensive >> network fetch, to bytes that the JVM clearly already has >> 'squirrelled away' somewhere... >> >> For me this is hugely problematic as there is no guarentee >> that the bytes fetched from such a stream will match the >> bytes that the JVM is using for the class in question.... >> >> Java offers all sorts of opportunities (JVMTI agents, Java >> agents, even custom ClassLoaders) for mutating the incoming >> classfile's bytes. All of which the stream fetched via >> getResourceAsStream() completely sidesteps. >> >> >> See above (or the JVMTI spec) for which bytes get passed through >> to agents.? The story is more complex than it first appears. >> >> --Dan >> >> >> This may well be considered 'out of bounds' for the Classfile >> API, but I think we should consider it, as it seems to be >> something that users of the Classfile API will need going >> forward. >> >> Gary >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.sandoz at oracle.com Thu Mar 9 23:55:26 2023 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 9 Mar 2023 23:55:26 +0000 Subject: Classfile API CodeBuilder::transforming clarification In-Reply-To: References: Message-ID: <4B152516-3E8B-41A3-854D-20C2DF21946E@oracle.com> ClassfileBuilder::transform and CodeBuilder::transforming are closely connected in behavior (with different implementation performance profiles); they differ in the source of events that are pumped through the transformer, which then pumps the flat-mapped results into the receiving builder. ClassfileBuilder::transform could be generalized to accept Iterable. AFAICT its behavior is not tied to the specifics of CompoundElement and could more generally accept a sequence of elements of a compound element or produced by something else. For code, ClassfileBuilder::transform could be implemented using CodeBuilder::transforming, where the handler is iter::forEach, and where iter is a variable of type Iterable. From this perspective CodeBuilder::transforming seems like the more general builder-based transform primitive. However, I don?t have a strong sense whether this generality is useful for anything accept the building of code. In that respect associating the transform with a block seems very useful (I agree with the argument order). Overriding block may not do it justice, it seems very powerful, which makes we wanna keep the transform name e.g, transformBlock. Paul. > On Mar 9, 2023, at 10:51 AM, Brian Goetz wrote: > > Coming back to this ... > > At the top level, we have ClassModel::transform; this takes a materialized ClassModel and applies a ClassTransform, returning the new byte[]. > > This method does not have analogues in {Method,Field,Code}Model, nor does it override ClassfileBuilder::transform; it takes a whole class in and produces a whole class out. Therefore, I think this method lives in the wrong place; it should live in Classfile, alongside parse/build, where all the other "operate on an entire classfile" operations live. The following overloads make sense: > > byte[] transformClass(byte[] classfile, ClassTransform transform) > byte[] transformClass(ClassModel classModel, ClassTransform transform) > > This also makes transformation more discoverable, since it lives alongside of build and parse. > > We have _static_ methods transforming{Code,Fields,Methods,MethodBodies} which are lifting operations on transforms. These seem fine. > > The confusing ones are CodeBuilder::transform and ClassfileBuilder::transforming. > > // ClassfileBuilder > /** > * Apply a transform to a model, directing results to this builder. > * @param model the model to transform > * @param transform the transform to apply > */ > default void transform(CompoundElement model, ClassfileTransform transform) { > @SuppressWarnings("unchecked") > B builder = (B) this; > var resolved = transform.resolve(builder); > resolved.startHandler().run(); > model.forEachElement(resolved.consumer()); > resolved.endHandler().run(); > } > > The purpose of this method is to run a transform on a compound element, with the output of the transform bound to the (receiver) builder. This is not so much intended to be called directly by users, as much as an implementation convenience for methods like MethodBuilder::transformCode, because the dance of resolving and applying a transform is easy to get wrong. Arguably this method doesn't belong as an instance method on Builder, as much as a static helper method somewhere that takes (model, transform, builder). If we can't find any uses of ClassfileBuilder::transform outside of the classfile API implementation itself, perhaps it should be moved to a static helper somewhere, such as in ClassfileTransform, and give it a name like applyTransform(model, transform, builder). > > Which leaves us with CodeBuilder::transforming. I don't find any uses of it in the implementation itself, and I had to piece together what it does... > > // CodeBuilder > /** > * Apply a transform to the code built by a handler, directing results to this builder. > * > * @param transform the transform to apply to the code built by the handler > * @param handler the handler that receives a {@linkplain CodeBuilder} to > * build the code. > * @return this builder > */ > default CodeBuilder transforming(CodeTransform transform, Consumer handler) { > var resolved = transform.resolve(this); > resolved.startHandler().run(); > handler.accept(new TransformingCodeBuilder(this, resolved.consumer())); > resolved.endHandler().run(); > return this; > } > > What it does is combine generating all or part of a method body with CodeBuilder, with transforming that code with a transform, binding the output of the transform to the current builder. The use case for this is when we have captured useful logic in a transform, and we want this to be applied equally to new methods (or parts of methods) we generate as to existing methods we find in the classfile. > > Now that I've swapped in what this does ... I agree with Adam's suggestion that it is misnamed. My first choice is to overload block(): > > CodeBuilder block(Consumer handler) > CodeBuilder block(CodeTransform transform, Consumer handler) > > Both are primarily block-building, the latter transforms the block on the fly. > > I prefer ordering the arguments (transform, block) rather than the other way, because anyone who uses this has likely already captured the transform in a variable, whereas the handler is usually an ad-hoc lambda. > > > > > > On 3/7/2023 5:09 AM, Adam Sotona wrote: >> Hi, >> During the Classfile API reviews there have been raised concerns about `CodeBuilder::transforming` method, for details see: >> https://github.com/openjdk/jdk/pull/10982/files/074dd30401a68638a24c157595caeb96b3511614#r1123858513 >> >> I would like to (re-)open this discussion here to find the best suitable form of the following method: >> /** >> * Apply a transform to the code built by a handler, directing results to this builder. >> * >> * @param transform the transform to apply to the code built by the handler >> * @param handler the handler that receives a {@linkplain CodeBuilder} to >> * build the code. >> * @return this builder >> */ >> default CodeBuilder transforming(CodeTransform transform, Consumer handler) >> >> My proposal is to align it more with `CodeBuilder::block` method and emphasize more the bytecode block than the transformation itself. >> I propose to change the method name and arguments to: >> /** >> * Add a transformed lexical block to the method being built. >> *

>> * Within this block, the {@link #startLabel()} and {@link #endLabel()} correspond >> * to the start and end of the block, and the {@link BlockCodeBuilder#breakLabel()} >> * also corresponds to the end of the block. >> * >> * @param handler handler that receives a {@linkplain BlockCodeBuilder} to >> * generate the body of the lexical block. >> * @param transform the transform to apply to the lexical block generated by handler >> * @return this builder >> */ >> default CodeBuilder transformedBlock(Consumer handler, CodeTransform transform) >> >> Or alternatively: >> default CodeBuilder transformBlock(Consumer handler, CodeTransform transform) >> >> or: >> default CodeBuilder transformingBlock(Consumer handler, CodeTransform transform) >> >> or just simple: >> default CodeBuilder block(Consumer handler, CodeTransform transform) >> >> >> Please let me know which version do you prefer or propose alternatives. >> >> Thanks, >> Adam From brian.goetz at oracle.com Fri Mar 10 14:54:21 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 10 Mar 2023 09:54:21 -0500 Subject: Classfile API CodeBuilder::transforming clarification In-Reply-To: <4B152516-3E8B-41A3-854D-20C2DF21946E@oracle.com> References: <4B152516-3E8B-41A3-854D-20C2DF21946E@oracle.com> Message-ID: <43772af6-598e-717a-5aa4-d03b757906cb@oracle.com> > ClassfileBuilder::transform and CodeBuilder::transforming are closely connected in behavior (with different implementation performance profiles); they differ in the source of events that are pumped through the transformer, which then pumps the flat-mapped results into the receiving builder. > > ClassfileBuilder::transform could be generalized to accept Iterable. AFAICT its behavior is not tied to the specifics of CompoundElement and could more generally accept a sequence of elements of a compound element or produced by something else. Yes.? I think the first thing to address here is that ClassfileBuilder::transform is in the wrong place.? It is not really behavior on Builder, nor will users use it that way; it is implementation detail for "apply a transform on a model into a builder ".? (The only place the implementation uses `this` is to actually pass `this` to builder/transform factories.? So I think this should be a static method somewhere.? Even though it mostly serves the implementation, it should be accessible, so ClassfileTransform seems the natural home.? And it probably should be called something like applyTransform(transform, model, builder). CodeBuilder::transforming is useful to users; as Adam points out, it is akin to "add a block, but transform first."? And I can easily imagine that in complex transform cases, you'll want to be able to apply transforms to ad-hoc built code as well as existing code. > For code, ClassfileBuilder::transform could be implemented using CodeBuilder::transforming, where the handler is iter::forEach, and where iter is a variable of type Iterable. > > From this perspective CodeBuilder::transforming seems like the more general builder-based transform primitive. > > However, I don?t have a strong sense whether this generality is useful for anything accept the building of code. In that respect associating the transform with a block seems very useful (I agree with the argument order). Overriding block may not do it justice, it seems very powerful, which makes we wanna keep the transform name e.g, transformBlock. > > Paul. > >> On Mar 9, 2023, at 10:51 AM, Brian Goetz wrote: >> >> Coming back to this ... >> >> At the top level, we have ClassModel::transform; this takes a materialized ClassModel and applies a ClassTransform, returning the new byte[]. >> >> This method does not have analogues in {Method,Field,Code}Model, nor does it override ClassfileBuilder::transform; it takes a whole class in and produces a whole class out. Therefore, I think this method lives in the wrong place; it should live in Classfile, alongside parse/build, where all the other "operate on an entire classfile" operations live. The following overloads make sense: >> >> byte[] transformClass(byte[] classfile, ClassTransform transform) >> byte[] transformClass(ClassModel classModel, ClassTransform transform) >> >> This also makes transformation more discoverable, since it lives alongside of build and parse. >> >> We have _static_ methods transforming{Code,Fields,Methods,MethodBodies} which are lifting operations on transforms. These seem fine. >> >> The confusing ones are CodeBuilder::transform and ClassfileBuilder::transforming. >> >> // ClassfileBuilder >> /** >> * Apply a transform to a model, directing results to this builder. >> * @param model the model to transform >> * @param transform the transform to apply >> */ >> default void transform(CompoundElement model, ClassfileTransform transform) { >> @SuppressWarnings("unchecked") >> B builder = (B) this; >> var resolved = transform.resolve(builder); >> resolved.startHandler().run(); >> model.forEachElement(resolved.consumer()); >> resolved.endHandler().run(); >> } >> >> The purpose of this method is to run a transform on a compound element, with the output of the transform bound to the (receiver) builder. This is not so much intended to be called directly by users, as much as an implementation convenience for methods like MethodBuilder::transformCode, because the dance of resolving and applying a transform is easy to get wrong. Arguably this method doesn't belong as an instance method on Builder, as much as a static helper method somewhere that takes (model, transform, builder). If we can't find any uses of ClassfileBuilder::transform outside of the classfile API implementation itself, perhaps it should be moved to a static helper somewhere, such as in ClassfileTransform, and give it a name like applyTransform(model, transform, builder). >> >> Which leaves us with CodeBuilder::transforming. I don't find any uses of it in the implementation itself, and I had to piece together what it does... >> >> // CodeBuilder >> /** >> * Apply a transform to the code built by a handler, directing results to this builder. >> * >> * @param transform the transform to apply to the code built by the handler >> * @param handler the handler that receives a {@linkplain CodeBuilder} to >> * build the code. >> * @return this builder >> */ >> default CodeBuilder transforming(CodeTransform transform, Consumer handler) { >> var resolved = transform.resolve(this); >> resolved.startHandler().run(); >> handler.accept(new TransformingCodeBuilder(this, resolved.consumer())); >> resolved.endHandler().run(); >> return this; >> } >> >> What it does is combine generating all or part of a method body with CodeBuilder, with transforming that code with a transform, binding the output of the transform to the current builder. The use case for this is when we have captured useful logic in a transform, and we want this to be applied equally to new methods (or parts of methods) we generate as to existing methods we find in the classfile. >> >> Now that I've swapped in what this does ... I agree with Adam's suggestion that it is misnamed. My first choice is to overload block(): >> >> CodeBuilder block(Consumer handler) >> CodeBuilder block(CodeTransform transform, Consumer handler) >> >> Both are primarily block-building, the latter transforms the block on the fly. >> >> I prefer ordering the arguments (transform, block) rather than the other way, because anyone who uses this has likely already captured the transform in a variable, whereas the handler is usually an ad-hoc lambda. >> >> >> >> >> >> On 3/7/2023 5:09 AM, Adam Sotona wrote: >>> Hi, >>> During the Classfile API reviews there have been raised concerns about `CodeBuilder::transforming` method, for details see: >>> https://github.com/openjdk/jdk/pull/10982/files/074dd30401a68638a24c157595caeb96b3511614#r1123858513 >>> >>> I would like to (re-)open this discussion here to find the best suitable form of the following method: >>> /** >>> * Apply a transform to the code built by a handler, directing results to this builder. >>> * >>> * @param transform the transform to apply to the code built by the handler >>> * @param handler the handler that receives a {@linkplain CodeBuilder} to >>> * build the code. >>> * @return this builder >>> */ >>> default CodeBuilder transforming(CodeTransform transform, Consumer handler) >>> >>> My proposal is to align it more with `CodeBuilder::block` method and emphasize more the bytecode block than the transformation itself. >>> I propose to change the method name and arguments to: >>> /** >>> * Add a transformed lexical block to the method being built. >>> *

>>> * Within this block, the {@link #startLabel()} and {@link #endLabel()} correspond >>> * to the start and end of the block, and the {@link BlockCodeBuilder#breakLabel()} >>> * also corresponds to the end of the block. >>> * >>> * @param handler handler that receives a {@linkplain BlockCodeBuilder} to >>> * generate the body of the lexical block. >>> * @param transform the transform to apply to the lexical block generated by handler >>> * @return this builder >>> */ >>> default CodeBuilder transformedBlock(Consumer handler, CodeTransform transform) >>> >>> Or alternatively: >>> default CodeBuilder transformBlock(Consumer handler, CodeTransform transform) >>> >>> or: >>> default CodeBuilder transformingBlock(Consumer handler, CodeTransform transform) >>> >>> or just simple: >>> default CodeBuilder block(Consumer handler, CodeTransform transform) >>> >>> >>> Please let me know which version do you prefer or propose alternatives. >>> >>> Thanks, >>> Adam From liangchenblue at gmail.com Mon Mar 13 17:49:02 2023 From: liangchenblue at gmail.com (-) Date: Mon, 13 Mar 2023 12:49:02 -0500 Subject: New constant API addition proposals for Classfile API Message-ID: Hi, After migrating 26/30 classes in test/jdk/java to Classfile API (https://github.com/liachmodded/jdk/commit/65ca55bc2e838823471de40f0e16f9770d932117), I suggest to add these new APIs to constant API for ease of Classfile API usage: 1. In ConstantDescs: public static final String CONSTRUCTOR_NAME = ""; public static final String CLASS_INITIALIZER_NAME = ""; public static final MethodTypeDesc MTD_void = MethodTypeDesc.of(CD_void); Since every instance class always has a constructor, and most likely, its constructor has to call Object's no-arg constructor. The same method type descriptor is shared for the static initializer too. 2. In ConstantDescs: public static final DirectMethodHandleDesc BSM_CLASS_DATA = ... public static final DirectMethodHandleDesc BSM_CLASS_DATA_AT = ... Representing MethodHandles.classData and MethodHandles.classDataAt, used by hidden classes frequently. 3. In DynamicConstantDesc: public static DynamicConstantDesc ofTyped(DirectMethodHandleDesc bootstrapMethod, ClassDesc constantType, ConstantDesc... bootstrapArgs) This is equivalent to calling ofNamed with ConstantDescs.DEFAULT_NAME, useful for classData and classDataAt bootstrap methods. From brian.goetz at oracle.com Mon Mar 13 17:54:23 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 13 Mar 2023 13:54:23 -0400 Subject: New constant API addition proposals for Classfile API In-Reply-To: References: Message-ID: <8e5b1fe0-b3ed-8657-8c97-a01dc34d3149@oracle.com> (1) and (2) make total sense; (3) seems a little much, since it is just defaulting in a single argument? On 3/13/2023 1:49 PM, - wrote: > Hi, > After migrating 26/30 classes in test/jdk/java to Classfile API > (https://github.com/liachmodded/jdk/commit/65ca55bc2e838823471de40f0e16f9770d932117), > I suggest to add these new APIs to constant API for ease of Classfile > API usage: > > 1. In ConstantDescs: > public static final String CONSTRUCTOR_NAME = ""; > public static final String CLASS_INITIALIZER_NAME = ""; > public static final MethodTypeDesc MTD_void = MethodTypeDesc.of(CD_void); > > Since every instance class always has a constructor, and most likely, > its constructor has to call Object's no-arg constructor. The same > method type descriptor is shared for the static initializer too. > > 2. In ConstantDescs: > public static final DirectMethodHandleDesc BSM_CLASS_DATA = ... > public static final DirectMethodHandleDesc BSM_CLASS_DATA_AT = ... > > Representing MethodHandles.classData and MethodHandles.classDataAt, > used by hidden classes frequently. > > 3. In DynamicConstantDesc: > public static DynamicConstantDesc ofTyped(DirectMethodHandleDesc > bootstrapMethod, ClassDesc constantType, ConstantDesc... > bootstrapArgs) > > This is equivalent to calling ofNamed with ConstantDescs.DEFAULT_NAME, > useful for classData and classDataAt bootstrap methods. -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Mon Mar 13 18:41:32 2023 From: liangchenblue at gmail.com (-) Date: Mon, 13 Mar 2023 13:41:32 -0500 Subject: New constant API addition proposals for Classfile API In-Reply-To: <8e5b1fe0-b3ed-8657-8c97-a01dc34d3149@oracle.com> References: <8e5b1fe0-b3ed-8657-8c97-a01dc34d3149@oracle.com> Message-ID: Agreed, that I indeed have yet to see usages of (3) besides classData condys. It will be dropped. I have a patch ready for (2) and will open a pull request for it. For (1), I have another question: I feel CONSTRUCTOR_NAME and CLASS_INITIALIZER_NAME are not concise enough. Can you think of more concise names to facilitate their usage? On Mon, Mar 13, 2023 at 12:54?PM Brian Goetz wrote: > > (1) and (2) make total sense; (3) seems a little much, since it is just defaulting in a single argument? > > > On 3/13/2023 1:49 PM, - wrote: > > Hi, > After migrating 26/30 classes in test/jdk/java to Classfile API > (https://github.com/liachmodded/jdk/commit/65ca55bc2e838823471de40f0e16f9770d932117), > I suggest to add these new APIs to constant API for ease of Classfile > API usage: > > 1. In ConstantDescs: > public static final String CONSTRUCTOR_NAME = ""; > public static final String CLASS_INITIALIZER_NAME = ""; > public static final MethodTypeDesc MTD_void = MethodTypeDesc.of(CD_void); > > Since every instance class always has a constructor, and most likely, > its constructor has to call Object's no-arg constructor. The same > method type descriptor is shared for the static initializer too. > > 2. In ConstantDescs: > public static final DirectMethodHandleDesc BSM_CLASS_DATA = ... > public static final DirectMethodHandleDesc BSM_CLASS_DATA_AT = ... > > Representing MethodHandles.classData and MethodHandles.classDataAt, > used by hidden classes frequently. > > 3. In DynamicConstantDesc: > public static DynamicConstantDesc ofTyped(DirectMethodHandleDesc > bootstrapMethod, ClassDesc constantType, ConstantDesc... > bootstrapArgs) > > This is equivalent to calling ofNamed with ConstantDescs.DEFAULT_NAME, > useful for classData and classDataAt bootstrap methods. > > From forax at univ-mlv.fr Mon Mar 13 19:09:22 2023 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 13 Mar 2023 20:09:22 +0100 (CET) Subject: New constant API addition proposals for Classfile API In-Reply-To: <8e5b1fe0-b3ed-8657-8c97-a01dc34d3149@oracle.com> References: <8e5b1fe0-b3ed-8657-8c97-a01dc34d3149@oracle.com> Message-ID: <717342768.10186129.1678734562431.JavaMail.zimbra@u-pem.fr> As a data point, i've never used classData() directly from a condy, mostly because either - I had some other constraints to check, - the requirement that the name is "_" make it uneasy to read the generated bytecode, - classData() is called in the static block. regards, R?mi > From: "Brian Goetz" > To: "-" , "classfile-api-dev" > > Sent: Monday, March 13, 2023 6:54:23 PM > Subject: Re: New constant API addition proposals for Classfile API > (1) and (2) make total sense; (3) seems a little much, since it is just > defaulting in a single argument? > On 3/13/2023 1:49 PM, - wrote: >> Hi, >> After migrating 26/30 classes in test/jdk/java to Classfile API >> ( [ >> https://github.com/liachmodded/jdk/commit/65ca55bc2e838823471de40f0e16f9770d932117 >> | >> https://github.com/liachmodded/jdk/commit/65ca55bc2e838823471de40f0e16f9770d932117 >> ] ), >> I suggest to add these new APIs to constant API for ease of Classfile >> API usage: >> 1. In ConstantDescs: >> public static final String CONSTRUCTOR_NAME = ""; >> public static final String CLASS_INITIALIZER_NAME = ""; >> public static final MethodTypeDesc MTD_void = MethodTypeDesc.of(CD_void); >> Since every instance class always has a constructor, and most likely, >> its constructor has to call Object's no-arg constructor. The same >> method type descriptor is shared for the static initializer too. >> 2. In ConstantDescs: >> public static final DirectMethodHandleDesc BSM_CLASS_DATA = ... >> public static final DirectMethodHandleDesc BSM_CLASS_DATA_AT = ... >> Representing MethodHandles.classData and MethodHandles.classDataAt, >> used by hidden classes frequently. >> 3. In DynamicConstantDesc: >> public static DynamicConstantDesc ofTyped(DirectMethodHandleDesc >> bootstrapMethod, ClassDesc constantType, ConstantDesc... >> bootstrapArgs) >> This is equivalent to calling ofNamed with ConstantDescs.DEFAULT_NAME, >> useful for classData and classDataAt bootstrap methods. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jorn.vernee at oracle.com Tue Mar 14 15:31:37 2023 From: jorn.vernee at oracle.com (Jorn Vernee) Date: Tue, 14 Mar 2023 16:31:37 +0100 Subject: Usage feedback rewriting jdk.internal.foreign.abi.BindingSpecializer Message-ID: <5f7e3cb5-4458-4aca-b420-e43e937cf9e2@oracle.com> Hello, I've re-written jdk.internal.foreign.abi.BindingSpecializer to use the new class file API. The patch can be found here: https://github.com/openjdk/panama-foreign/compare/foreign-memaccess+abi...JornVernee:panama-foreign:CFA I did this mostly based on just the javadoc. Some things I noted while doing this: 1. The translation is really straightforward, and there were a lot of places where simplification was possible. Such as, using CodeBuilder::constantInstruction is much nicer than manually having to find the right bytecode for the constant you have. Another nice thing was the CodeBuilder::parameterSlot method. I previously had to translate parameter indices to slots manually. Great work! 2. I'm not sure what the expected way of mapping a Class (or ClassDesc) into a TypeKind is. I couldn't find a factory for that in TypeKind. The code I have often needs to map Class objects into TypeKinds. I ended up writing a manual 'mapper' method instead [1]. 3. Sometimes I need to emit a constant zero for a particular type. If I know the type is e.g. long, I can use `cb.constantInstruction(0L)`, but sometimes the type is not statically known, and it would be nice if there was a CodeBuilder::constantZero(TypeKind) for that. I've implemented this manually as well [2] 4. The same goes for `dup`. It would be nice if there was a CodeBuilder::dup(TypeKind) that automatically selects either dup or dup2. See [3] 5. I'll throw in a bike shed comment as well :) CodeBuilder::labelBinding sounds a bit strange. From the name, I initially wasn't sure whether this was returning an existing binding for a previously bound label, or binding the label I passed it. (it also doesn't help that the method doesn't have any javadoc at the moment). I think the name "bindLabel" (or just "bind", as we have in HotSpot) would be more intuitive. Cheers, Jorn [1]: https://github.com/openjdk/panama-foreign/compare/foreign-memaccess+abi...JornVernee:panama-foreign:CFA#diff-2a53a53e1a977d7dbd5591e63c3b735d4db31744294625ae0d856d2a902d996eR414 [2]: https://github.com/openjdk/panama-foreign/compare/foreign-memaccess+abi...JornVernee:panama-foreign:CFA#diff-2a53a53e1a977d7dbd5591e63c3b735d4db31744294625ae0d856d2a902d996eR959 [3]: https://github.com/openjdk/panama-foreign/compare/foreign-memaccess+abi...JornVernee:panama-foreign:CFA#diff-2a53a53e1a977d7dbd5591e63c3b735d4db31744294625ae0d856d2a902d996eR947 From brian.goetz at oracle.com Tue Mar 14 16:14:53 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 14 Mar 2023 12:14:53 -0400 Subject: Usage feedback rewriting jdk.internal.foreign.abi.BindingSpecializer In-Reply-To: <5f7e3cb5-4458-4aca-b420-e43e937cf9e2@oracle.com> References: <5f7e3cb5-4458-4aca-b420-e43e937cf9e2@oracle.com> Message-ID: <48f69654-cb98-7bbc-5c24-9ac9b824c5bb@oracle.com> > 1. The translation is really straightforward, and there were a lot of > places where simplification was possible. Such as, using > CodeBuilder::constantInstruction is much nicer than manually having to > find the right bytecode for the constant you have. Another nice thing > was the CodeBuilder::parameterSlot method. I previously had to > translate parameter indices to slots manually. Great work! Thanks -- we suspect there is more low-hanging-fruit in this area, so if you run into obvious candidates, please suggest. > 2. I'm not sure what the expected way of mapping a Class (or > ClassDesc) into a TypeKind is. I couldn't find a factory for that in > TypeKind. The code I have often needs to map Class objects into > TypeKinds. I ended up writing a manual 'mapper' method instead [1]. There is TypeKind.fromDescriptor(s); you can go from either Class or ClassDesc to descriptor fairly easily -- is that good enough?? Do the docs need to be updated to guide you towards that? > 3. Sometimes I need to emit a constant zero for a particular type. If > I know the type is e.g. long, I can use `cb.constantInstruction(0L)`, > but sometimes the type is not statically known, and it would be nice > if there was a CodeBuilder::constantZero(TypeKind) for that. I've > implemented this manually as well [2] Not sure what you are asking for here.? There's no constant pool form for "short constant".? There's bipush/sipush, though.? What bytecode are you hoping to get out? > 4. The same goes for `dup`. It would be nice if there was a > CodeBuilder::dup(TypeKind) that automatically selects either dup or > dup2. See [3] That seems pretty simple to add. > 5. I'll throw in a bike shed comment as well :) > CodeBuilder::labelBinding sounds a bit strange. From the name, I > initially wasn't sure whether this was returning an existing binding > for a previously bound label, or binding the label I passed it. (it > also doesn't help that the method doesn't have any javadoc at the > moment). I think the name "bindLabel" (or just "bind", as we have in > HotSpot) would be more intuitive. Yeah, I agree.? (I think we probably had something more like you suggest, and it got changed in a refactoring that probably made sense locally but may make less sense to users.)? Another candidate name is labelTarget(label) but I think `bindLabel` is more clear as to what it does. From liangchenblue at gmail.com Tue Mar 14 16:23:34 2023 From: liangchenblue at gmail.com (-) Date: Tue, 14 Mar 2023 11:23:34 -0500 Subject: Usage feedback rewriting jdk.internal.foreign.abi.BindingSpecializer In-Reply-To: <5f7e3cb5-4458-4aca-b420-e43e937cf9e2@oracle.com> References: <5f7e3cb5-4458-4aca-b420-e43e937cf9e2@oracle.com> Message-ID: For TypeKind, I think we can add a public static TypeKind from(TypeDescriptor.OfField desc) { return fromDescriptor(desc.descriptorString()); } which accepts both Class and ClassDesc. On Tue, Mar 14, 2023 at 10:31?AM Jorn Vernee wrote: > > Hello, > > I've re-written jdk.internal.foreign.abi.BindingSpecializer to use the > new class file API. The patch can be found here: > https://github.com/openjdk/panama-foreign/compare/foreign-memaccess+abi...JornVernee:panama-foreign:CFA > > I did this mostly based on just the javadoc. > > Some things I noted while doing this: > > 1. The translation is really straightforward, and there were a lot of > places where simplification was possible. Such as, using > CodeBuilder::constantInstruction is much nicer than manually having to > find the right bytecode for the constant you have. Another nice thing > was the CodeBuilder::parameterSlot method. I previously had to translate > parameter indices to slots manually. Great work! > > 2. I'm not sure what the expected way of mapping a Class (or > ClassDesc) into a TypeKind is. I couldn't find a factory for that in > TypeKind. The code I have often needs to map Class objects into > TypeKinds. I ended up writing a manual 'mapper' method instead [1]. > > 3. Sometimes I need to emit a constant zero for a particular type. If I > know the type is e.g. long, I can use `cb.constantInstruction(0L)`, but > sometimes the type is not statically known, and it would be nice if > there was a CodeBuilder::constantZero(TypeKind) for that. I've > implemented this manually as well [2] > > 4. The same goes for `dup`. It would be nice if there was a > CodeBuilder::dup(TypeKind) that automatically selects either dup or > dup2. See [3] > > 5. I'll throw in a bike shed comment as well :) > CodeBuilder::labelBinding sounds a bit strange. From the name, I > initially wasn't sure whether this was returning an existing binding for > a previously bound label, or binding the label I passed it. (it also > doesn't help that the method doesn't have any javadoc at the moment). I > think the name "bindLabel" (or just "bind", as we have in HotSpot) would > be more intuitive. > > Cheers, > Jorn > > [1]: > https://github.com/openjdk/panama-foreign/compare/foreign-memaccess+abi...JornVernee:panama-foreign:CFA#diff-2a53a53e1a977d7dbd5591e63c3b735d4db31744294625ae0d856d2a902d996eR414 > [2]: > https://github.com/openjdk/panama-foreign/compare/foreign-memaccess+abi...JornVernee:panama-foreign:CFA#diff-2a53a53e1a977d7dbd5591e63c3b735d4db31744294625ae0d856d2a902d996eR959 > [3]: > https://github.com/openjdk/panama-foreign/compare/foreign-memaccess+abi...JornVernee:panama-foreign:CFA#diff-2a53a53e1a977d7dbd5591e63c3b735d4db31744294625ae0d856d2a902d996eR947 > From jorn.vernee at oracle.com Tue Mar 14 16:31:31 2023 From: jorn.vernee at oracle.com (Jorn Vernee) Date: Tue, 14 Mar 2023 17:31:31 +0100 Subject: [External] : Re: Usage feedback rewriting jdk.internal.foreign.abi.BindingSpecializer In-Reply-To: References: <5f7e3cb5-4458-4aca-b420-e43e937cf9e2@oracle.com> Message-ID: <599e6345-b487-b27c-29a2-dc05fe3c1922@oracle.com> Yes, I think this would be nice. I noticed the fromDescriptor factory, but I guess it didn't feel like a great fit since I have to go from a high level type (Class/ClassDesc), to a low-level type (String), back to a high-level type (TypeKind). Considering the rest of the API all mostly accepts high-level types, like ClassDesc, this feels a bit 'off'. Like I'm missing a better way of doing the translation. I think what TypeKind::fromDescriptor does is functionally good enough though. Jorn On 14/03/2023 17:23, - wrote: > For TypeKind, I think we can add a > > public static TypeKind from(TypeDescriptor.OfField desc) { > return fromDescriptor(desc.descriptorString()); > } > > which accepts both Class and ClassDesc. > > On Tue, Mar 14, 2023 at 10:31?AM Jorn Vernee wrote: >> Hello, >> >> I've re-written jdk.internal.foreign.abi.BindingSpecializer to use the >> new class file API. The patch can be found here: >> https://urldefense.com/v3/__https://github.com/openjdk/panama-foreign/compare/foreign-memaccess*abi...JornVernee:panama-foreign:CFA__;Kw!!ACWV5N9M2RV99hQ!JOdvTlR2GTsZLrsAPt2-Y0Dw9_fFcfeLjl4rGsnme208eXiTxO_-UOruCz8eei5qhe31l1JA0XufFWxeBnULDQNf$ >> >> I did this mostly based on just the javadoc. >> >> Some things I noted while doing this: >> >> 1. The translation is really straightforward, and there were a lot of >> places where simplification was possible. Such as, using >> CodeBuilder::constantInstruction is much nicer than manually having to >> find the right bytecode for the constant you have. Another nice thing >> was the CodeBuilder::parameterSlot method. I previously had to translate >> parameter indices to slots manually. Great work! >> >> 2. I'm not sure what the expected way of mapping a Class (or >> ClassDesc) into a TypeKind is. I couldn't find a factory for that in >> TypeKind. The code I have often needs to map Class objects into >> TypeKinds. I ended up writing a manual 'mapper' method instead [1]. >> >> 3. Sometimes I need to emit a constant zero for a particular type. If I >> know the type is e.g. long, I can use `cb.constantInstruction(0L)`, but >> sometimes the type is not statically known, and it would be nice if >> there was a CodeBuilder::constantZero(TypeKind) for that. I've >> implemented this manually as well [2] >> >> 4. The same goes for `dup`. It would be nice if there was a >> CodeBuilder::dup(TypeKind) that automatically selects either dup or >> dup2. See [3] >> >> 5. I'll throw in a bike shed comment as well :) >> CodeBuilder::labelBinding sounds a bit strange. From the name, I >> initially wasn't sure whether this was returning an existing binding for >> a previously bound label, or binding the label I passed it. (it also >> doesn't help that the method doesn't have any javadoc at the moment). I >> think the name "bindLabel" (or just "bind", as we have in HotSpot) would >> be more intuitive. >> >> Cheers, >> Jorn >> >> [1]: >> https://urldefense.com/v3/__https://github.com/openjdk/panama-foreign/compare/foreign-memaccess*abi...JornVernee:panama-foreign:CFA*diff-2a53a53e1a977d7dbd5591e63c3b735d4db31744294625ae0d856d2a902d996eR414__;KyM!!ACWV5N9M2RV99hQ!JOdvTlR2GTsZLrsAPt2-Y0Dw9_fFcfeLjl4rGsnme208eXiTxO_-UOruCz8eei5qhe31l1JA0XufFWxeBoH8s3f-$ >> [2]: >> https://urldefense.com/v3/__https://github.com/openjdk/panama-foreign/compare/foreign-memaccess*abi...JornVernee:panama-foreign:CFA*diff-2a53a53e1a977d7dbd5591e63c3b735d4db31744294625ae0d856d2a902d996eR959__;KyM!!ACWV5N9M2RV99hQ!JOdvTlR2GTsZLrsAPt2-Y0Dw9_fFcfeLjl4rGsnme208eXiTxO_-UOruCz8eei5qhe31l1JA0XufFWxeBgtl3c6k$ >> [3]: >> https://urldefense.com/v3/__https://github.com/openjdk/panama-foreign/compare/foreign-memaccess*abi...JornVernee:panama-foreign:CFA*diff-2a53a53e1a977d7dbd5591e63c3b735d4db31744294625ae0d856d2a902d996eR947__;KyM!!ACWV5N9M2RV99hQ!JOdvTlR2GTsZLrsAPt2-Y0Dw9_fFcfeLjl4rGsnme208eXiTxO_-UOruCz8eei5qhe31l1JA0XufFWxeBuV70b4X$ >> From jorn.vernee at oracle.com Tue Mar 14 16:41:43 2023 From: jorn.vernee at oracle.com (Jorn Vernee) Date: Tue, 14 Mar 2023 17:41:43 +0100 Subject: Usage feedback rewriting jdk.internal.foreign.abi.BindingSpecializer In-Reply-To: <48f69654-cb98-7bbc-5c24-9ac9b824c5bb@oracle.com> References: <5f7e3cb5-4458-4aca-b420-e43e937cf9e2@oracle.com> <48f69654-cb98-7bbc-5c24-9ac9b824c5bb@oracle.com> Message-ID: <6d66fb85-0368-f30c-05f7-d3db89299a10@oracle.com> Sniped some parts, replies inline. >> 2. I'm not sure what the expected way of mapping a Class (or >> ClassDesc) into a TypeKind is. I couldn't find a factory for that in >> TypeKind. The code I have often needs to map Class objects into >> TypeKinds. I ended up writing a manual 'mapper' method instead [1]. > > There is TypeKind.fromDescriptor(s); you can go from either Class or > ClassDesc to descriptor fairly easily -- is that good enough? Do the > docs need to be updated to guide you towards that? See my other email to liang. TypeKind.fromDescriptor(s) works, but it feels a bit 'off' since I have a high-level type like Class, or ClassDesc, and have to go down to a low-level type, String, first. This seems different from the rest of the API which accepts ClassDesc everywhere. (I thought I might be missing a better way of doing the Class -> TypeKind translation) > >> 3. Sometimes I need to emit a constant zero for a particular type. If >> I know the type is e.g. long, I can use `cb.constantInstruction(0L)`, >> but sometimes the type is not statically known, and it would be nice >> if there was a CodeBuilder::constantZero(TypeKind) for that. I've >> implemented this manually as well [2] > > Not sure what you are asking for here.? There's no constant pool form > for "short constant".? There's bipush/sipush, though.? What bytecode > are you hoping to get out? I'd expect this: ??? cb.constantZero(ShortType) to result in iconst_0 being emitted. Where 'cb' is an instance of `CodeBuilder`. (except, 'ShortType' would not be statically know, but derived from a Class value that is fed in to the generation code). Jorn From brian.goetz at oracle.com Tue Mar 14 17:06:03 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 14 Mar 2023 13:06:03 -0400 Subject: Usage feedback rewriting jdk.internal.foreign.abi.BindingSpecializer In-Reply-To: <6d66fb85-0368-f30c-05f7-d3db89299a10@oracle.com> References: <5f7e3cb5-4458-4aca-b420-e43e937cf9e2@oracle.com> <48f69654-cb98-7bbc-5c24-9ac9b824c5bb@oracle.com> <6d66fb85-0368-f30c-05f7-d3db89299a10@oracle.com> Message-ID: <62ea199d-a2e5-8807-9513-0df16ec6b46d@oracle.com> >> Not sure what you are asking for here. There's no constant pool form >> for "short constant".? There's bipush/sipush, though.? What bytecode >> are you hoping to get out? > I'd expect this: > > ??? cb.constantZero(ShortType) > > to result in iconst_0 being emitted. Where 'cb' is an instance of > `CodeBuilder`. (except, 'ShortType' would not be statically know, but > derived from a Class value that is fed in to the generation code). Oh, I get it now -- you want the default field value for a given type kind.? Yes, that makes sense, something like ??? cb.loadDefault(typeKind) From jorn.vernee at oracle.com Tue Mar 14 17:37:15 2023 From: jorn.vernee at oracle.com (Jorn Vernee) Date: Tue, 14 Mar 2023 18:37:15 +0100 Subject: Usage feedback rewriting jdk.internal.foreign.abi.BindingSpecializer In-Reply-To: <62ea199d-a2e5-8807-9513-0df16ec6b46d@oracle.com> References: <5f7e3cb5-4458-4aca-b420-e43e937cf9e2@oracle.com> <48f69654-cb98-7bbc-5c24-9ac9b824c5bb@oracle.com> <6d66fb85-0368-f30c-05f7-d3db89299a10@oracle.com> <62ea199d-a2e5-8807-9513-0df16ec6b46d@oracle.com> Message-ID: Exactly. I liken it to the behavior produced by MethodHandles::zero [1], which "... returns the default value for that type ...". Jorn [1]: https://docs.oracle.com/en/java/javase/19/docs/api/java.base/java/lang/invoke/MethodHandles.html#zero(java.lang.Class) On 14/03/2023 18:06, Brian Goetz wrote: > >>> Not sure what you are asking for here. There's no constant pool form >>> for "short constant".? There's bipush/sipush, though.? What bytecode >>> are you hoping to get out? >> I'd expect this: >> >> ??? cb.constantZero(ShortType) >> >> to result in iconst_0 being emitted. Where 'cb' is an instance of >> `CodeBuilder`. (except, 'ShortType' would not be statically know, but >> derived from a Class value that is fed in to the generation code). > > Oh, I get it now -- you want the default field value for a given type > kind.? Yes, that makes sense, something like > > ??? cb.loadDefault(typeKind) > > From maurizio.cimadamore at oracle.com Wed Mar 15 18:48:44 2023 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 15 Mar 2023 18:48:44 +0000 Subject: Usage feedback rewriting jdk.internal.foreign.abi.BindingSpecializer In-Reply-To: <5f7e3cb5-4458-4aca-b420-e43e937cf9e2@oracle.com> References: <5f7e3cb5-4458-4aca-b420-e43e937cf9e2@oracle.com> Message-ID: Great work. IMHO, the nicest bits of the patch are at the very beginning and at the very end. At the very beginning, because, instead of using loose strings for a mix of method names, descriptors etc, we can now use appropriate abstractions (ClassDesc, MethodTypeDesc). At the very end because instead of rolling in our own "high-level" methods to generate the bytecode to emit e.g. a constant, we can just rely on the API to do that for us. Maurizio On 14/03/2023 15:31, Jorn Vernee wrote: > Hello, > > I've re-written jdk.internal.foreign.abi.BindingSpecializer to use the > new class file API. The patch can be found here: > https://github.com/openjdk/panama-foreign/compare/foreign-memaccess+abi...JornVernee:panama-foreign:CFA > > I did this mostly based on just the javadoc. > > Some things I noted while doing this: > > 1. The translation is really straightforward, and there were a lot of > places where simplification was possible. Such as, using > CodeBuilder::constantInstruction is much nicer than manually having to > find the right bytecode for the constant you have. Another nice thing > was the CodeBuilder::parameterSlot method. I previously had to > translate parameter indices to slots manually. Great work! > > 2. I'm not sure what the expected way of mapping a Class (or > ClassDesc) into a TypeKind is. I couldn't find a factory for that in > TypeKind. The code I have often needs to map Class objects into > TypeKinds. I ended up writing a manual 'mapper' method instead [1]. > > 3. Sometimes I need to emit a constant zero for a particular type. If > I know the type is e.g. long, I can use `cb.constantInstruction(0L)`, > but sometimes the type is not statically known, and it would be nice > if there was a CodeBuilder::constantZero(TypeKind) for that. I've > implemented this manually as well [2] > > 4. The same goes for `dup`. It would be nice if there was a > CodeBuilder::dup(TypeKind) that automatically selects either dup or > dup2. See [3] > > 5. I'll throw in a bike shed comment as well :) > CodeBuilder::labelBinding sounds a bit strange. From the name, I > initially wasn't sure whether this was returning an existing binding > for a previously bound label, or binding the label I passed it. (it > also doesn't help that the method doesn't have any javadoc at the > moment). I think the name "bindLabel" (or just "bind", as we have in > HotSpot) would be more intuitive. > > Cheers, > Jorn > > [1]: > https://github.com/openjdk/panama-foreign/compare/foreign-memaccess+abi...JornVernee:panama-foreign:CFA#diff-2a53a53e1a977d7dbd5591e63c3b735d4db31744294625ae0d856d2a902d996eR414 > [2]: > https://github.com/openjdk/panama-foreign/compare/foreign-memaccess+abi...JornVernee:panama-foreign:CFA#diff-2a53a53e1a977d7dbd5591e63c3b735d4db31744294625ae0d856d2a902d996eR959 > [3]: > https://github.com/openjdk/panama-foreign/compare/foreign-memaccess+abi...JornVernee:panama-foreign:CFA#diff-2a53a53e1a977d7dbd5591e63c3b735d4db31744294625ae0d856d2a902d996eR947 > From liangchenblue at gmail.com Wed Mar 15 19:17:55 2023 From: liangchenblue at gmail.com (-) Date: Wed, 15 Mar 2023 14:17:55 -0500 Subject: Other usages of Classfile API Message-ID: Since Classfile API is already seeing increased usage within JDK for its convenience, I wonder about the feasibility of a few potential usages: 1. In other JDK projects that change bytecode format, such as Valhalla. I guess the Classfile API within these projects will just adapt to the custom additions due to how it's distributed. Do we plan to switch the bytecode generation in Valhalla to Classfile too? It seems its constant API, which doesn't consider value or primitive classes, needs an update in the first place. 2. External tools, like asmtools. Don't know if Classfile API can simplify its logic or not, and whether it can use Classfile API in the first place. 3. Reflection API for Annotations. The reflection API currently pulls annotation attribute bytes directly from the class bytes in the VM, and Classfile API might be able to handle the bytes to annotation conversion better. From brian.goetz at oracle.com Wed Mar 15 20:02:44 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 15 Mar 2023 16:02:44 -0400 Subject: Other usages of Classfile API In-Reply-To: References: Message-ID: All of these are potential uses in the long run. On 3/15/2023 3:17 PM, - wrote: > Since Classfile API is already seeing increased usage within JDK for > its convenience, I wonder about the feasibility of a few potential > usages: > > 1. In other JDK projects that change bytecode format, such as > Valhalla. I guess the Classfile API within these projects will just > adapt to the custom additions due to how it's distributed. Do we plan > to switch the bytecode generation in Valhalla to Classfile too? It > seems its constant API, which doesn't consider value or primitive > classes, needs an update in the first place. This is really three questions ;) ?- Once the classfile API is in the JDK, Valhalla can extend it to support the new classfile flags, new bytecodes, and new descriptor strings.? Later, when Valhalla integrates, these changes to the API will come in with it. ?- The compiler and related tools currently have their own classfile library.? It would be nice, eventually, to replace this with the official classfile API.? We will likely start with `javap`, but the compiler is likely to be the last piece to change. ?- The constants API will need some changes to support value class descriptors, but this is waiting for the Valhalla translation scheme to be locked down.? (It is still possible we will find a way to do without Q descriptors.) > 2. External tools, like asmtools. Don't know if Classfile API can > simplify its logic or not, and whether it can use Classfile API in the > first place. Over time, we will migrate internal tools and JDK usages to the classfile API.? This is an area where the community can likely help,? if you're looking :) > 3. Reflection API for Annotations. The reflection API currently pulls > annotation attribute bytes directly from the class bytes in the VM, > and Classfile API might be able to handle the bytes to annotation > conversion better. I hadn't considered this one, but yes, this is a possible application. It will surely take a long time to fully migrate away from ASM, classreader, and other forms of bytecode analysis, but we'd like to get there. -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Thu Mar 16 16:04:39 2023 From: liangchenblue at gmail.com (-) Date: Thu, 16 Mar 2023 11:04:39 -0500 Subject: ClassHierarchyResolver using Reflection information Message-ID: For context, in 8294961 https://github.com/openjdk/jdk/pull/10991/files#r1133086265 I wondered about whether to use a hierarchy resolver for LambdaMetafactory, that I think resolution may encounter problems as the default resolver may not be able to resolve user-supplied interfaces. In addition, many of the class file generation usages I've seen include firing events via an event bus, calling patched external addon code (as seen in Minecraft modding), in which Reflection-based hierarchy resolution would work better than reading bytecode files. Thus, I've prepared a patch creating ClassHierarchyResolver https://github.com/openjdk/jdk/commit/0c888ba1e2953cf946012244446f4f8c05ba5d77 to ease up resolution with reflection, when a ClassLoader (for older APIs) or a MethodHandle.Lookup (for modern APIs) is available. Does this appear feasible? Chen From liangchenblue at gmail.com Fri Mar 17 06:24:38 2023 From: liangchenblue at gmail.com (-) Date: Fri, 17 Mar 2023 01:24:38 -0500 Subject: Reverse operation of ClassDesc::of Message-ID: While using ClassDesc, I find that it is somewhat hard to convert it to a binary or internal name as calling substring makes the code much more complex, especially when we know that the given ClassDesc is a class or interface. Thus, I propose an API serving as the inverse of ClassDesc::of(String), which takes a binary name from Class::getName and returns a ClassDesc of a class or interface. The new API would be parallel to existing Class::getName, and I am tempted to call it ClassDesc::getName. In addition, it may be of interest to expand the legal inputs of ClassDesc::of(String) to accept whatever Class::forName(String) does, notably array classes with names like [Ljava.lang.Object; or [I. Chen Liang From adam.sotona at oracle.com Fri Mar 17 12:11:51 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Fri, 17 Mar 2023 12:11:51 +0000 Subject: ClassHierarchyResolver using Reflection information In-Reply-To: References: Message-ID: I like your proposal and I would like to see it as a fallback solution for DEFAULT_CLASS_HIERARCHY_RESOLVER when the class is not available as resource stream. /** * Default singleton instance of {@linkplain ClassHierarchyResolver} * using {@link ClassLoader#getSystemResourceAsStream(String)} * as the {@code ClassStreamResolver} with fallback to * {@link ClassLoader.getSystemClassLoader()} class loading resolver. */ ClassHierarchyResolver DEFAULT_CLASS_HIERARCHY_RESOLVER = new ClassHierarchyImpl.CachedClassHierarchyResolver( new Function() { @Override public InputStream apply(ClassDesc classDesc) { return ClassLoader.getSystemResourceAsStream(Util.toInternalName(classDesc) + ".class"); } }).orElse(ClassHierarchyResolver.of(ClassLoader.getSystemClassLoader())); Thanks, Adam On 16.03.2023 17:05, "liangchenblue at gmail.com" wrote: For context, in 8294961 https://github.com/openjdk/jdk/pull/10991/files#r1133086265 I wondered about whether to use a hierarchy resolver for LambdaMetafactory, that I think resolution may encounter problems as the default resolver may not be able to resolve user-supplied interfaces. In addition, many of the class file generation usages I've seen include firing events via an event bus, calling patched external addon code (as seen in Minecraft modding), in which Reflection-based hierarchy resolution would work better than reading bytecode files. Thus, I've prepared a patch creating ClassHierarchyResolver https://github.com/openjdk/jdk/commit/0c888ba1e2953cf946012244446f4f8c05ba5d77 to ease up resolution with reflection, when a ClassLoader (for older APIs) or a MethodHandle.Lookup (for modern APIs) is available. Does this appear feasible? Chen -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Fri Mar 17 18:25:57 2023 From: liangchenblue at gmail.com (-) Date: Fri, 17 Mar 2023 13:25:57 -0500 Subject: ClassHierarchyResolver using Reflection information In-Reply-To: References: Message-ID: I've submitted a patch at https://github.com/openjdk/jdk/pull/13082 Yet with it comes 3 questions: 1. Should the resolver fail fast on IllegalAccessException from the lookup? This usually indicates the hierarchy resolver is set up improperly, and proceeding may simply yield verification errors in class loading that are hard to track. For bytecode-generating APIs, throwing access errors for the Lookup eagerly is also more preferable than later silent generation failure. The main concern for me to fail fast is that I don't understand how the Classfile API propagates resolver exceptions. If wrapping the IllegalAccessException in an IllegalAccessError is safe, then I may change it and add a test case with HashMap. 2. Whether the default resolver should be reading from jrt alone, reflection alone, or jrt then reflection. I personally believe reflection alone is more reliable, for classes may be redefined with instrumentation or jfr, which may not be reflected in the system resources. This idea came to me while I was working on jfr and instrumentation tests. Luckily, it seems they don't touch class hierarchy that the default resolver suffices. 3. In addition, I don't think chaining system class loader reflection after system resource retrieval is really meaningful: is there any case where reflection always works while the system resource retrieval always fails? Chen On Fri, Mar 17, 2023 at 7:11?AM Adam Sotona wrote: > > I like your proposal and I would like to see it as a fallback solution for DEFAULT_CLASS_HIERARCHY_RESOLVER when the class is not available as resource stream. > > > > /** > > * Default singleton instance of {@linkplain ClassHierarchyResolver} > > * using {@link ClassLoader#getSystemResourceAsStream(String)} > > * as the {@code ClassStreamResolver} with fallback to > > * {@link ClassLoader.getSystemClassLoader()} class loading resolver. > > */ > > ClassHierarchyResolver DEFAULT_CLASS_HIERARCHY_RESOLVER > > = new ClassHierarchyImpl.CachedClassHierarchyResolver( > > new Function() { > > @Override > > public InputStream apply(ClassDesc classDesc) { > > return ClassLoader.getSystemResourceAsStream(Util.toInternalName(classDesc) + ".class"); > > } > > }).orElse(ClassHierarchyResolver.of(ClassLoader.getSystemClassLoader())); > > > > Thanks, > > Adam > > > > On 16.03.2023 17:05, "liangchenblue at gmail.com" wrote: > > > > For context, in 8294961 > https://github.com/openjdk/jdk/pull/10991/files#r1133086265 I wondered > about whether to use a hierarchy resolver for LambdaMetafactory, that > I think resolution may encounter problems as the default resolver may > not be able to resolve user-supplied interfaces. > > In addition, many of the class file generation usages I've seen > include firing events via an event bus, calling patched external addon > code (as seen in Minecraft modding), in which Reflection-based > hierarchy resolution would work better than reading bytecode files. > > Thus, I've prepared a patch creating ClassHierarchyResolver > https://github.com/openjdk/jdk/commit/0c888ba1e2953cf946012244446f4f8c05ba5d77 > to ease up resolution with reflection, when a ClassLoader (for older > APIs) or a MethodHandle.Lookup (for modern APIs) is available. > > Does this appear feasible? > > Chen From brian.goetz at oracle.com Fri Mar 17 18:34:31 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 17 Mar 2023 14:34:31 -0400 Subject: ClassHierarchyResolver using Reflection information In-Reply-To: References: Message-ID: <0883d896-3723-f15b-2573-6a7316feebcf@oracle.com> I'm not so comfortable adding this to DEFAULT_CHA.? In addition to adding another degree of environmental dependence, and the issues of additional exceptions you raise, I suspect it may also have a performance cost.? I would like for programmers to opt into which CHA they use, and the default should be the most minimal.? This means having a menu of CHAs to choose from, rather than guessing what the user wants. On 3/17/2023 2:25 PM, - wrote: > I've submitted a patch at https://github.com/openjdk/jdk/pull/13082 > Yet with it comes 3 questions: > > 1. Should the resolver fail fast on IllegalAccessException from the > lookup? This usually indicates the hierarchy resolver is set up > improperly, and proceeding may simply yield verification errors in > class loading that are hard to track. For bytecode-generating APIs, > throwing access errors for the Lookup eagerly is also more preferable > than later silent generation failure. > > The main concern for me to fail fast is that I don't understand how > the Classfile API propagates resolver exceptions. If wrapping the > IllegalAccessException in an IllegalAccessError is safe, then I may > change it and add a test case with HashMap. > > 2. Whether the default resolver should be reading from jrt alone, > reflection alone, or jrt then reflection. I personally believe > reflection alone is more reliable, for classes may be redefined with > instrumentation or jfr, which may not be reflected in the system > resources. > > This idea came to me while I was working on jfr and instrumentation > tests. Luckily, it seems they don't touch class hierarchy that the > default resolver suffices. > > 3. In addition, I don't think chaining system class loader reflection > after system resource retrieval is really meaningful: is there any > case where reflection always works while the system resource retrieval > always fails? > > Chen > > On Fri, Mar 17, 2023 at 7:11?AM Adam Sotona wrote: >> I like your proposal and I would like to see it as a fallback solution for DEFAULT_CLASS_HIERARCHY_RESOLVER when the class is not available as resource stream. >> >> >> >> /** >> >> * Default singleton instance of {@linkplain ClassHierarchyResolver} >> >> * using {@link ClassLoader#getSystemResourceAsStream(String)} >> >> * as the {@code ClassStreamResolver} with fallback to >> >> * {@link ClassLoader.getSystemClassLoader()} class loading resolver. >> >> */ >> >> ClassHierarchyResolver DEFAULT_CLASS_HIERARCHY_RESOLVER >> >> = new ClassHierarchyImpl.CachedClassHierarchyResolver( >> >> new Function() { >> >> @Override >> >> public InputStream apply(ClassDesc classDesc) { >> >> return ClassLoader.getSystemResourceAsStream(Util.toInternalName(classDesc) + ".class"); >> >> } >> >> }).orElse(ClassHierarchyResolver.of(ClassLoader.getSystemClassLoader())); >> >> >> >> Thanks, >> >> Adam >> >> >> >> On 16.03.2023 17:05, "liangchenblue at gmail.com" wrote: >> >> >> >> For context, in 8294961 >> https://github.com/openjdk/jdk/pull/10991/files#r1133086265 I wondered >> about whether to use a hierarchy resolver for LambdaMetafactory, that >> I think resolution may encounter problems as the default resolver may >> not be able to resolve user-supplied interfaces. >> >> In addition, many of the class file generation usages I've seen >> include firing events via an event bus, calling patched external addon >> code (as seen in Minecraft modding), in which Reflection-based >> hierarchy resolution would work better than reading bytecode files. >> >> Thus, I've prepared a patch creating ClassHierarchyResolver >> https://github.com/openjdk/jdk/commit/0c888ba1e2953cf946012244446f4f8c05ba5d77 >> to ease up resolution with reflection, when a ClassLoader (for older >> APIs) or a MethodHandle.Lookup (for modern APIs) is available. >> >> Does this appear feasible? >> >> Chen From michael.van.acken at gmail.com Sat Mar 18 04:06:18 2023 From: michael.van.acken at gmail.com (Michael van Acken) Date: Sat, 18 Mar 2023 05:06:18 +0100 Subject: ClassHierarchyResolver using Reflection information In-Reply-To: <0883d896-3723-f15b-2573-6a7316feebcf@oracle.com> References: <0883d896-3723-f15b-2573-6a7316feebcf@oracle.com> Message-ID: Am Fr., 17. M?rz 2023 um 19:34 Uhr schrieb Brian Goetz < brian.goetz at oracle.com>: > I'm not so comfortable adding this to DEFAULT_CHA. In addition to > adding another degree of environmental dependence, and the issues of > additional exceptions you raise, I suspect it may also have a > performance cost. I would like for programmers to opt into which CHA > they use, and the default should be the most minimal. This means having > a menu of CHAs to choose from, rather than guessing what the user wants. > The default classHierarchyResolver has an opaque (to me) error mode that tripped me up. Similar to my experience with ASM some years ago, it works fine for a lot (most?) of code, but then some input causes it to silently produce a stack map the verifier does not agree with. For me, this happens usually quite late in the game and at first is completely baffling. Back in July it was mostly luck and half buried memories that sent me into the direction of CHR as the solution to the verifier error. If the default CHR had raised an exception at the point where information was missing, the path to the fix (preloading the CHR with information about future classes) would have been clearer. --mva > > On 3/17/2023 2:25 PM, - wrote: > > I've submitted a patch at https://github.com/openjdk/jdk/pull/13082 > > Yet with it comes 3 questions: > > > > 1. Should the resolver fail fast on IllegalAccessException from the > > lookup? This usually indicates the hierarchy resolver is set up > > improperly, and proceeding may simply yield verification errors in > > class loading that are hard to track. For bytecode-generating APIs, > > throwing access errors for the Lookup eagerly is also more preferable > > than later silent generation failure. > > > > The main concern for me to fail fast is that I don't understand how > > the Classfile API propagates resolver exceptions. If wrapping the > > IllegalAccessException in an IllegalAccessError is safe, then I may > > change it and add a test case with HashMap. > > > > 2. Whether the default resolver should be reading from jrt alone, > > reflection alone, or jrt then reflection. I personally believe > > reflection alone is more reliable, for classes may be redefined with > > instrumentation or jfr, which may not be reflected in the system > > resources. > > > > This idea came to me while I was working on jfr and instrumentation > > tests. Luckily, it seems they don't touch class hierarchy that the > > default resolver suffices. > > > > 3. In addition, I don't think chaining system class loader reflection > > after system resource retrieval is really meaningful: is there any > > case where reflection always works while the system resource retrieval > > always fails? > > > > Chen > > > > On Fri, Mar 17, 2023 at 7:11?AM Adam Sotona > wrote: > >> I like your proposal and I would like to see it as a fallback solution > for DEFAULT_CLASS_HIERARCHY_RESOLVER when the class is not available as > resource stream. > >> > >> > >> > >> /** > >> > >> * Default singleton instance of {@linkplain > ClassHierarchyResolver} > >> > >> * using {@link ClassLoader#getSystemResourceAsStream(String)} > >> > >> * as the {@code ClassStreamResolver} with fallback to > >> > >> * {@link ClassLoader.getSystemClassLoader()} class loading > resolver. > >> > >> */ > >> > >> ClassHierarchyResolver DEFAULT_CLASS_HIERARCHY_RESOLVER > >> > >> = new ClassHierarchyImpl.CachedClassHierarchyResolver( > >> > >> new Function() { > >> > >> @Override > >> > >> public InputStream apply(ClassDesc classDesc) { > >> > >> return > ClassLoader.getSystemResourceAsStream(Util.toInternalName(classDesc) + > ".class"); > >> > >> } > >> > >> > }).orElse(ClassHierarchyResolver.of(ClassLoader.getSystemClassLoader())); > >> > >> > >> > >> Thanks, > >> > >> Adam > >> > >> > >> > >> On 16.03.2023 17:05, "liangchenblue at gmail.com" > wrote: > >> > >> > >> > >> For context, in 8294961 > >> https://github.com/openjdk/jdk/pull/10991/files#r1133086265 I wondered > >> about whether to use a hierarchy resolver for LambdaMetafactory, that > >> I think resolution may encounter problems as the default resolver may > >> not be able to resolve user-supplied interfaces. > >> > >> In addition, many of the class file generation usages I've seen > >> include firing events via an event bus, calling patched external addon > >> code (as seen in Minecraft modding), in which Reflection-based > >> hierarchy resolution would work better than reading bytecode files. > >> > >> Thus, I've prepared a patch creating ClassHierarchyResolver > >> > https://github.com/openjdk/jdk/commit/0c888ba1e2953cf946012244446f4f8c05ba5d77 > >> to ease up resolution with reflection, when a ClassLoader (for older > >> APIs) or a MethodHandle.Lookup (for modern APIs) is available. > >> > >> Does this appear feasible? > >> > >> Chen > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zjx001202 at gmail.com Sat Mar 18 07:41:13 2023 From: zjx001202 at gmail.com (Glavo) Date: Sat, 18 Mar 2023 15:41:13 +0800 Subject: Track Classfile API changes on the classfile-api-dev mailing list Message-ID: Hi everyone, I extracted Classfile API from JDK, ported it back to Java 17, and published it to Maven Central as a standalone library. Here is its repository: https://github.com/Glavo/classfile (I've posted the above on classfile-api-dev, but I don't see it in the mailing list archives. I don't know if it was sent successfully, so I'll repeat it here) To maintain it, I am very concerned about changes to the Classfile API and sync the changes to the above repository. Unfortunately, the Classfile API does not appear to have a specific JBS label, and I cannot track related PRs from the classfile-api-dev, so I have to check the history of the Classfile API source folder at regular intervals. In order to facilitate tracking, I hope that a label will be assigned to the Classfile API and send relevant PR notification emails to the classfile-api-dev mailing list. Is that okay? Glavo -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Mon Mar 20 09:50:07 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Mon, 20 Mar 2023 09:50:07 +0000 Subject: [External] : Re: ClassHierarchyResolver using Reflection information In-Reply-To: References: <0883d896-3723-f15b-2573-6a7316feebcf@oracle.com> Message-ID: I agree with your concerns. The benevolence of the CHR was decision of an early development stage, it supposed to be temporary and now it is a clear bug. So from my perspective - yes, fail fast and fail when any required class is not resolved. Classfile API as well as ASM are designed to handle majority of cases without user knowledge of CHR or class loading details. I guess only few ASM users know how to pass custom class loader for correct CHR for stack maps generation. I?ll try to summarize benefits and negatives of my proposal to extend default CHR to fall back to a class loading CHR. Benefits of CHR by classfile parsing: * Performance ? ?skip-parsing? of constant pool + classfile header is in order of magnitude faster than class loading * Simplicity/Stability ? it is not necessary to provide all dependencies to identify if the type is an interface or what is its super. It is not necessary to have complete classpath with implementation of some API to generate code using the API. Negatives of CHR by classfile parsing: * Availability - a classfile may not be available or accessible as a resource for parsing * Different results ? a classfile may be modified when loaded, however the modification would have to change its super class or its status of being an interface to make any impact. By adding a fallback to class loading CHR we will eliminate the ?Availability? negative, with no performance loose. I think that the default should be rock-solid, because 99% of users will use the default. Thanks, Adam BTW: Maybe even more beneficial would be an ability to question already loaded classes first (without loading them), parse them as resource if not already loaded and fallback to load them if not available as resource. On 18.03.2023 5:06, "Michael van Acken" wrote: Am Fr., 17. M?rz 2023 um 19:34 Uhr schrieb Brian Goetz >: I'm not so comfortable adding this to DEFAULT_CHA. In addition to adding another degree of environmental dependence, and the issues of additional exceptions you raise, I suspect it may also have a performance cost. I would like for programmers to opt into which CHA they use, and the default should be the most minimal. This means having a menu of CHAs to choose from, rather than guessing what the user wants. The default classHierarchyResolver has an opaque (to me) error mode that tripped me up. Similar to my experience with ASM some years ago, it works fine for a lot (most?) of code, but then some input causes it to silently produce a stack map the verifier does not agree with. For me, this happens usually quite late in the game and at first is completely baffling. Back in July it was mostly luck and half buried memories that sent me into the direction of CHR as the solution to the verifier error. If the default CHR had raised an exception at the point where information was missing, the path to the fix (preloading the CHR with information about future classes) would have been clearer. --mva On 3/17/2023 2:25 PM, - wrote: > I've submitted a patch at https://github.com/openjdk/jdk/pull/13082 > Yet with it comes 3 questions: > > 1. Should the resolver fail fast on IllegalAccessException from the > lookup? This usually indicates the hierarchy resolver is set up > improperly, and proceeding may simply yield verification errors in > class loading that are hard to track. For bytecode-generating APIs, > throwing access errors for the Lookup eagerly is also more preferable > than later silent generation failure. > > The main concern for me to fail fast is that I don't understand how > the Classfile API propagates resolver exceptions. If wrapping the > IllegalAccessException in an IllegalAccessError is safe, then I may > change it and add a test case with HashMap. > > 2. Whether the default resolver should be reading from jrt alone, > reflection alone, or jrt then reflection. I personally believe > reflection alone is more reliable, for classes may be redefined with > instrumentation or jfr, which may not be reflected in the system > resources. > > This idea came to me while I was working on jfr and instrumentation > tests. Luckily, it seems they don't touch class hierarchy that the > default resolver suffices. > > 3. In addition, I don't think chaining system class loader reflection > after system resource retrieval is really meaningful: is there any > case where reflection always works while the system resource retrieval > always fails? > > Chen > > On Fri, Mar 17, 2023 at 7:11?AM Adam Sotona > wrote: >> I like your proposal and I would like to see it as a fallback solution for DEFAULT_CLASS_HIERARCHY_RESOLVER when the class is not available as resource stream. >> >> >> >> /** >> >> * Default singleton instance of {@linkplain ClassHierarchyResolver} >> >> * using {@link ClassLoader#getSystemResourceAsStream(String)} >> >> * as the {@code ClassStreamResolver} with fallback to >> >> * {@link ClassLoader.getSystemClassLoader()} class loading resolver. >> >> */ >> >> ClassHierarchyResolver DEFAULT_CLASS_HIERARCHY_RESOLVER >> >> = new ClassHierarchyImpl.CachedClassHierarchyResolver( >> >> new Function() { >> >> @Override >> >> public InputStream apply(ClassDesc classDesc) { >> >> return ClassLoader.getSystemResourceAsStream(Util.toInternalName(classDesc) + ".class"); >> >> } >> >> }).orElse(ClassHierarchyResolver.of(ClassLoader.getSystemClassLoader())); >> >> >> >> Thanks, >> >> Adam >> >> >> >> On 16.03.2023 17:05, "liangchenblue at gmail.com" > wrote: >> >> >> >> For context, in 8294961 >> https://github.com/openjdk/jdk/pull/10991/files#r1133086265 I wondered >> about whether to use a hierarchy resolver for LambdaMetafactory, that >> I think resolution may encounter problems as the default resolver may >> not be able to resolve user-supplied interfaces. >> >> In addition, many of the class file generation usages I've seen >> include firing events via an event bus, calling patched external addon >> code (as seen in Minecraft modding), in which Reflection-based >> hierarchy resolution would work better than reading bytecode files. >> >> Thus, I've prepared a patch creating ClassHierarchyResolver >> https://github.com/openjdk/jdk/commit/0c888ba1e2953cf946012244446f4f8c05ba5d77 >> to ease up resolution with reflection, when a ClassLoader (for older >> APIs) or a MethodHandle.Lookup (for modern APIs) is available. >> >> Does this appear feasible? >> >> Chen -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Mon Mar 20 13:25:30 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 20 Mar 2023 09:25:30 -0400 Subject: [External] : Re: ClassHierarchyResolver using Reflection information In-Reply-To: References: <0883d896-3723-f15b-2573-6a7316feebcf@oracle.com> Message-ID: <248a1a23-4bcd-076b-5124-5e418b3292f2@oracle.com> If the user provides no resolver at all, what is the new behavior?? Do we just assume Object is the common supertype for any pair of classes? On 3/20/2023 7:17 AM, Adam Sotona wrote: > > FYI: I?ve filled a new bug and proposed patch: *8304502: Classfile API > class hierarchy makes assumptions when class is not resolved?#13099 > * > > Thanks, > > Adam > > On 20.03.2023 10:50, "Adam Sotona" wrote: > > I agree with your concerns. The benevolence of the CHR was decision of > an early development stage, it supposed to be temporary and now it is > a clear bug. > > So from my perspective - ?yes, fail fast and fail when any required > class is not resolved. > > Classfile API as well as ASM are designed to handle majority of cases > without user knowledge of CHR or class loading details. I guess only > few ASM users know how to pass custom class loader for correct CHR for > stack maps generation. I?ll try to summarize benefits and negatives of > my proposal to extend default CHR to fall back to a class loading CHR. > > Benefits of CHR by classfile parsing: > > -Performance ? ?skip-parsing? of constant pool + classfile header is > in order of magnitude faster than class loading > > -Simplicity/Stability ? it is not necessary to provide all > dependencies to identify if the type is an interface or what is its > super. It is not necessary to have complete classpath with > implementation of some API to generate code using the API. > > Negatives of CHR by classfile parsing: > > -Availability ?- a classfile may not be available or accessible as a > resource for parsing > > -Different results ? a classfile may be modified when loaded, however > the modification would have to change its super class or its status of > being an interface to make any impact. > > By adding a fallback to class loading CHR we will eliminate the > ?Availability? negative, with no performance loose. > > I think that the default should be rock-solid, because 99% of users > will use the default. > > Thanks, > > Adam > > BTW: Maybe even more beneficial would be an ability to question > already loaded classes first (without loading them), parse them as > resource if not already loaded and fallback to load them if not > available as resource. > > On 18.03.2023 5:06, "Michael van Acken" > wrote: > > Am Fr., 17. M?rz 2023 um 19:34?Uhr schrieb Brian Goetz > : > > I'm not so comfortable adding this to DEFAULT_CHA.? In addition to > adding another degree of environmental dependence, and the issues of > additional exceptions you raise, I suspect it may also have a > performance cost.? I would like for programmers to opt into which CHA > they use, and the default should be the most minimal. This means > having > a menu of CHAs to choose from, rather than guessing what the user > wants. > > The default classHierarchyResolver has an opaque (to me) error > mode that tripped me up.? Similar to my experience with ASM some > years ago, it works fine for a lot (most?) of code, but then some > input causes it to silently produce a stack map the verifier does > not agree with.? For me, this happens usually quite late in the > game and at first is completely baffling.? Back in July it was > mostly luck and half buried memories that sent me into the > direction of CHR as the solution to the verifier error.? If the > default CHR had raised an exception at the point where > information was missing, the path to the fix (preloading the CHR > with information about future classes) would have been clearer. > > --mva > > > On 3/17/2023 2:25 PM, - wrote: > > I've submitted a patch at > https://github.com/openjdk/jdk/pull/13082 > > > Yet with it comes 3 questions: > > > > 1. Should the resolver fail fast on IllegalAccessException from the > > lookup? This usually indicates the hierarchy resolver is set up > > improperly, and proceeding may simply yield verification errors in > > class loading that are hard to track. For bytecode-generating APIs, > > throwing access errors for the Lookup eagerly is also more > preferable > > than later silent generation failure. > > > > The main concern for me to fail fast is that I don't understand how > > the Classfile API propagates resolver exceptions. If wrapping the > > IllegalAccessException in an IllegalAccessError is safe, then I may > > change it and add a test case with HashMap. > > > > 2. Whether the default resolver should be reading from jrt alone, > > reflection alone, or jrt then reflection. I personally believe > > reflection alone is more reliable, for classes may be redefined with > > instrumentation or jfr, which may not be reflected in the system > > resources. > > > > This idea came to me while I was working on jfr and instrumentation > > tests. Luckily, it seems they don't touch class hierarchy that the > > default resolver suffices. > > > > 3. In addition, I don't think chaining system class loader > reflection > > after system resource retrieval is really meaningful: is there any > > case where reflection always works while the system resource > retrieval > > always fails? > > > > Chen > > > > On Fri, Mar 17, 2023 at 7:11?AM Adam Sotona > wrote: > >> I like your proposal and I would like to see it as a fallback > solution for DEFAULT_CLASS_HIERARCHY_RESOLVER when the class is > not available as resource stream. > >> > >> > >> > >>? ? ?/** > >> > >>? ? ? ?* Default singleton instance of {@linkplain > ClassHierarchyResolver} > >> > >>? ? ? ?* using {@link ClassLoader#getSystemResourceAsStream(String)} > >> > >>? ? ? ?* as the {@code ClassStreamResolver} with fallback to > >> > >>? ? ? ?* {@link ClassLoader.getSystemClassLoader()} class > loading resolver. > >> > >>? ? ? ?*/ > >> > >>? ? ? ClassHierarchyResolver DEFAULT_CLASS_HIERARCHY_RESOLVER > >> > >>? ? ? ? ? ? ? = new ClassHierarchyImpl.CachedClassHierarchyResolver( > >> > >>? ? ? ? ? ? ? new Function() { > >> > >>? ? ? ? ? ? ? ? ? @Override > >> > >>? ? ? ? ? ? ? ? ? public InputStream apply(ClassDesc classDesc) { > >> > >>? ? ? ? ? ? ? ? ? ? ? return > ClassLoader.getSystemResourceAsStream(Util.toInternalName(classDesc) > + ".class"); > >> > >>? ? ? ? ? ? ? ? ? } > >> > >> > }).orElse(ClassHierarchyResolver.of(ClassLoader.getSystemClassLoader())); > >> > >> > >> > >> Thanks, > >> > >> Adam > >> > >> > >> > >> On 16.03.2023 17:05, "liangchenblue at gmail.com" > wrote: > >> > >> > >> > >> For context, in 8294961 > >> https://github.com/openjdk/jdk/pull/10991/files#r1133086265 > > I wondered > >> about whether to use a hierarchy resolver for > LambdaMetafactory, that > >> I think resolution may encounter problems as the default > resolver may > >> not be able to resolve user-supplied interfaces. > >> > >> In addition, many of the class file generation usages I've seen > >> include firing events via an event bus, calling patched > external addon > >> code (as seen in Minecraft modding), in which Reflection-based > >> hierarchy resolution would work better than reading bytecode files. > >> > >> Thus, I've prepared a patch creating ClassHierarchyResolver > >> > https://github.com/openjdk/jdk/commit/0c888ba1e2953cf946012244446f4f8c05ba5d77 > > >> to ease up resolution with reflection, when a ClassLoader (for > older > >> APIs) or a MethodHandle.Lookup (for modern APIs) is available. > >> > >> Does this appear feasible? > >> > >> Chen > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Mon Mar 20 13:56:01 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Mon, 20 Mar 2023 13:56:01 +0000 Subject: [External] : Re: ClassHierarchyResolver using Reflection information In-Reply-To: <248a1a23-4bcd-076b-5124-5e418b3292f2@oracle.com> References: <0883d896-3723-f15b-2573-6a7316feebcf@oracle.com> <248a1a23-4bcd-076b-5124-5e418b3292f2@oracle.com> Message-ID: New behavior is ?no assumptions? and throw IllegalArgumentException if information about type necessary to build stack map (or verify stack map) is not obtained from the resolver (or better say chain of resolvers). If user provides no resolver ? the default is used. The way how default resolver gets the information from system classloader is subject of discussion below (resource parsing and fallback to class loading). Custom resolver can be constructed as a fallback chain from all provided resolver types (including the default resolver instance). On 20.03.2023 14:25, "Brian Goetz" wrote: If the user provides no resolver at all, what is the new behavior? Do we just assume Object is the common supertype for any pair of classes? On 3/20/2023 7:17 AM, Adam Sotona wrote: FYI: I?ve filled a new bug and proposed patch: 8304502: Classfile API class hierarchy makes assumptions when class is not resolved #13099 Thanks, Adam On 20.03.2023 10:50, "Adam Sotona" wrote: I agree with your concerns. The benevolence of the CHR was decision of an early development stage, it supposed to be temporary and now it is a clear bug. So from my perspective - yes, fail fast and fail when any required class is not resolved. Classfile API as well as ASM are designed to handle majority of cases without user knowledge of CHR or class loading details. I guess only few ASM users know how to pass custom class loader for correct CHR for stack maps generation. I?ll try to summarize benefits and negatives of my proposal to extend default CHR to fall back to a class loading CHR. Benefits of CHR by classfile parsing: 1. Performance ? ?skip-parsing? of constant pool + classfile header is in order of magnitude faster than class loading 2. Simplicity/Stability ? it is not necessary to provide all dependencies to identify if the type is an interface or what is its super. It is not necessary to have complete classpath with implementation of some API to generate code using the API. Negatives of CHR by classfile parsing: 3. Availability - a classfile may not be available or accessible as a resource for parsing 4. Different results ? a classfile may be modified when loaded, however the modification would have to change its super class or its status of being an interface to make any impact. By adding a fallback to class loading CHR we will eliminate the ?Availability? negative, with no performance loose. I think that the default should be rock-solid, because 99% of users will use the default. Thanks, Adam BTW: Maybe even more beneficial would be an ability to question already loaded classes first (without loading them), parse them as resource if not already loaded and fallback to load them if not available as resource. On 18.03.2023 5:06, "Michael van Acken" wrote: Am Fr., 17. M?rz 2023 um 19:34 Uhr schrieb Brian Goetz >: I'm not so comfortable adding this to DEFAULT_CHA. In addition to adding another degree of environmental dependence, and the issues of additional exceptions you raise, I suspect it may also have a performance cost. I would like for programmers to opt into which CHA they use, and the default should be the most minimal. This means having a menu of CHAs to choose from, rather than guessing what the user wants. The default classHierarchyResolver has an opaque (to me) error mode that tripped me up. Similar to my experience with ASM some years ago, it works fine for a lot (most?) of code, but then some input causes it to silently produce a stack map the verifier does not agree with. For me, this happens usually quite late in the game and at first is completely baffling. Back in July it was mostly luck and half buried memories that sent me into the direction of CHR as the solution to the verifier error. If the default CHR had raised an exception at the point where information was missing, the path to the fix (preloading the CHR with information about future classes) would have been clearer. --mva On 3/17/2023 2:25 PM, - wrote: > I've submitted a patch at https://github.com/openjdk/jdk/pull/13082 > Yet with it comes 3 questions: > > 1. Should the resolver fail fast on IllegalAccessException from the > lookup? This usually indicates the hierarchy resolver is set up > improperly, and proceeding may simply yield verification errors in > class loading that are hard to track. For bytecode-generating APIs, > throwing access errors for the Lookup eagerly is also more preferable > than later silent generation failure. > > The main concern for me to fail fast is that I don't understand how > the Classfile API propagates resolver exceptions. If wrapping the > IllegalAccessException in an IllegalAccessError is safe, then I may > change it and add a test case with HashMap. > > 2. Whether the default resolver should be reading from jrt alone, > reflection alone, or jrt then reflection. I personally believe > reflection alone is more reliable, for classes may be redefined with > instrumentation or jfr, which may not be reflected in the system > resources. > > This idea came to me while I was working on jfr and instrumentation > tests. Luckily, it seems they don't touch class hierarchy that the > default resolver suffices. > > 3. In addition, I don't think chaining system class loader reflection > after system resource retrieval is really meaningful: is there any > case where reflection always works while the system resource retrieval > always fails? > > Chen > > On Fri, Mar 17, 2023 at 7:11?AM Adam Sotona > wrote: >> I like your proposal and I would like to see it as a fallback solution for DEFAULT_CLASS_HIERARCHY_RESOLVER when the class is not available as resource stream. >> >> >> >> /** >> >> * Default singleton instance of {@linkplain ClassHierarchyResolver} >> >> * using {@link ClassLoader#getSystemResourceAsStream(String)} >> >> * as the {@code ClassStreamResolver} with fallback to >> >> * {@link ClassLoader.getSystemClassLoader()} class loading resolver. >> >> */ >> >> ClassHierarchyResolver DEFAULT_CLASS_HIERARCHY_RESOLVER >> >> = new ClassHierarchyImpl.CachedClassHierarchyResolver( >> >> new Function() { >> >> @Override >> >> public InputStream apply(ClassDesc classDesc) { >> >> return ClassLoader.getSystemResourceAsStream(Util.toInternalName(classDesc) + ".class"); >> >> } >> >> }).orElse(ClassHierarchyResolver.of(ClassLoader.getSystemClassLoader())); >> >> >> >> Thanks, >> >> Adam >> >> >> >> On 16.03.2023 17:05, "liangchenblue at gmail.com" > wrote: >> >> >> >> For context, in 8294961 >> https://github.com/openjdk/jdk/pull/10991/files#r1133086265 I wondered >> about whether to use a hierarchy resolver for LambdaMetafactory, that >> I think resolution may encounter problems as the default resolver may >> not be able to resolve user-supplied interfaces. >> >> In addition, many of the class file generation usages I've seen >> include firing events via an event bus, calling patched external addon >> code (as seen in Minecraft modding), in which Reflection-based >> hierarchy resolution would work better than reading bytecode files. >> >> Thus, I've prepared a patch creating ClassHierarchyResolver >> https://github.com/openjdk/jdk/commit/0c888ba1e2953cf946012244446f4f8c05ba5d77 >> to ease up resolution with reflection, when a ClassLoader (for older >> APIs) or a MethodHandle.Lookup (for modern APIs) is available. >> >> Does this appear feasible? >> >> Chen -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Mon Mar 20 14:00:07 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Mon, 20 Mar 2023 14:00:07 +0000 Subject: [External] : Re: ClassHierarchyResolver using Reflection information In-Reply-To: <248a1a23-4bcd-076b-5124-5e418b3292f2@oracle.com> References: <0883d896-3723-f15b-2573-6a7316feebcf@oracle.com> <248a1a23-4bcd-076b-5124-5e418b3292f2@oracle.com> Message-ID: New behavior is ?no assumptions? and throw IllegalArgumentException if information about type necessary to build stack map (or verify stack map) is not obtained from the resolver (or better say chain of resolvers). If user provides no resolver ? the default is used. The way how default resolver gets the information from system classloader is subject of discussion below (resource parsing and fallback to class loading). Custom resolver can be constructed as a fallback chain from all provided resolver types (including the default resolver instance). On 20.03.2023 14:25, "Brian Goetz" wrote: If the user provides no resolver at all, what is the new behavior? Do we just assume Object is the common supertype for any pair of classes? -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Mon Mar 20 15:47:35 2023 From: liangchenblue at gmail.com (-) Date: Mon, 20 Mar 2023 10:47:35 -0500 Subject: [External] : Re: ClassHierarchyResolver using Reflection information In-Reply-To: References: <0883d896-3723-f15b-2573-6a7316feebcf@oracle.com> <248a1a23-4bcd-076b-5124-5e418b3292f2@oracle.com> Message-ID: If the choice of default resolver is still subject to debate, I can remove the delegation to system class loader based resolver in https://github.com/openjdk/jdk/pull/13082, and we can change it in another patch once we reach a consensus. The addition of lookup and class loader based resolvers can be used by users who explicitly specify a resolver in the options. On Mon, Mar 20, 2023 at 9:00?AM Adam Sotona wrote: > > New behavior is ?no assumptions? and throw IllegalArgumentException if information about type necessary to build stack map (or verify stack map) is not obtained from the resolver (or better say chain of resolvers). > > > > If user provides no resolver ? the default is used. > > > > The way how default resolver gets the information from system classloader is subject of discussion below (resource parsing and fallback to class loading). > > Custom resolver can be constructed as a fallback chain from all provided resolver types (including the default resolver instance). > > > > > > > > On 20.03.2023 14:25, "Brian Goetz" wrote: > > > If the user provides no resolver at all, what is the new behavior? Do we just assume Object is the common supertype for any pair of classes? > > From brian.goetz at oracle.com Mon Mar 20 16:51:14 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 20 Mar 2023 12:51:14 -0400 Subject: [External] : Re: ClassHierarchyResolver using Reflection information In-Reply-To: References: <0883d896-3723-f15b-2573-6a7316feebcf@oracle.com> <248a1a23-4bcd-076b-5124-5e418b3292f2@oracle.com> Message-ID: <17d29737-43f9-2713-da67-459ab59f1cc8@oracle.com> After looking over this corner of the API, I think there's a little more design work that needs to go into it.? So let's step back from the details of "what's the default" until we have a chance to look a little more holistically at this part of the API.? I'll try to write more about it soon. On 3/20/2023 11:47 AM, - wrote: > If the choice of default resolver is still subject to debate, I can > remove the delegation to system class loader based resolver in > https://urldefense.com/v3/__https://github.com/openjdk/jdk/pull/13082__;!!ACWV5N9M2RV99hQ!K7UWlrg2NHotSKOzMvmfnenKZ1dDuOZ9zXcQvlKOdB21VvwTRIDuI2NyRNK6Hq844XIkt_6Q-qFt5DsFDUzgKJ31$ , and we can change it in > another patch once we reach a consensus. The addition of lookup and > class loader based resolvers can be used by users who explicitly > specify a resolver in the options. > > On Mon, Mar 20, 2023 at 9:00?AM Adam Sotona wrote: >> New behavior is ?no assumptions? and throw IllegalArgumentException if information about type necessary to build stack map (or verify stack map) is not obtained from the resolver (or better say chain of resolvers). >> >> >> >> If user provides no resolver ? the default is used. >> >> >> >> The way how default resolver gets the information from system classloader is subject of discussion below (resource parsing and fallback to class loading). >> >> Custom resolver can be constructed as a fallback chain from all provided resolver types (including the default resolver instance). >> >> >> >> >> >> >> >> On 20.03.2023 14:25, "Brian Goetz" wrote: >> >> >> If the user provides no resolver at all, what is the new behavior? Do we just assume Object is the common supertype for any pair of classes? >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Mon Mar 20 18:19:45 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 20 Mar 2023 14:19:45 -0400 Subject: [External] : Re: ClassHierarchyResolver using Reflection information In-Reply-To: <17d29737-43f9-2713-da67-459ab59f1cc8@oracle.com> References: <0883d896-3723-f15b-2573-6a7316feebcf@oracle.com> <248a1a23-4bcd-076b-5124-5e418b3292f2@oracle.com> <17d29737-43f9-2713-da67-459ab59f1cc8@oracle.com> Message-ID: The CHR interface is straightforward enough -- given a symbolic reference to a class, provide two bits of information: superclass and is-interface. We have at least four sensible implementations, which can be combined: ?- do nothing ?- work from an explicit static list (suitable for compilers who are generating classes that do not yet exist) ?- Extract `.class` resources from the running JDK ?- Extract live Class objects from the running JDK These go in increasing order of work done and environmental dependency; earlier items will be faster and more reproducible; latter items will be more usable.? The CHR interface also supports chaining with `orElse`, so that we can start with one strategy and fall back to another. Separately, there is caching, which should be orthogonal but which is built into the default resolver.? There's a public CachedClassHierarchyResolver wrapper (probably should be private) which delegates to another CHR and memoizes the result in a synchronized HashMap.? (There's a static factory ofCached which delegates to this, but it is unused.) More importantly, the default adopts the cache strategy of "cache lives forever / shared among all clients of the API."? I get that we want to avoid re-resolving these classes every time, but if we're going to include class loading as a way to put elements in the cache, there is a risk that it becomes polluted with "private" stale data (and also retains memory forever.) So things I would like to fix include: ?- The basic implementations are not in a form that makes them easily reusable; ?- It is not obvious exactly what the default resolver does (and the name may be wrong), or when the user would want to select an alternate strategy, or what its caching strategy is, but it should definitely be defined in terms of building blocks the user understands; ?- The caching strategy of the default is questionable and probably needs more configuration. We currently define the default as: ??? new CachedCHR( cd -> ClassLoader.getSystemResourceAsStream(Util.toInternalName(classDesc) + ".class")) and it is being proposed that it be redefined as that, plus falling back to class loading if that fails. What I'd like users to come away with is: ?- There's a menu of basic options, which can be combined; ?- There's a way to wrap the basic options with caching, and ideally the user can have some control over sharing / lifetime / caching strategy; ?- There's a default, which is defined strictly in terms of the menu above, which they can reason about. We can start with exposing the basics.? We already have CHR.of(interfaces, supersMap).? We need factories or constants for "do nothing", "system resources from classloader X", "live classes from classloader X", etc. We can refine caching so that CachedCHR is a private implementation, and all public API is funneled through ofCached.? It is possible we will want to expose some more control over the cache, such as wrapping the whole cache with a SoftRef, or allowing the user to specify the backing map; this is one more factory. I think we should rename DEFAULT_CHR to something like GLOBAL_CHR to reflect the fact that it comes with a long-lived, JVM-wide cache.? GLOBAL_CHR can be specified to be CachedCHR.of(systemResourcesCHR()) or CachedCHR.of(systemResourcesChr().orElse(systemClassloaderCHR())). We can _then_ define DEFAULT_CHR to point to GLOBAL_CHR, and people who want a non-shared cache can substitute one. Smaller observation: we have shied away from using public records in the API, instead having a public interface and a private record-backed implementation.? ClassHierarchyInfo is currently a public record. Stepping back, the global cache suggests that we might want to refactor the front-door entry point to the API.? Right now, we have a pile of static methods in Classfile (build/parse), many of which take sets of options.? The point of the global cache is to avoid re-resolving information every time, but the global granularity is questionable -- it seems much more likely that an agent would want to create a static per-agent configuration, shared across uses of that agent, but not shared with other agents.? This suggests that perhaps the methods on Classfile really should be instance methods on an object that encapsulates the option set as well as accumulating cache state.? (And if the agent is unloaded, all the cache goes with it.) Something like: ??? public class ClassfileReaderWriterThingie { ??????? public ClassfileReaderWriterThingie(Option... options) { ... } ??????? public ClassModel parse(byte[] bs) { ... } ??????? public ClassModel parse(Path p) { ... } ??????? public byte[] build(ClassDesc thisClass, Consumer handler) { ... } ??????? // drop overload taking options, since its in the ctor ??????? public byte[] build(ClassEntry thisClassEntry, CPB cpBuilder, Consumer handler) { ... } ??????? // buildTo and buildModule overloads ??? } This is a somewhat more intrusive API change (though not too much), so my suggestion is to proceed on the above first and then revisit this one. (One point being that if we're going to have a long-lived cache, it should probably be encapsulated in an object the user can control.) On 3/20/2023 12:51 PM, Brian Goetz wrote: > After looking over this corner of the API, I think there's a little > more design work that needs to go into it.? So let's step back from > the details of "what's the default" until we have a chance to look a > little more holistically at this part of the API.? I'll try to write > more about it soon. > > On 3/20/2023 11:47 AM, - wrote: >> If the choice of default resolver is still subject to debate, I can >> remove the delegation to system class loader based resolver in >> https://urldefense.com/v3/__https://github.com/openjdk/jdk/pull/13082__;!!ACWV5N9M2RV99hQ!K7UWlrg2NHotSKOzMvmfnenKZ1dDuOZ9zXcQvlKOdB21VvwTRIDuI2NyRNK6Hq844XIkt_6Q-qFt5DsFDUzgKJ31$ , and we can change it in >> another patch once we reach a consensus. The addition of lookup and >> class loader based resolvers can be used by users who explicitly >> specify a resolver in the options. >> >> On Mon, Mar 20, 2023 at 9:00?AM Adam Sotona wrote: >>> New behavior is ?no assumptions? and throw IllegalArgumentException if information about type necessary to build stack map (or verify stack map) is not obtained from the resolver (or better say chain of resolvers). >>> >>> >>> >>> If user provides no resolver ? the default is used. >>> >>> >>> >>> The way how default resolver gets the information from system classloader is subject of discussion below (resource parsing and fallback to class loading). >>> >>> Custom resolver can be constructed as a fallback chain from all provided resolver types (including the default resolver instance). >>> >>> >>> >>> >>> >>> >>> >>> On 20.03.2023 14:25, "Brian Goetz" wrote: >>> >>> >>> If the user provides no resolver at all, what is the new behavior? Do we just assume Object is the common supertype for any pair of classes? >>> >>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Mon Mar 20 18:29:05 2023 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 20 Mar 2023 19:29:05 +0100 (CET) Subject: [External] : Re: ClassHierarchyResolver using Reflection information In-Reply-To: References: <248a1a23-4bcd-076b-5124-5e418b3292f2@oracle.com> <17d29737-43f9-2713-da67-459ab59f1cc8@oracle.com> Message-ID: <1173644065.15928935.1679336945185.JavaMail.zimbra@u-pem.fr> For the caching part, it can be easier to ask each implementation to cache it's own results (like a ClassLoader does) given that depending on the implementation, the way to cache is quite different. regards, R?mi > From: "Brian Goetz" > To: "-" , "Adam Sotona" > Cc: "Michael van Acken" , "classfile-api-dev" > , asotona at openjdk.org > Sent: Monday, March 20, 2023 7:19:45 PM > Subject: Re: [External] : Re: ClassHierarchyResolver using Reflection > information > The CHR interface is straightforward enough -- given a symbolic reference to a > class, provide two bits of information: superclass and is-interface. > We have at least four sensible implementations, which can be combined: > - do nothing > - work from an explicit static list (suitable for compilers who are generating > classes that do not yet exist) > - Extract `.class` resources from the running JDK > - Extract live Class objects from the running JDK > These go in increasing order of work done and environmental dependency; earlier > items will be faster and more reproducible; latter items will be more usable. > The CHR interface also supports chaining with `orElse`, so that we can start > with one strategy and fall back to another. > Separately, there is caching, which should be orthogonal but which is built into > the default resolver. There's a public CachedClassHierarchyResolver wrapper > (probably should be private) which delegates to another CHR and memoizes the > result in a synchronized HashMap. (There's a static factory ofCached which > delegates to this, but it is unused.) > More importantly, the default adopts the cache strategy of "cache lives forever > / shared among all clients of the API." I get that we want to avoid > re-resolving these classes every time, but if we're going to include class > loading as a way to put elements in the cache, there is a risk that it becomes > polluted with "private" stale data (and also retains memory forever.) > So things I would like to fix include: > - The basic implementations are not in a form that makes them easily reusable; > - It is not obvious exactly what the default resolver does (and the name may be > wrong), or when the user would want to select an alternate strategy, or what > its caching strategy is, but it should definitely be defined in terms of > building blocks the user understands; > - The caching strategy of the default is questionable and probably needs more > configuration. > We currently define the default as: > new CachedCHR( cd -> > ClassLoader.getSystemResourceAsStream(Util.toInternalName(classDesc) + > ".class")) > and it is being proposed that it be redefined as that, plus falling back to > class loading if that fails. > What I'd like users to come away with is: > - There's a menu of basic options, which can be combined; > - There's a way to wrap the basic options with caching, and ideally the user can > have some control over sharing / lifetime / caching strategy; > - There's a default, which is defined strictly in terms of the menu above, which > they can reason about. > We can start with exposing the basics. We already have CHR.of(interfaces, > supersMap). We need factories or constants for "do nothing", "system resources > from classloader X", "live classes from classloader X", etc. > We can refine caching so that CachedCHR is a private implementation, and all > public API is funneled through ofCached. It is possible we will want to expose > some more control over the cache, such as wrapping the whole cache with a > SoftRef, or allowing the user to specify the backing map; this is one more > factory. > I think we should rename DEFAULT_CHR to something like GLOBAL_CHR to reflect the > fact that it comes with a long-lived, JVM-wide cache. GLOBAL_CHR can be > specified to be CachedCHR.of(systemResourcesCHR()) or > CachedCHR.of(systemResourcesChr().orElse(systemClassloaderCHR())). > We can _then_ define DEFAULT_CHR to point to GLOBAL_CHR, and people who want a > non-shared cache can substitute one. > Smaller observation: we have shied away from using public records in the API, > instead having a public interface and a private record-backed implementation. > ClassHierarchyInfo is currently a public record. > Stepping back, the global cache suggests that we might want to refactor the > front-door entry point to the API. Right now, we have a pile of static methods > in Classfile (build/parse), many of which take sets of options. The point of > the global cache is to avoid re-resolving information every time, but the > global granularity is questionable -- it seems much more likely that an agent > would want to create a static per-agent configuration, shared across uses of > that agent, but not shared with other agents. This suggests that perhaps the > methods on Classfile really should be instance methods on an object that > encapsulates the option set as well as accumulating cache state. (And if the > agent is unloaded, all the cache goes with it.) > Something like: > public class ClassfileReaderWriterThingie { > public ClassfileReaderWriterThingie(Option... options) { ... } > public ClassModel parse(byte[] bs) { ... } > public ClassModel parse(Path p) { ... } > public byte[] build(ClassDesc thisClass, Consumer handler) { ... } > // drop overload taking options, since its in the ctor > public byte[] build(ClassEntry thisClassEntry, CPB cpBuilder, Consumer > handler) { ... } > // buildTo and buildModule overloads > } > This is a somewhat more intrusive API change (though not too much), so my > suggestion is to proceed on the above first and then revisit this one. > (One point being that if we're going to have a long-lived cache, it should > probably be encapsulated in an object the user can control.) > On 3/20/2023 12:51 PM, Brian Goetz wrote: >> After looking over this corner of the API, I think there's a little more design >> work that needs to go into it. So let's step back from the details of "what's >> the default" until we have a chance to look a little more holistically at this >> part of the API. I'll try to write more about it soon. >> On 3/20/2023 11:47 AM, - wrote: >>> If the choice of default resolver is still subject to debate, I can >>> remove the delegation to system class loader based resolver in [ >>> https://urldefense.com/v3/__https://github.com/openjdk/jdk/pull/13082__;!!ACWV5N9M2RV99hQ!K7UWlrg2NHotSKOzMvmfnenKZ1dDuOZ9zXcQvlKOdB21VvwTRIDuI2NyRNK6Hq844XIkt_6Q-qFt5DsFDUzgKJ31$ >>> | >>> https://urldefense.com/v3/__https://github.com/openjdk/jdk/pull/13082__;!!ACWV5N9M2RV99hQ!K7UWlrg2NHotSKOzMvmfnenKZ1dDuOZ9zXcQvlKOdB21VvwTRIDuI2NyRNK6Hq844XIkt_6Q-qFt5DsFDUzgKJ31$ >>> ] , and we can change it in >>> another patch once we reach a consensus. The addition of lookup and >>> class loader based resolvers can be used by users who explicitly >>> specify a resolver in the options. >>> On Mon, Mar 20, 2023 at 9:00?AM Adam Sotona [ mailto:adam.sotona at oracle.com | >>> ] wrote: >>>> New behavior is ?no assumptions? and throw IllegalArgumentException if >>>> information about type necessary to build stack map (or verify stack map) is >>>> not obtained from the resolver (or better say chain of resolvers). >>>> If user provides no resolver ? the default is used. >>>> The way how default resolver gets the information from system classloader is >>>> subject of discussion below (resource parsing and fallback to class loading). >>>> Custom resolver can be constructed as a fallback chain from all provided >>>> resolver types (including the default resolver instance). >>>> On 20.03.2023 14:25, "Brian Goetz" [ mailto:brian.goetz at oracle.com | >>>> ] wrote: >>>> If the user provides no resolver at all, what is the new behavior? Do we just >>>> assume Object is the common supertype for any pair of classes? -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Mon Mar 20 18:34:47 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 20 Mar 2023 14:34:47 -0400 Subject: [External] : Re: ClassHierarchyResolver using Reflection information In-Reply-To: <1173644065.15928935.1679336945185.JavaMail.zimbra@u-pem.fr> References: <248a1a23-4bcd-076b-5124-5e418b3292f2@oracle.com> <17d29737-43f9-2713-da67-459ab59f1cc8@oracle.com> <1173644065.15928935.1679336945185.JavaMail.zimbra@u-pem.fr> Message-ID: <8112a54d-efe9-2f47-7dbf-7274c3ec2842@oracle.com> I don't disagree; my primary concern is that currently we have a default whose behavior is fairly surprising with respect to cross-user interference and long-term resource utilization. (Encapsulating the cache inside a ClassfileReaderWriterThingie goes a long way towards mitigating that.) On 3/20/2023 2:29 PM, Remi Forax wrote: > For the caching part, it can be easier to ask each implementation to > cache it's own results (like a ClassLoader does) given that depending > on the implementation, the way to cache is quite different. > > regards, > R?mi > > ------------------------------------------------------------------------ > > *From: *"Brian Goetz" > *To: *"-" , "Adam Sotona" > > *Cc: *"Michael van Acken" , > "classfile-api-dev" , > asotona at openjdk.org > *Sent: *Monday, March 20, 2023 7:19:45 PM > *Subject: *Re: [External] : Re: ClassHierarchyResolver using > Reflection information > > The CHR interface is straightforward enough -- given a symbolic > reference to a class, provide two bits of information: superclass > and is-interface. > > We have at least four sensible implementations, which can be > combined: > ?- do nothing > ?- work from an explicit static list (suitable for compilers who > are generating classes that do not yet exist) > ?- Extract `.class` resources from the running JDK > ?- Extract live Class objects from the running JDK > > These go in increasing order of work done and environmental > dependency; earlier items will be faster and more reproducible; > latter items will be more usable.? The CHR interface also supports > chaining with `orElse`, so that we can start with one strategy and > fall back to another. > > Separately, there is caching, which should be orthogonal but which > is built into the default resolver.? There's a public > CachedClassHierarchyResolver wrapper (probably should be private) > which delegates to another CHR and memoizes the result in a > synchronized HashMap.? (There's a static factory ofCached which > delegates to this, but it is unused.) > > More importantly, the default adopts the cache strategy of "cache > lives forever / shared among all clients of the API."? I get that > we want to avoid re-resolving these classes every time, but if > we're going to include class loading as a way to put elements in > the cache, there is a risk that it becomes polluted with "private" > stale data (and also retains memory forever.) > > So things I would like to fix include: > > ?- The basic implementations are not in a form that makes them > easily reusable; > ?- It is not obvious exactly what the default resolver does (and > the name may be wrong), or when the user would want to select an > alternate strategy, or what its caching strategy is, but it should > definitely be defined in terms of building blocks the user > understands; > ?- The caching strategy of the default is questionable and > probably needs more configuration. > > We currently define the default as: > > ??? new CachedCHR( cd -> > ClassLoader.getSystemResourceAsStream(Util.toInternalName(classDesc) > + ".class")) > > and it is being proposed that it be redefined as that, plus > falling back to class loading if that fails. > > What I'd like users to come away with is: > > ?- There's a menu of basic options, which can be combined; > ?- There's a way to wrap the basic options with caching, and > ideally the user can have some control over sharing / lifetime / > caching strategy; > ?- There's a default, which is defined strictly in terms of the > menu above, which they can reason about. > > We can start with exposing the basics.? We already have > CHR.of(interfaces, supersMap).? We need factories or constants for > "do nothing", "system resources from classloader X", "live classes > from classloader X", etc. > > We can refine caching so that CachedCHR is a private > implementation, and all public API is funneled through ofCached.? > It is possible we will want to expose some more control over the > cache, such as wrapping the whole cache with a SoftRef, or > allowing the user to specify the backing map; this is one more > factory. > > I think we should rename DEFAULT_CHR to something like GLOBAL_CHR > to reflect the fact that it comes with a long-lived, JVM-wide > cache.? GLOBAL_CHR can be specified to be > CachedCHR.of(systemResourcesCHR()) or > CachedCHR.of(systemResourcesChr().orElse(systemClassloaderCHR())). > > We can _then_ define DEFAULT_CHR to point to GLOBAL_CHR, and > people who want a non-shared cache can substitute one. > > Smaller observation: we have shied away from using public records > in the API, instead having a public interface and a private > record-backed implementation. ClassHierarchyInfo is currently a > public record. > > > Stepping back, the global cache suggests that we might want to > refactor the front-door entry point to the API. Right now, we have > a pile of static methods in Classfile (build/parse), many of which > take sets of options.? The point of the global cache is to avoid > re-resolving information every time, but the global granularity is > questionable -- it seems much more likely that an agent would want > to create a static per-agent configuration, shared across uses of > that agent, but not shared with other agents.? This suggests that > perhaps the methods on Classfile really should be instance methods > on an object that encapsulates the option set as well as > accumulating cache state.? (And if the agent is unloaded, all the > cache goes with it.) > > Something like: > > ??? public class ClassfileReaderWriterThingie { > ??????? public ClassfileReaderWriterThingie(Option... options) { ... } > > ??????? public ClassModel parse(byte[] bs) { ... } > ??????? public ClassModel parse(Path p) { ... } > > ??????? public byte[] build(ClassDesc thisClass, Consumer > handler) { ... } > ??????? // drop overload taking options, since its in the ctor > ??????? public byte[] build(ClassEntry thisClassEntry, CPB > cpBuilder, Consumer handler) { ... } > > ??????? // buildTo and buildModule overloads > ??? } > > This is a somewhat more intrusive API change (though not too > much), so my suggestion is to proceed on the above first and then > revisit this one. > > (One point being that if we're going to have a long-lived cache, > it should probably be encapsulated in an object the user can control.) > > > On 3/20/2023 12:51 PM, Brian Goetz wrote: > > After looking over this corner of the API, I think there's a > little more design work that needs to go into it.? So let's > step back from the details of "what's the default" until we > have a chance to look a little more holistically at this part > of the API.? I'll try to write more about it soon. > > On 3/20/2023 11:47 AM, - wrote: > > If the choice of default resolver is still subject to debate, I can > remove the delegation to system class loader based resolver in > https://urldefense.com/v3/__https://github.com/openjdk/jdk/pull/13082__;!!ACWV5N9M2RV99hQ!K7UWlrg2NHotSKOzMvmfnenKZ1dDuOZ9zXcQvlKOdB21VvwTRIDuI2NyRNK6Hq844XIkt_6Q-qFt5DsFDUzgKJ31$ , and we can change it in > another patch once we reach a consensus. The addition of lookup and > class loader based resolvers can be used by users who explicitly > specify a resolver in the options. > > On Mon, Mar 20, 2023 at 9:00?AM Adam Sotona wrote: > > New behavior is ?no assumptions? and throw IllegalArgumentException if information about type necessary to build stack map (or verify stack map) is not obtained from the resolver (or better say chain of resolvers). > > > > If user provides no resolver ? the default is used. > > > > The way how default resolver gets the information from system classloader is subject of discussion below (resource parsing and fallback to class loading). > > Custom resolver can be constructed as a fallback chain from all provided resolver types (including the default resolver instance). > > > > > > > > On 20.03.2023 14:25, "Brian Goetz" wrote: > > > If the user provides no resolver at all, what is the new behavior? Do we just assume Object is the common supertype for any pair of classes? > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Mon Mar 20 19:02:53 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 20 Mar 2023 15:02:53 -0400 Subject: [External] : Re: ClassHierarchyResolver using Reflection information In-Reply-To: References: <0883d896-3723-f15b-2573-6a7316feebcf@oracle.com> <248a1a23-4bcd-076b-5124-5e418b3292f2@oracle.com> <17d29737-43f9-2713-da67-459ab59f1cc8@oracle.com> Message-ID: <00beea48-fe6e-7b86-3857-69596f481ff5@oracle.com> Thinking a little more ... if we do this: On 3/20/2023 2:19 PM, Brian Goetz wrote: > > Stepping back, the global cache suggests that we might want to > refactor the front-door entry point to the API.? Right now, we have a > pile of static methods in Classfile (build/parse), many of which take > sets of options.? The point of the global cache is to avoid > re-resolving information every time, but the global granularity is > questionable -- it seems much more likely that an agent would want to > create a static per-agent configuration, shared across uses of that > agent, but not shared with other agents.? This suggests that perhaps > the methods on Classfile really should be instance methods on an > object that encapsulates the option set as well as accumulating cache > state.? (And if the agent is unloaded, all the cache goes with it.) > > Something like: > > ??? public class ClassfileReaderWriterThingie { > ??????? public ClassfileReaderWriterThingie(Option... options) { ... } > > ??????? public ClassModel parse(byte[] bs) { ... } > ??????? public ClassModel parse(Path p) { ... } > > ??????? public byte[] build(ClassDesc thisClass, Consumer handler) > { ... } > ??????? // drop overload taking options, since its in the ctor > ??????? public byte[] build(ClassEntry thisClassEntry, CPB cpBuilder, > Consumer handler) { ... } > > ??????? // buildTo and buildModule overloads > ??? } then we can completely eliminate the static "default" resolver (with attendant cache), and the act of creating a ClassfileRWT should (if no resolver option is selected) create a cached resolver _for that CRWT_.? This completely eliminates the shared static cache, which will surely come back to bite us, without making it any more difficult for users. -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Wed Mar 22 03:00:52 2023 From: liangchenblue at gmail.com (-) Date: Tue, 21 Mar 2023 22:00:52 -0500 Subject: Track Classfile API changes on the classfile-api-dev mailing list In-Reply-To: References: Message-ID: Unfortunately, this is the way it is. The Classfile API currently belongs to "other" under "other-libs" in the bug system, so here's a query that you can use to check Classfile API issues: https://bugs.openjdk.org/browse/JDK-8304502?jql=project%20%3D%20JDK%20AND%20component%20%3D%20other-libs%20AND%20text%20~%20%22Classfile%20API%22%20AND%20Subcomponent%20%3D%20other%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC Which you can check at regular intervals in addition to the Git history. Chen On Sat, Mar 18, 2023 at 2:42?AM Glavo wrote: > > Hi everyone, > > I extracted Classfile API from JDK, ported it back to Java 17, and published it to Maven Central as a standalone library. Here is its repository: > > https://github.com/Glavo/classfile > > (I've posted the above on classfile-api-dev, but I don't see it in the mailing list archives. I don't know if it was sent successfully, so I'll repeat it here) > > To maintain it, I am very concerned about changes to the Classfile API and sync the changes to the above repository. > Unfortunately, the Classfile API does not appear to have a specific JBS label, and I cannot track related PRs from the classfile-api-dev, > so I have to check the history of the Classfile API source folder at regular intervals. > > In order to facilitate tracking, I hope that a label will be assigned to the Classfile API and send relevant PR notification emails to the classfile-api-dev mailing list. > Is that okay? > > Glavo From liangchenblue at gmail.com Wed Mar 22 03:38:03 2023 From: liangchenblue at gmail.com (-) Date: Tue, 21 Mar 2023 22:38:03 -0500 Subject: API suggestions Message-ID: Hi, After housekeeping the few tests after the latest integration of additional constants in ConstantDescs, I have found two potentially common patterns in bytecode manipulation and wish to suggest new APIs: 1. Injection of code in the beginning of methods Currently, our CodeTransform only has an endHandler; however, this is less useful, for a method can have multiple exit points. Usually to track method calls, bytecode manipulation, such as those in jfr and instrumentation, inject at the beginning of methods, which is guaranteed to be on the code path. Thus, I suggest a `startHandler(Consumer)` for CodeTransform for this purpose, instead of having to explicitly override atStart(CodeBuilder) in non-lambda CodeTransform. 2. Adding new annotations In ASM, annotation injection is done simply via extra annotation visitor calls, which makes it compatible for either single or multiple annotations. In contrast, in the Classfile API, if we want to add an annotation, feeding a new RuntimeVisibleAnnotationAttribute will destroy the previous RuntimeVisibleAnnotationAttribute, and handling via a stateful ClassTransform is a bit too lengthy for users. I don't know if we should have an API like Map::computeIfAbsent for attributes or if we will simplify it via other means. Chen Liang From brian.goetz at oracle.com Wed Mar 22 12:11:19 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 22 Mar 2023 08:11:19 -0400 Subject: API suggestions In-Reply-To: References: Message-ID: <27e10eb9-2c47-f333-1d44-55dc168d2b26@oracle.com> Before we get into what API changes are needed, let's sync on the problem first. 1.? CodeTransform has an `atStart` method just as it has an `atEnd` method, and I presume overriding `atStart(builder)` is adequate to your need.? Is the issue simply that there is no canned combinator for startHandler as there is for endHandler? 2.? Adding an annotation is complicated, and we explored a number of solutions before settling in the current place.? There are three cases: ?- There is no RVAA, so you have to add one ?- There is an RVAA, but it already has your annotation, so there's nothing to do ?- There is an RVAA, and it doesn't have your annotation, so you have to transform it (Actually there is a fourth case: it has your annotation, but with _different properties_; there is @Foo(3) but you want to add @Foo(4).? Now what do you do?? Replace it?? Leave it?? Add both?) The trickiness is compounded by the fact that the latter two cases can be handled by transforming the element for RVAA, but the first case can't be handled until the end handler, and you have to remember whether you saw an RVAA or not.? So the process is inherently stateful.? Some options were covered in this thread (which talked about NestMembers, but the story is the same for annotations.) https://mail.openjdk.org/pipermail/classfile-api-dev/2022-August/000108.html Here's the recommended code for that situation: ???? () -> new ClassTransform() { ???????? boolean foundNM = false; ???????? void accept(ClassBuilder b, ClassElement e) { ???????????? switch (e) { ???????????????? case NestMembersAttribute a -> { transform a; foundNM = true; } ???????????????? ... other transform stuff ... ???????????? } ???????? } ???????? void atEnd(ClassBuilder b) { ???????????? if (!foundNM) ???????????????? builder.accept(NestMembersAttribute.of(...)); ???????? } ???? } It's irritating because you have to do three things -- keep track of some state, rewrite an attribute if you see it, and generate an attribute if you don't, but it's really not so lengthy or hard to follow. I can imagine capturing this in a combinator, where you provide two lambdas, one a transform on your favorite attribute, and one a supplier of that attribute, but I don't think it will be that much easier to use, it just automate managing and acting on the boolean. On 3/21/2023 11:38 PM, - wrote: > Hi, > After housekeeping the few tests after the latest integration of > additional constants in ConstantDescs, I have found two potentially > common patterns in bytecode manipulation and wish to suggest new APIs: > > 1. Injection of code in the beginning of methods > Currently, our CodeTransform only has an endHandler; however, this is > less useful, for a method can have multiple exit points. Usually to > track method calls, bytecode manipulation, such as those in jfr and > instrumentation, inject at the beginning of methods, which is > guaranteed to be on the code path. > > Thus, I suggest a `startHandler(Consumer)` for > CodeTransform for this purpose, instead of having to explicitly > override atStart(CodeBuilder) in non-lambda CodeTransform. > > 2. Adding new annotations > In ASM, annotation injection is done simply via extra annotation > visitor calls, which makes it compatible for either single or multiple > annotations. In contrast, in the Classfile API, if we want to add an > annotation, feeding a new RuntimeVisibleAnnotationAttribute will > destroy the previous RuntimeVisibleAnnotationAttribute, and handling > via a stateful ClassTransform is a bit too lengthy for users. > > I don't know if we should have an API like Map::computeIfAbsent for > attributes or if we will simplify it via other means. > > Chen Liang -------------- next part -------------- An HTML attachment was scrubbed... URL: From heidinga at redhat.com Wed Mar 22 12:42:07 2023 From: heidinga at redhat.com (Dan Heidinga) Date: Wed, 22 Mar 2023 08:42:07 -0400 Subject: API suggestions In-Reply-To: <27e10eb9-2c47-f333-1d44-55dc168d2b26@oracle.com> References: <27e10eb9-2c47-f333-1d44-55dc168d2b26@oracle.com> Message-ID: On Wed, Mar 22, 2023 at 8:19?AM Brian Goetz wrote: > Before we get into what API changes are needed, let's sync on the problem > first. > > 1. CodeTransform has an `atStart` method just as it has an `atEnd` > method, and I presume overriding `atStart(builder)` is adequate to your > need. Is the issue simply that there is no canned combinator for > startHandler as there is for endHandler? > > 2. Adding an annotation is complicated, and we explored a number of > solutions before settling in the current place. There are three cases: > > - There is no RVAA, so you have to add one > - There is an RVAA, but it already has your annotation, so there's > nothing to do > - There is an RVAA, and it doesn't have your annotation, so you have to > transform it > > (Actually there is a fourth case: it has your annotation, but with > _different properties_; there is @Foo(3) but you want to add @Foo(4). Now > what do you do? Replace it? Leave it? Add both?) > > The trickiness is compounded by the fact that the latter two cases can be > handled by transforming the element for RVAA, but the first case can't be > handled until the end handler, and you have to remember whether you saw an > RVAA or not. So the process is inherently stateful. Some options were > covered in this thread (which talked about NestMembers, but the story is > the same for annotations.) > > > https://mail.openjdk.org/pipermail/classfile-api-dev/2022-August/000108.html > > Here's the recommended code for that situation: > > () -> new ClassTransform() { > boolean foundNM = false; > > void accept(ClassBuilder b, ClassElement e) { > switch (e) { > case NestMembersAttribute a -> { transform a; foundNM = > true; } > ... other transform stuff ... > } > } > > void atEnd(ClassBuilder b) { > if (!foundNM) > builder.accept(NestMembersAttribute.of(...)); > } > } > > It's irritating because you have to do three things -- keep track of some > state, rewrite an attribute if you see it, and generate an attribute if you > don't, but it's really not so lengthy or hard to follow. > Here's a link to my various attempts to find a better way to express this [0] when exploring NestMember attribute changes. Wrapping Brian's suggested pattern in a named class makes it a little easier to read at the point of use but there's no single pattern for this (that I've found) that results in significantly cleaner code. TLDR: Brian's proposed code is the best you can do for this. --Dan [0] https://github.com/DanHeidinga/jdk-sandbox/blob/9c5bf15522e76ebc106a433758c87623dec4827b/src/jdk.jlink/share/classes/jdk/tools/jlink/internal/plugins/GenerateLambdaClassesPlugin.java#L238-L385 > > I can imagine capturing this in a combinator, where you provide two > lambdas, one a transform on your favorite attribute, and one a supplier of > that attribute, but I don't think it will be that much easier to use, it > just automate managing and acting on the boolean. > > > > > On 3/21/2023 11:38 PM, - wrote: > > Hi, > After housekeeping the few tests after the latest integration of > additional constants in ConstantDescs, I have found two potentially > common patterns in bytecode manipulation and wish to suggest new APIs: > > 1. Injection of code in the beginning of methods > Currently, our CodeTransform only has an endHandler; however, this is > less useful, for a method can have multiple exit points. Usually to > track method calls, bytecode manipulation, such as those in jfr and > instrumentation, inject at the beginning of methods, which is > guaranteed to be on the code path. > > Thus, I suggest a `startHandler(Consumer)` for > CodeTransform for this purpose, instead of having to explicitly > override atStart(CodeBuilder) in non-lambda CodeTransform. > > 2. Adding new annotations > In ASM, annotation injection is done simply via extra annotation > visitor calls, which makes it compatible for either single or multiple > annotations. In contrast, in the Classfile API, if we want to add an > annotation, feeding a new RuntimeVisibleAnnotationAttribute will > destroy the previous RuntimeVisibleAnnotationAttribute, and handling > via a stateful ClassTransform is a bit too lengthy for users. > > I don't know if we should have an API like Map::computeIfAbsent for > attributes or if we will simplify it via other means. > > Chen Liang > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Wed Mar 22 13:29:04 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 22 Mar 2023 09:29:04 -0400 Subject: API suggestions In-Reply-To: References: <27e10eb9-2c47-f333-1d44-55dc168d2b26@oracle.com> Message-ID: <5ee108fa-2ac0-aa4a-c866-32ce774814fd@oracle.com> Here's the combinator I had in mind: ??? static CodeTransform statefulReplace(Class target, Function replacer, Supplier supplier) { ??????? return new CodeTransform() { ??????????? boolean found = false; ??????????? @Override ??????????? public void accept(CodeBuilder builder, CodeElement element) { ??????????????? if (target.isAssignableFrom(element.getClass())) { ??????????????????? found = true; ??????????????????? builder.accept(replacer.apply((T) element)); ??????????????? } ??????????? } ??????????? @Override ??????????? public void atEnd(CodeBuilder builder) { ??????????????? if (!found) ??????????????????? builder.accept(supplier.get()); ??????????? } ??????? }; ??? } So then you could say ??? CodeTransform = statefulReplace(RVAA.class, ??????????????????????????????????? rvaa -> RVAA.of(...), ??????????????????????????????????? () -> RVAA.of(...)); It's cute, but I don't think it carries its weight in the API. I think the real gap here is one of documentation, not abstraction; finding the low-energy-state solution is not obvious enough.? Once you find the solution, making it incrementally smaller with a combinator (or ten) is not as important. Public Service Announcement: people think that code is the important contribution, but sometimes doc is a more valuable contribution.? A "transform writers guide" would be an awesome contribution. On 3/22/2023 8:42 AM, Dan Heidinga wrote: > > > On Wed, Mar 22, 2023 at 8:19?AM Brian Goetz > wrote: > > Before we get into what API changes are needed, let's sync on the > problem first. > > 1.? CodeTransform has an `atStart` method just as it has an > `atEnd` method, and I presume overriding `atStart(builder)` is > adequate to your need.? Is the issue simply that there is no > canned combinator for startHandler as there is for endHandler? > > 2.? Adding an annotation is complicated, and we explored a number > of solutions before settling in the current place.? There are > three cases: > > ?- There is no RVAA, so you have to add one > ?- There is an RVAA, but it already has your annotation, so > there's nothing to do > ?- There is an RVAA, and it doesn't have your annotation, so you > have to transform it > > (Actually there is a fourth case: it has your annotation, but with > _different properties_; there is @Foo(3) but you want to add > @Foo(4).? Now what do you do?? Replace it?? Leave it?? Add both?) > > The trickiness is compounded by the fact that the latter two cases > can be handled by transforming the element for RVAA, but the first > case can't be handled until the end handler, and you have to > remember whether you saw an RVAA or not.? So the process is > inherently stateful.? Some options were covered in this thread > (which talked about NestMembers, but the story is the same for > annotations.) > > https://mail.openjdk.org/pipermail/classfile-api-dev/2022-August/000108.html > > Here's the recommended code for that situation: > > ???? () -> new ClassTransform() { > ???????? boolean foundNM = false; > > ???????? void accept(ClassBuilder b, ClassElement e) { > ???????????? switch (e) { > ???????????????? case NestMembersAttribute a -> { transform a; > foundNM = > true; } > ???????????????? ... other transform stuff ... > ???????????? } > ???????? } > > ???????? void atEnd(ClassBuilder b) { > ???????????? if (!foundNM) > builder.accept(NestMembersAttribute.of(...)); > ???????? } > ???? } > > It's irritating because you have to do three things -- keep track > of some state, rewrite an attribute if you see it, and generate an > attribute if you don't, but it's really not so lengthy or hard to > follow. > > > Here's a link to my various attempts to find a better way to express > this [0] when exploring NestMember attribute changes.? Wrapping > Brian's suggested pattern in a named class makes it a little easier to > read at the point of use but there's no single pattern for this (that > I've found) that results in significantly cleaner code. > > TLDR: Brian's proposed code is the best you can do for this. > > --Dan > > [0] > https://github.com/DanHeidinga/jdk-sandbox/blob/9c5bf15522e76ebc106a433758c87623dec4827b/src/jdk.jlink/share/classes/jdk/tools/jlink/internal/plugins/GenerateLambdaClassesPlugin.java#L238-L385 > > > > I can imagine capturing this in a combinator, where you provide > two lambdas, one a transform on your favorite attribute, and one a > supplier of that attribute, but I don't think it will be that much > easier to use, it just automate managing and acting on the boolean. > > > > > On 3/21/2023 11:38 PM, - wrote: >> Hi, >> After housekeeping the few tests after the latest integration of >> additional constants in ConstantDescs, I have found two potentially >> common patterns in bytecode manipulation and wish to suggest new APIs: >> >> 1. Injection of code in the beginning of methods >> Currently, our CodeTransform only has an endHandler; however, this is >> less useful, for a method can have multiple exit points. Usually to >> track method calls, bytecode manipulation, such as those in jfr and >> instrumentation, inject at the beginning of methods, which is >> guaranteed to be on the code path. >> >> Thus, I suggest a `startHandler(Consumer)` for >> CodeTransform for this purpose, instead of having to explicitly >> override atStart(CodeBuilder) in non-lambda CodeTransform. >> >> 2. Adding new annotations >> In ASM, annotation injection is done simply via extra annotation >> visitor calls, which makes it compatible for either single or multiple >> annotations. In contrast, in the Classfile API, if we want to add an >> annotation, feeding a new RuntimeVisibleAnnotationAttribute will >> destroy the previous RuntimeVisibleAnnotationAttribute, and handling >> via a stateful ClassTransform is a bit too lengthy for users. >> >> I don't know if we should have an API like Map::computeIfAbsent for >> attributes or if we will simplify it via other means. >> >> Chen Liang > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Wed Mar 22 18:55:08 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Wed, 22 Mar 2023 18:55:08 +0000 Subject: [External] : Re: ClassHierarchyResolver using Reflection information In-Reply-To: References: <0883d896-3723-f15b-2573-6a7316feebcf@oracle.com> <248a1a23-4bcd-076b-5124-5e418b3292f2@oracle.com> <17d29737-43f9-2713-da67-459ab59f1cc8@oracle.com> Message-ID: Based on the discussion in this thread I propose to make a first step and adjust ClassHierarchy factory methods following way: * ofCached(ClassHierarchyResolver resolver) * ofClassLoading(ClassLoader loader) * ofResourceParsing(ClassLoader loader) * ofParsing(Function classStreamResolver) * of(MethodHandles.Lookup lookup) * of(Collection interfaces, Map classToSuperClass) So the default can be defined as: DEFAULT_CLASS_HIERARCHY_RESOLVER = ofCached(ofResourceParsing(ClassLoader.getSystemClassLoader()).orElse(ofClassLoading(ClassLoader.getSystemClassLoader()))); Please help me to polish exact methods naming. Thank you, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Wed Mar 22 19:00:22 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Wed, 22 Mar 2023 19:00:22 +0000 Subject: [External] : Re: ClassHierarchyResolver using Reflection information In-Reply-To: References: <0883d896-3723-f15b-2573-6a7316feebcf@oracle.com> <248a1a23-4bcd-076b-5124-5e418b3292f2@oracle.com> <17d29737-43f9-2713-da67-459ab59f1cc8@oracle.com> Message-ID: Optionally one more possible factory method: * of(Predicate isInterface, Function superclassOf) From: Adam Sotona Date: Wednesday, 22 March 2023 19:55 To: Brian Goetz , liangchenblue at gmail.com Cc: Michael van Acken , classfile-api-dev at openjdk.org , asotona at openjdk.org Subject: Re: [External] : Re: ClassHierarchyResolver using Reflection information Based on the discussion in this thread I propose to make a first step and adjust ClassHierarchy factory methods following way: * ofCached(ClassHierarchyResolver resolver) * ofClassLoading(ClassLoader loader) * ofResourceParsing(ClassLoader loader) * ofParsing(Function classStreamResolver) * of(MethodHandles.Lookup lookup) * of(Collection interfaces, Map classToSuperClass) So the default can be defined as: DEFAULT_CLASS_HIERARCHY_RESOLVER = ofCached(ofResourceParsing(ClassLoader.getSystemClassLoader()).orElse(ofClassLoading(ClassLoader.getSystemClassLoader()))); Please help me to polish exact methods naming. Thank you, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Wed Mar 22 19:09:51 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 22 Mar 2023 15:09:51 -0400 Subject: [External] : Re: ClassHierarchyResolver using Reflection information In-Reply-To: References: <0883d896-3723-f15b-2573-6a7316feebcf@oracle.com> <248a1a23-4bcd-076b-5124-5e418b3292f2@oracle.com> <17d29737-43f9-2713-da67-459ab59f1cc8@oracle.com> Message-ID: <380e6bd4-448f-f3be-d012-74a7f4e45ca2@oracle.com> This is a good first step.? In fact, I was just tinkering with a similar patch -- I'll send you what I've got.? More inline. On 3/22/2023 2:55 PM, Adam Sotona wrote: > > Based on the discussion in this thread I propose to make a first step > and adjust ClassHierarchy factory methods following way: > > * ofCached(ClassHierarchyResolver resolver) > Yes, caching should be something that is wrapped around a CHR, not complected with resource parsing.? The one thing we lose is the optimization where we opportunistically infer that a class is an interface if we discover it in the `implements` section of a classfile -- but I don't mind saying goodbye to that optimization. > * ofClassLoading(ClassLoader loader) > * ofResourceParsing(ClassLoader loader) > * ofParsing(Function classStreamResolver) > How do these last two differ? > * of(MethodHandles.Lookup lookup) > * of(Collection interfaces, Map > classToSuperClass) > > So the default can be defined as: > > DEFAULT_CLASS_HIERARCHY_RESOLVER = > ofCached(ofResourceParsing(ClassLoader.getSystemClassLoader()).orElse(ofClassLoading(ClassLoader.getSystemClassLoader()))); > I recommend making one more step: turn the static default field into a factory: static ClassHierarchyResolver standardResolver() { return ClassHierarchyResolver.ofCached(ClassHierarchyResolver.ofResource( new Function() { @Override public InputStream apply(ClassDesc classDesc) { return ClassLoader.getSystemResourceAsStream(Util.toInternalName(classDesc) +".class"); } })); } Also, I think ofCached() should optionally let you specify a Map factory; some users might prefer a bound LHM, or to forego the synchronization if they know use will be single-threaded: CHR ofCached(CHR) CHR ofCached(CHR, Supplier>) > Please help me to polish exact methods naming. > > Thank you, > > Adam > -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Thu Mar 23 00:37:17 2023 From: liangchenblue at gmail.com (-) Date: Wed, 22 Mar 2023 19:37:17 -0500 Subject: API suggestions In-Reply-To: <5ee108fa-2ac0-aa4a-c866-32ce774814fd@oracle.com> References: <27e10eb9-2c47-f333-1d44-55dc168d2b26@oracle.com> <5ee108fa-2ac0-aa4a-c866-32ce774814fd@oracle.com> Message-ID: Alternatively, we can probably use var annos = classModel.findAttribute(Attributes.RUNTIME_VISIBLE_ANNOTATIONS); if (annos.isPresent()) transform = transform.andThen(ClassTransform.dropping(ce -> ce instanceof RuntimeVisibleAnnotationsAttribute)); // prepare the updated annotations Back to the first question, I still believe something like CodeTransform.startHandler(cb -> { cb.constantInstruction(index); cb.invokestatic(CD_MyClass, "checkpoint", MTD_void_int); }); is more concise than: new CodeTransform() { @Override public void atStart(CodeBuilder cb) { cb.constantInstruction(index); cb.invokestatic(CD_MyClass, "checkpoint", MTD_void_int); } @Override public void accept(CodeBuilder cb, CodeElement ce) { cb.with(ce); } } I still believe a CodeTransform.startHandler(Consumer) should be added for the frequency of such call tracking patterns and the convenience, unlike the attribute update pattern, which is much less applicable. On Wed, Mar 22, 2023 at 8:29?AM Brian Goetz wrote: > > Here's the combinator I had in mind: > > static CodeTransform statefulReplace(Class target, > Function replacer, > Supplier supplier) { > return new CodeTransform() { > boolean found = false; > > @Override > public void accept(CodeBuilder builder, CodeElement element) { > if (target.isAssignableFrom(element.getClass())) { > found = true; > builder.accept(replacer.apply((T) element)); > } > } > > @Override > public void atEnd(CodeBuilder builder) { > if (!found) > builder.accept(supplier.get()); > } > }; > } > > So then you could say > > CodeTransform = statefulReplace(RVAA.class, > rvaa -> RVAA.of(...), > () -> RVAA.of(...)); > > It's cute, but I don't think it carries its weight in the API. I think the real gap here is one of documentation, not abstraction; finding the low-energy-state solution is not obvious enough. Once you find the solution, making it incrementally smaller with a combinator (or ten) is not as important. > > Public Service Announcement: people think that code is the important contribution, but sometimes doc is a more valuable contribution. A "transform writers guide" would be an awesome contribution. > > > On 3/22/2023 8:42 AM, Dan Heidinga wrote: > > > > On Wed, Mar 22, 2023 at 8:19?AM Brian Goetz wrote: >> >> Before we get into what API changes are needed, let's sync on the problem first. >> >> 1. CodeTransform has an `atStart` method just as it has an `atEnd` method, and I presume overriding `atStart(builder)` is adequate to your need. Is the issue simply that there is no canned combinator for startHandler as there is for endHandler? >> >> 2. Adding an annotation is complicated, and we explored a number of solutions before settling in the current place. There are three cases: >> >> - There is no RVAA, so you have to add one >> - There is an RVAA, but it already has your annotation, so there's nothing to do >> - There is an RVAA, and it doesn't have your annotation, so you have to transform it >> >> (Actually there is a fourth case: it has your annotation, but with _different properties_; there is @Foo(3) but you want to add @Foo(4). Now what do you do? Replace it? Leave it? Add both?) >> >> The trickiness is compounded by the fact that the latter two cases can be handled by transforming the element for RVAA, but the first case can't be handled until the end handler, and you have to remember whether you saw an RVAA or not. So the process is inherently stateful. Some options were covered in this thread (which talked about NestMembers, but the story is the same for annotations.) >> >> https://mail.openjdk.org/pipermail/classfile-api-dev/2022-August/000108.html >> >> Here's the recommended code for that situation: >> >> () -> new ClassTransform() { >> boolean foundNM = false; >> >> void accept(ClassBuilder b, ClassElement e) { >> switch (e) { >> case NestMembersAttribute a -> { transform a; foundNM = >> true; } >> ... other transform stuff ... >> } >> } >> >> void atEnd(ClassBuilder b) { >> if (!foundNM) >> builder.accept(NestMembersAttribute.of(...)); >> } >> } >> >> It's irritating because you have to do three things -- keep track of some state, rewrite an attribute if you see it, and generate an attribute if you don't, but it's really not so lengthy or hard to follow. > > > Here's a link to my various attempts to find a better way to express this [0] when exploring NestMember attribute changes. Wrapping Brian's suggested pattern in a named class makes it a little easier to read at the point of use but there's no single pattern for this (that I've found) that results in significantly cleaner code. > > TLDR: Brian's proposed code is the best you can do for this. > > --Dan > > [0] https://github.com/DanHeidinga/jdk-sandbox/blob/9c5bf15522e76ebc106a433758c87623dec4827b/src/jdk.jlink/share/classes/jdk/tools/jlink/internal/plugins/GenerateLambdaClassesPlugin.java#L238-L385 > >> >> >> I can imagine capturing this in a combinator, where you provide two lambdas, one a transform on your favorite attribute, and one a supplier of that attribute, but I don't think it will be that much easier to use, it just automate managing and acting on the boolean. >> >> >> >> >> On 3/21/2023 11:38 PM, - wrote: >> >> Hi, >> After housekeeping the few tests after the latest integration of >> additional constants in ConstantDescs, I have found two potentially >> common patterns in bytecode manipulation and wish to suggest new APIs: >> >> 1. Injection of code in the beginning of methods >> Currently, our CodeTransform only has an endHandler; however, this is >> less useful, for a method can have multiple exit points. Usually to >> track method calls, bytecode manipulation, such as those in jfr and >> instrumentation, inject at the beginning of methods, which is >> guaranteed to be on the code path. >> >> Thus, I suggest a `startHandler(Consumer)` for >> CodeTransform for this purpose, instead of having to explicitly >> override atStart(CodeBuilder) in non-lambda CodeTransform. >> >> 2. Adding new annotations >> In ASM, annotation injection is done simply via extra annotation >> visitor calls, which makes it compatible for either single or multiple >> annotations. In contrast, in the Classfile API, if we want to add an >> annotation, feeding a new RuntimeVisibleAnnotationAttribute will >> destroy the previous RuntimeVisibleAnnotationAttribute, and handling >> via a stateful ClassTransform is a bit too lengthy for users. >> >> I don't know if we should have an API like Map::computeIfAbsent for >> attributes or if we will simplify it via other means. >> >> Chen Liang >> >> > From brian.goetz at oracle.com Thu Mar 23 01:53:33 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 22 Mar 2023 21:53:33 -0400 Subject: API suggestions In-Reply-To: References: <27e10eb9-2c47-f333-1d44-55dc168d2b26@oracle.com> <5ee108fa-2ac0-aa4a-c866-32ce774814fd@oracle.com> Message-ID: Yes, the lack of a startHandler() combinator probably stems from the fact that atStart() was added later, and we didn't go back and add the combinators.? So its a fine addition, let's just make sure to cover all the model types. On 3/22/2023 8:37 PM, - wrote: > Alternatively, we can probably use > > var annos = classModel.findAttribute(Attributes.RUNTIME_VISIBLE_ANNOTATIONS); > if (annos.isPresent()) transform = > transform.andThen(ClassTransform.dropping(ce -> ce instanceof > RuntimeVisibleAnnotationsAttribute)); > // prepare the updated annotations > > Back to the first question, I still believe something like > > CodeTransform.startHandler(cb -> { > cb.constantInstruction(index); > cb.invokestatic(CD_MyClass, "checkpoint", MTD_void_int); > }); > > is more concise than: > > new CodeTransform() { > @Override > public void atStart(CodeBuilder cb) { > cb.constantInstruction(index); > cb.invokestatic(CD_MyClass, "checkpoint", MTD_void_int); > } > > @Override > public void accept(CodeBuilder cb, CodeElement ce) { > cb.with(ce); > } > } > > I still believe a CodeTransform.startHandler(Consumer) > should be added for the frequency of such call tracking patterns and > the convenience, unlike the attribute update pattern, which is much > less applicable. > > On Wed, Mar 22, 2023 at 8:29?AM Brian Goetz wrote: >> Here's the combinator I had in mind: >> >> static CodeTransform statefulReplace(Class target, >> Function replacer, >> Supplier supplier) { >> return new CodeTransform() { >> boolean found = false; >> >> @Override >> public void accept(CodeBuilder builder, CodeElement element) { >> if (target.isAssignableFrom(element.getClass())) { >> found = true; >> builder.accept(replacer.apply((T) element)); >> } >> } >> >> @Override >> public void atEnd(CodeBuilder builder) { >> if (!found) >> builder.accept(supplier.get()); >> } >> }; >> } >> >> So then you could say >> >> CodeTransform = statefulReplace(RVAA.class, >> rvaa -> RVAA.of(...), >> () -> RVAA.of(...)); >> >> It's cute, but I don't think it carries its weight in the API. I think the real gap here is one of documentation, not abstraction; finding the low-energy-state solution is not obvious enough. Once you find the solution, making it incrementally smaller with a combinator (or ten) is not as important. >> >> Public Service Announcement: people think that code is the important contribution, but sometimes doc is a more valuable contribution. A "transform writers guide" would be an awesome contribution. >> >> >> On 3/22/2023 8:42 AM, Dan Heidinga wrote: >> >> >> >> On Wed, Mar 22, 2023 at 8:19?AM Brian Goetz wrote: >>> Before we get into what API changes are needed, let's sync on the problem first. >>> >>> 1. CodeTransform has an `atStart` method just as it has an `atEnd` method, and I presume overriding `atStart(builder)` is adequate to your need. Is the issue simply that there is no canned combinator for startHandler as there is for endHandler? >>> >>> 2. Adding an annotation is complicated, and we explored a number of solutions before settling in the current place. There are three cases: >>> >>> - There is no RVAA, so you have to add one >>> - There is an RVAA, but it already has your annotation, so there's nothing to do >>> - There is an RVAA, and it doesn't have your annotation, so you have to transform it >>> >>> (Actually there is a fourth case: it has your annotation, but with _different properties_; there is @Foo(3) but you want to add @Foo(4). Now what do you do? Replace it? Leave it? Add both?) >>> >>> The trickiness is compounded by the fact that the latter two cases can be handled by transforming the element for RVAA, but the first case can't be handled until the end handler, and you have to remember whether you saw an RVAA or not. So the process is inherently stateful. Some options were covered in this thread (which talked about NestMembers, but the story is the same for annotations.) >>> >>> https://mail.openjdk.org/pipermail/classfile-api-dev/2022-August/000108.html >>> >>> Here's the recommended code for that situation: >>> >>> () -> new ClassTransform() { >>> boolean foundNM = false; >>> >>> void accept(ClassBuilder b, ClassElement e) { >>> switch (e) { >>> case NestMembersAttribute a -> { transform a; foundNM = >>> true; } >>> ... other transform stuff ... >>> } >>> } >>> >>> void atEnd(ClassBuilder b) { >>> if (!foundNM) >>> builder.accept(NestMembersAttribute.of(...)); >>> } >>> } >>> >>> It's irritating because you have to do three things -- keep track of some state, rewrite an attribute if you see it, and generate an attribute if you don't, but it's really not so lengthy or hard to follow. >> >> Here's a link to my various attempts to find a better way to express this [0] when exploring NestMember attribute changes. Wrapping Brian's suggested pattern in a named class makes it a little easier to read at the point of use but there's no single pattern for this (that I've found) that results in significantly cleaner code. >> >> TLDR: Brian's proposed code is the best you can do for this. >> >> --Dan >> >> [0]https://urldefense.com/v3/__https://github.com/DanHeidinga/jdk-sandbox/blob/9c5bf15522e76ebc106a433758c87623dec4827b/src/jdk.jlink/share/classes/jdk/tools/jlink/internal/plugins/GenerateLambdaClassesPlugin.java*L238-L385__;Iw!!ACWV5N9M2RV99hQ!Mzj3D4_YF8kfkC5I494fKMxul3D8K_m-Fzc4g89e1jmMMjRKYU7afJwvz5bTjsb0zk8jxY6IoJPFdPKFoZBxyrCm$ >> >>> >>> I can imagine capturing this in a combinator, where you provide two lambdas, one a transform on your favorite attribute, and one a supplier of that attribute, but I don't think it will be that much easier to use, it just automate managing and acting on the boolean. >>> >>> >>> >>> >>> On 3/21/2023 11:38 PM, - wrote: >>> >>> Hi, >>> After housekeeping the few tests after the latest integration of >>> additional constants in ConstantDescs, I have found two potentially >>> common patterns in bytecode manipulation and wish to suggest new APIs: >>> >>> 1. Injection of code in the beginning of methods >>> Currently, our CodeTransform only has an endHandler; however, this is >>> less useful, for a method can have multiple exit points. Usually to >>> track method calls, bytecode manipulation, such as those in jfr and >>> instrumentation, inject at the beginning of methods, which is >>> guaranteed to be on the code path. >>> >>> Thus, I suggest a `startHandler(Consumer)` for >>> CodeTransform for this purpose, instead of having to explicitly >>> override atStart(CodeBuilder) in non-lambda CodeTransform. >>> >>> 2. Adding new annotations >>> In ASM, annotation injection is done simply via extra annotation >>> visitor calls, which makes it compatible for either single or multiple >>> annotations. In contrast, in the Classfile API, if we want to add an >>> annotation, feeding a new RuntimeVisibleAnnotationAttribute will >>> destroy the previous RuntimeVisibleAnnotationAttribute, and handling >>> via a stateful ClassTransform is a bit too lengthy for users. >>> >>> I don't know if we should have an API like Map::computeIfAbsent for >>> attributes or if we will simplify it via other means. >>> >>> Chen Liang >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Fri Mar 31 11:54:56 2023 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 31 Mar 2023 12:54:56 +0100 Subject: CodeBuilder::allocateLocal Message-ID: <6c9e4dae-6403-9d00-c91b-ce7b00eff6f3@oracle.com> Hi, while reviewing this: https://git.openjdk.org/jdk/pull/13247 I couldn't help but getting rubbed the wrong way seeing "allocate" next to "Local". I realize this might be a subjective bias (I've been staring at memory allocators for too long :-) ), but I don't think that the concept of "allocation" belongs to local variables? I think a more neutral "newLocal", "makeLocal", "createLocal" might work better in this regard. Cheers Maurizio From brian.goetz at oracle.com Fri Mar 31 14:51:25 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 31 Mar 2023 10:51:25 -0400 Subject: CodeBuilder::allocateLocal In-Reply-To: <6c9e4dae-6403-9d00-c91b-ce7b00eff6f3@oracle.com> References: <6c9e4dae-6403-9d00-c91b-ce7b00eff6f3@oracle.com> Message-ID: <685ebed0-1d2b-2d42-cc5f-7607e71f228a@oracle.com> Yes, really this is shorthand for allocating a local _slot_.? And this really is allocation -- its a slab allocator with stack-based allocation/deallocation of sub-slabs. Names like "newLocal" don't really capture what's going on, because we're not, for example, making a data structure that represents a local, we're reserving a slot for it. I had considered "allocateSlot" which is more accurate but I thought "slot" by itself might be obvious.? Other options include: ??? allocateLocalSlot ??? reserveLocal ??? reserveLocalSlot ??? more? On 3/31/2023 7:54 AM, Maurizio Cimadamore wrote: > Hi, > while reviewing this: > > https://git.openjdk.org/jdk/pull/13247 > > I couldn't help but getting rubbed the wrong way seeing "allocate" > next to "Local". I realize this might be a subjective bias (I've been > staring at memory allocators for too long :-) ), but I don't think > that the concept of "allocation" belongs to local variables? I think a > more neutral "newLocal", "makeLocal", "createLocal" might work better > in this regard. > > Cheers > Maurizio > -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Fri Mar 31 15:03:23 2023 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 31 Mar 2023 16:03:23 +0100 Subject: CodeBuilder::allocateLocal In-Reply-To: <685ebed0-1d2b-2d42-cc5f-7607e71f228a@oracle.com> References: <6c9e4dae-6403-9d00-c91b-ce7b00eff6f3@oracle.com> <685ebed0-1d2b-2d42-cc5f-7607e71f228a@oracle.com> Message-ID: On 31/03/2023 15:51, Brian Goetz wrote: > Yes, really this is shorthand for allocating a local _slot_.? And this > really is allocation -- its a slab allocator with stack-based > allocation/deallocation of sub-slabs. Not sure I follow what you mean here. All the implementations of this methods just bump an index and return (which is what I'd expect)... am I missing something in the impl? > > Names like "newLocal" don't really capture what's going on, because > we're not, for example, making a data structure that represents a > local, we're reserving a slot for it. > > I had considered "allocateSlot" which is more accurate but I thought > "slot" by itself might be obvious.? Other options include: > > ??? allocateLocalSlot > ??? reserveLocal > ??? reserveLocalSlot > ??? more? I guess the "reserve" is a good one, at least subjectively. Maurizio > > On 3/31/2023 7:54 AM, Maurizio Cimadamore wrote: >> Hi, >> while reviewing this: >> >> https://git.openjdk.org/jdk/pull/13247 >> >> I couldn't help but getting rubbed the wrong way seeing "allocate" >> next to "Local". I realize this might be a subjective bias (I've been >> staring at memory allocators for too long :-) ), but I don't think >> that the concept of "allocation" belongs to local variables? I think >> a more neutral "newLocal", "makeLocal", "createLocal" might work >> better in this regard. >> >> Cheers >> Maurizio >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Fri Mar 31 15:38:42 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 31 Mar 2023 11:38:42 -0400 Subject: CodeBuilder::allocateLocal In-Reply-To: References: <6c9e4dae-6403-9d00-c91b-ce7b00eff6f3@oracle.com> <685ebed0-1d2b-2d42-cc5f-7607e71f228a@oracle.com> Message-ID: <51c87917-d65b-9e17-f4e1-59082012daea@oracle.com> Reserve is probably better than allocate, since a reservation is purely advisory -- someone could barge right in and sit down at your supposedly-reserved table anyway :) On 3/31/2023 11:03 AM, Maurizio Cimadamore wrote: > > > On 31/03/2023 15:51, Brian Goetz wrote: >> Yes, really this is shorthand for allocating a local _slot_.? And >> this really is allocation -- its a slab allocator with stack-based >> allocation/deallocation of sub-slabs. > Not sure I follow what you mean here. All the implementations of this > methods just bump an index and return (which is what I'd expect)... am > I missing something in the impl? >> >> Names like "newLocal" don't really capture what's going on, because >> we're not, for example, making a data structure that represents a >> local, we're reserving a slot for it. >> >> I had considered "allocateSlot" which is more accurate but I thought >> "slot" by itself might be obvious.? Other options include: >> >> ??? allocateLocalSlot >> ??? reserveLocal >> ??? reserveLocalSlot >> ??? more? > > I guess the "reserve" is a good one, at least subjectively. > > Maurizio > >> >> On 3/31/2023 7:54 AM, Maurizio Cimadamore wrote: >>> Hi, >>> while reviewing this: >>> >>> https://git.openjdk.org/jdk/pull/13247 >>> >>> I couldn't help but getting rubbed the wrong way seeing "allocate" >>> next to "Local". I realize this might be a subjective bias (I've >>> been staring at memory allocators for too long :-) ), but I don't >>> think that the concept of "allocation" belongs to local variables? I >>> think a more neutral "newLocal", "makeLocal", "createLocal" might >>> work better in this regard. >>> >>> Cheers >>> Maurizio >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: