From adam.sotona at oracle.com Wed May 3 07:16:38 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Wed, 3 May 2023 07:16:38 +0000 Subject: Package jdk.internal.classfile.java.lang.constant become obsolete Message-ID: Hi, Package jdk.internal.java.lang.constant containing ModuleDesc and PackageDesc become obsolete after JDK-8306729 ?Add nominal descriptors of modules and packages to Constants API?. I?ve created JDK-8307326 to handle the transition, with following content: All references to ModuleDesc and PackageDesc classes in jdk.internal.classfile.java.lang.constant package should be replaced with references to the relevant classes in java.lang.constant package (across all JDK sources, tests and JMH benchmarks). The package export hooks should be removed from java.base module-info, make files and test headers. The package should be deleted. All ?in progress? Classfile API and related works and online Javadoc should also follow up with updates after this refactoring. All comments and suggestions are welcome. Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Wed May 17 13:59:11 2023 From: liangchenblue at gmail.com (-) Date: Wed, 17 May 2023 08:59:11 -0500 Subject: Planned features of a classfile context object Message-ID: Hi, In the discussions a few weeks ago, we envisioned a Classfile context object shared across multiple Classfile processing scenarios, mainly for Class hierarchy information caching. I want to confirm that these are contents of the object, mainly promoted from individual options: - Class Hierarchy information caching - Custom attribute processing - A set of default options, propagated to individual processing (may be overridden classfile-wise) What else is planned for the context object currently? Please don't hesitate to propose. Thanks! Chen Liang From brian.goetz at oracle.com Wed May 17 15:21:27 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 17 May 2023 11:21:27 -0400 Subject: Planned features of a classfile context object In-Reply-To: References: Message-ID: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> These are basically what is on my list, though I had in mind to move _all_ option information to the context, and leave it out of individual processing decisions. On 5/17/2023 9:59 AM, - wrote: > Hi, > In the discussions a few weeks ago, we envisioned a Classfile context > object shared across multiple Classfile processing scenarios, mainly > for Class hierarchy information caching. > > I want to confirm that these are contents of the object, mainly > promoted from individual options: > - Class Hierarchy information caching > - Custom attribute processing > - A set of default options, propagated to individual processing (may > be overridden classfile-wise) > > What else is planned for the context object currently? Please don't > hesitate to propose. Thanks! > > Chen Liang -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Thu May 18 14:16:36 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 18 May 2023 10:16:36 -0400 Subject: Planned features of a classfile context object In-Reply-To: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> References: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> Message-ID: <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> Currently the ConstantPoolBuilder is another place we we attach options.? This was more of a sin of convenience than anything else; the theory was that if we are transforming a class, we'll parse it with a set of options, and then generate a new class which shares its CP, and the options came along for the ride. Searching for a better name that ClassfileContext.? The name Classfile is too good to waste, but it doesn't represent a classfile (that's ClassModel), so calling the new thing Classfile would likely be weird.? Also ClassfileContext would make it harder to discover the key entry points (build/parse). Classfile.Context is better in that it is more discoverable at least. Classfile{Parser,Generator} are more discoverable but only tell half the story, and there's no obvious "first half" of the story.? ClassfileReaderWriter is discoverable and honest but long.? Classfile{Broker,Manager,Mediator} sound like parodies of the Design Patterns era. A slightly cheeky but possibly viable option is "Classfiles"; while not a collection of classfiles, it is a collection of behaviors _about_ classfiles. ??? Classfiles.of(options).parse(bytes) ??? Classfiles.of(options).generate(handler) On 5/17/2023 11:21 AM, Brian Goetz wrote: > These are basically what is on my list, though I had in mind to move > _all_ option information to the context, and leave it out of > individual processing decisions. > > On 5/17/2023 9:59 AM, - wrote: >> Hi, >> In the discussions a few weeks ago, we envisioned a Classfile context >> object shared across multiple Classfile processing scenarios, mainly >> for Class hierarchy information caching. >> >> I want to confirm that these are contents of the object, mainly >> promoted from individual options: >> - Class Hierarchy information caching >> - Custom attribute processing >> - A set of default options, propagated to individual processing (may >> be overridden classfile-wise) >> >> What else is planned for the context object currently? Please don't >> hesitate to propose. Thanks! >> >> Chen Liang > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Thu May 18 14:32:58 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 18 May 2023 10:32:58 -0400 Subject: Bug in attribute writing Message-ID: I just noticed that in BoundAttribute.BoundUnknownAttribute: ??????? private void checkWriteSupported(Function condition) { ??????????? if (!condition.apply(classReader)) ??????????????? throw new UnsupportedOperationException("Write of unknown attribute " + attributeName() + " not supported to alien constant pool"); ??????? } we should at least be checking the option "support unknown attributes" before we say OK. Overall, I am not sure we have a fully consistent story for unsupported attributes here.? The basic problem with attributes for which we don't have an AttributeMapper is that we can't be sure about validity.? We don't know where the CP indexes are (though if we are sharing CPs, this is OK), and we don't know what other data might be off. (We also have a bigger problem with the type annotations attributes, which are known to have all sorts of non-CP offsets (nth bound of mth type variable, nth exception, indexes into code array, etc) which could easily but thrown off by transformation and there is no practical way to detect whether the original RVTA is still valid.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Fri May 19 07:52:13 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Fri, 19 May 2023 07:52:13 +0000 Subject: Planned features of a classfile context object In-Reply-To: <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> References: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> Message-ID: I think our current Classfile.Option is already not so far from the proposed global context holder approach. The most significant use cases to persist (class hierarchy resolution and custom attributes) can be very well cached by keeping the Classfile.Option instances and re-using them as many times as user needs. var options = List.of(Option.classHierarchyResolver(resolver), Option.attributeMapper(mapper)); Classfile.build(clsDesc, options, handler); Classfile.parse(bytes, options); //here we have only vararg now What I?m missing much much more than ?global options holder? is per-transformation options (or even option change for transformed method). For example I would like to use one class model to transform into multiple classes with different options, and I have to write: Classfile.parse(bytes, options1).transform(classTransform1); Classfile.parse(bytes, options2).transform(classTransform2); Instead of simple: var clm = Classfile.parse(bytes); clm.transform(classTransform1, options1); clm.transform(classTransform2, options2); Or maybe: var clm = Classfile.parse(bytes, options1); clm.transform(classTransform1); clm.setOptions(options2); clm.transform(classTransform2); However it raises the main question about what options are really tied to model parsing (I think it is only attributeMapper). And what options can be safely changed later. For example can I change processDebug, processLineNumbers or processUnknownAttributes on already parsed class model to get different results of the following model traversal? And there are majority of options unrelated to the parsed class model but affecting only building and transformations (generateStackmap, constantPoolSharing, fixShortJumps, patchDeadCode, classHierarchyResolver, filterDeadLabels). I think we should make more clear which options affect what actions (parsing, building, transformation) and give more flexibility to change them on the fly. Thanks, Adam From: classfile-api-dev on behalf of Brian Goetz Date: Thursday, 18 May 2023 16:16 To: liangchenblue at gmail.com , classfile-api-dev Subject: Re: Planned features of a classfile context object Currently the ConstantPoolBuilder is another place we we attach options. This was more of a sin of convenience than anything else; the theory was that if we are transforming a class, we'll parse it with a set of options, and then generate a new class which shares its CP, and the options came along for the ride. Searching for a better name that ClassfileContext. The name Classfile is too good to waste, but it doesn't represent a classfile (that's ClassModel), so calling the new thing Classfile would likely be weird. Also ClassfileContext would make it harder to discover the key entry points (build/parse). Classfile.Context is better in that it is more discoverable at least. Classfile{Parser,Generator} are more discoverable but only tell half the story, and there's no obvious "first half" of the story. ClassfileReaderWriter is discoverable and honest but long. Classfile{Broker,Manager,Mediator} sound like parodies of the Design Patterns era. A slightly cheeky but possibly viable option is "Classfiles"; while not a collection of classfiles, it is a collection of behaviors _about_ classfiles. Classfiles.of(options).parse(bytes) Classfiles.of(options).generate(handler) On 5/17/2023 11:21 AM, Brian Goetz wrote: These are basically what is on my list, though I had in mind to move _all_ option information to the context, and leave it out of individual processing decisions. On 5/17/2023 9:59 AM, - wrote: Hi, In the discussions a few weeks ago, we envisioned a Classfile context object shared across multiple Classfile processing scenarios, mainly for Class hierarchy information caching. I want to confirm that these are contents of the object, mainly promoted from individual options: - Class Hierarchy information caching - Custom attribute processing - A set of default options, propagated to individual processing (may be overridden classfile-wise) What else is planned for the context object currently? Please don't hesitate to propose. Thanks! Chen Liang -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Fri May 19 09:12:42 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Fri, 19 May 2023 09:12:42 +0000 Subject: Convenient methods to build switch and try/finally in Classfile API Message-ID: Hi, Classfile API provides convenient methods for if/then/else or try/catch, however similar methods to build switch or try/finally are missing. Here I would like to propose new CodeBuilder conveniences: * lookupswitch(Consumer switchHandler) * tableswitch(Consumer switchHandler) * tryWithFinalizer(Consumer tryHandler, Consumer finalizerHandler, Label... externalLabels) * tryWithFinalizer(Consumer tryHandler, Consumer finalizerHandler, boolean compactForm, Label... externalLabels) and new CodeBuilder.SwitchBuilder interface with methods: * switchCase(int caseValue, Consumer caseHandler) * switchCase(List caseValues, Consumer caseHandler) * defaultCase(Consumer defaultHandler) Sample use case (more can be seen in the tests at https://github.com/openjdk/jdk/pull/14056/files): cob.lookupswitch(swb -> swb .switchCase(1, b -> b.iinc(0,1).goto_(b.breakLabel())) .switchCase(2, b -> b.iinc(0,2).goto_(b.breakLabel())) .switchCase(3, b -> b.iinc(0,3).goto_(b.breakLabel())) .defaultCase(b -> b.iinc(0,100).goto_(b.breakLabel())) .switchCase(4, b -> b.iinc(0,4).goto_(b.breakLabel())) .switchCase(5, b -> b.iinc(0,5).goto_(b.breakLabel())) .switchCase(6, b -> b.iinc(0,6).goto_(b.breakLabel()))) cob.tryWithFinalizer( tryb -> tryb.goto_(externalLabel1), finb -> finb.constantInstruction(42).ireturn(), externalLabel1); The implementation work is in progress. I?m now fiddling with precise placing of finalizers in all possible situations. Trying to do not miss any exit from try block, do not produce dead code, do not fall to double finalizer execution and all that with appropriate test coverage. The implementation of finalizers now offers two possible modes: ? expanded (as javac does now), where the code flow is unaffected and finalizer block is copied before each exit from the try block ? compact, where identical exits are joined to common copy of the finalizer block (for example all returns are merged into one finalizer and return). Please let me know your comments and suggestions. Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Fri May 19 14:07:55 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 19 May 2023 10:07:55 -0400 Subject: Convenient methods to build switch and try/finally in Classfile API In-Reply-To: References: Message-ID: As I mentioned to Chen Liang, I would really like to place a moratorium on "new convenience methods" for a while, and I'd like to stop doing them one at a time.? Instead, let's gather a list of areas that potentially need improvement, and come back in a more organized way for a subsequent preview. I have no objection to eventually improving the API for both switches and try, but both of these areas have some risk in them, so I don't want to treat these as the same sort of convenience as `aload_0`.? Switches have messy scoping (it's all one big scope, which means we don't create a new locals context the way we do with if/else) and control flow (break, continue). We have an existing trying() construct but it doesn't handle finally.? Having two constructs that look like they correspond to the same language construct may be confusing. (Also, I'm not convinced that our trying() builder handles local variable slot allocation correctly either -- I think the try block does, but I am pretty sure the cache handler doesn't.? But we don't have tests that cover the interaction of the various convenience builders with local allocation. So let's put this on the list of things that need improvement, and meanwhile, continue to shore up basics. On 5/19/2023 5:12 AM, Adam Sotona wrote: > > Hi, > > Classfile API provides convenient methods for if/then/else or > try/catch, however similar methods to build switch or try/finally are > missing. > > Here I would like to propose new|CodeBuilder|conveniences: > > || > > * |lookupswitch(Consumer switchHandler)| > * |tableswitch(Consumer switchHandler)| > * |tryWithFinalizer(Consumer tryHandler, > Consumer finalizerHandler, Label... externalLabels)| > * |tryWithFinalizer(Consumer tryHandler, > Consumer finalizerHandler, boolean compactForm, > Label... externalLabels)| > > and new|CodeBuilder.SwitchBuilder|interface with methods: > > || > > * |switchCase(int caseValue, Consumer caseHandler)| > * |switchCase(List caseValues, Consumer > caseHandler)| > * |defaultCase(Consumer defaultHandler)| > > Sample use case (more can be seen in the tests at > https://github.com/openjdk/jdk/pull/14056/files): > > cob./lookupswitch/(swb -> swb > > ./switchCase/(1, b -> b./iinc/(0,1)./goto_/(b./breakLabel/())) > > ??????? ./switchCase/(2, b -> b./iinc/(0,2)./goto_/(b./breakLabel/())) > > ??????? ./switchCase/(3, b -> b./iinc/(0,3)./goto_/(b./breakLabel/())) > > ??????? ./defaultCase/(b -> b./iinc/(0,100)./goto_/(b./breakLabel/())) > > ??????? ./switchCase/(4, b -> b./iinc/(0,4)./goto_/(b./breakLabel/())) > > ??????? ./switchCase/(5, b -> b./iinc/(0,5)./goto_/(b./breakLabel/())) > > ??????? ./switchCase/(6, b -> b./iinc/(0,6)./goto_/(b./breakLabel/()))) > > cob.tryWithFinalizer( > ??????? tryb -> tryb.goto_(externalLabel1), > ??????? finb -> finb.constantInstruction(42).ireturn(), > ??????? externalLabel1); > > The implementation work is in progress. I?m now fiddling with precise > placing of finalizers in all possible situations. Trying to do not > miss any exit from try block, do not produce dead code, do not fall to > double finalizer execution and all that with appropriate test coverage. > > The implementation of finalizers now offers two possible modes: > > ?expanded (as javac does now), where the code flow is unaffected and > finalizer block is copied before each exit from the try block > > ?compact, where identical exits are joined to common copy of the > finalizer block (for example all returns are merged into one finalizer > and return). > > Please let me know your comments and suggestions. > > Thanks, > > Adam > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Fri May 19 14:15:40 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 19 May 2023 10:15:40 -0400 Subject: Planned features of a classfile context object In-Reply-To: References: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> Message-ID: I agree that much of what such a context would manage is options.? But I think we can also agree that the current treatment, where we stash Options in the CPBuilder, isn't right either. I can buy that sometimes we want to parse with different options than we generate with.? We can add this in as as goal for rationalizing the classfile context.? I suspect also that part of the problem is that our options are too granular: "process debug" is just a boolean, which could be broken down further. As a not-entirely-serious suggestion: ??? enum ProcessDebugOptions implements Option { ??????? DROP_DEBUG, DROP_DEBUG_ON_READ, DROP_DEBUG_ON_WRITE; ??? } Originally all our options were booleans, and we've moved past that structurally, but haven't really reexamined all the options we defined. I think the problem with "can't transform with different options" stems from the fact that transform lives only on the model, not on the context.? This would let you do what you want as: ??? ClassModel cm = CC.of(opts1).parse(bytes); ??? CC.of(opts2).transform(cm, transform); ??? CC.of(opts3).transform(cm, transform); This derives from the principle that all classfile entities are immutable and can be used in any context; no need to re-parse. On 5/19/2023 3:52 AM, Adam Sotona wrote: > > I think our current Classfile.Option is already not so far from the > proposed global context holder approach. > > The most significant use cases to persist (class hierarchy resolution > and custom attributes) can be very well cached by keeping the > Classfile.Option instances and re-using them as many times as user needs. > > var options = List.of(Option.classHierarchyResolver(resolver), > > ??????????????????????Option.attributeMapper(mapper)); > > Classfile.build(clsDesc, options, handler); > > Classfile.parse(bytes, options); //here we have only vararg now > > What I?m missing much much more than ?global options holder? is > per-transformation options (or even option change for transformed > method). > > For example I would like to use one class model to transform into > multiple classes with different options, and I have to write: > > Classfile.parse(bytes, options1).transform(classTransform1); > > Classfile.parse(bytes, options2).transform(classTransform2); > > Instead of simple: > > var clm = Classfile.parse(bytes); > > clm.transform(classTransform1, options1); > > clm.transform(classTransform2, options2); > > Or maybe: > > var clm = Classfile.parse(bytes, options1); > > clm.transform(classTransform1); > > clm.setOptions(options2); > > clm.transform(classTransform2); > > However it raises the main question about what options are really tied > to model parsing (I think it is only attributeMapper). > > And what options can be safely changed later. For example can I change > processDebug, processLineNumbers or processUnknownAttributes on > already parsed class model to get different results of the following > model traversal? > > And there are majority of options unrelated to the parsed class model > but affecting only building and transformations (generateStackmap, > constantPoolSharing, fixShortJumps, patchDeadCode, > classHierarchyResolver, filterDeadLabels). > > I think we should make more clear which options affect what actions > (parsing, building, transformation) and give more flexibility to > change them on the fly. > > Thanks, > > Adam > > *From: *classfile-api-dev on > behalf of Brian Goetz > *Date: *Thursday, 18 May 2023 16:16 > *To: *liangchenblue at gmail.com , > classfile-api-dev > *Subject: *Re: Planned features of a classfile context object > > Currently the ConstantPoolBuilder is another place we we attach > options.? This was more of a sin of convenience than anything else; > the theory was that if we are transforming a class, we'll parse it > with a set of options, and then generate a new class which shares its > CP, and the options came along for the ride. > > Searching for a better name that ClassfileContext.? The name Classfile > is too good to waste, but it doesn't represent a classfile (that's > ClassModel), so calling the new thing Classfile would likely be > weird.? Also ClassfileContext would make it harder to discover the key > entry points (build/parse).? Classfile.Context is better in that it is > more discoverable at least. > > Classfile{Parser,Generator} are more discoverable but only tell half > the story, and there's no obvious "first half" of the story.? > ClassfileReaderWriter is discoverable and honest but long.? > Classfile{Broker,Manager,Mediator} sound like parodies of the Design > Patterns era. > > A slightly cheeky but possibly viable option is "Classfiles"; while > not a collection of classfiles, it is a collection of behaviors > _about_ classfiles. > > ??? Classfiles.of(options).parse(bytes) > ??? Classfiles.of(options).generate(handler) > > > > > On 5/17/2023 11:21 AM, Brian Goetz wrote: > > These are basically what is on my list, though I had in mind to > move _all_ option information to the context, and leave it out of > individual processing decisions. > > On 5/17/2023 9:59 AM, - wrote: > > Hi, > > In the discussions a few weeks ago, we envisioned a Classfile context > > object shared across multiple Classfile processing scenarios, mainly > > for Class hierarchy information caching. > > I want to confirm that these are contents of the object, mainly > > promoted from individual options: > > - Class Hierarchy information caching > > - Custom attribute processing > > - A set of default options, propagated to individual processing (may > > be overridden classfile-wise) > > What else is planned for the context object currently? Please don't > > hesitate to propose. Thanks! > > Chen Liang > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Fri May 19 14:41:57 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Fri, 19 May 2023 14:41:57 +0000 Subject: Convenient methods to build switch and try/finally in Classfile API In-Reply-To: References: Message-ID: OK, let?s postpone these. From: Brian Goetz Date: Friday, 19 May 2023 16:08 To: Adam Sotona , classfile-api-dev at openjdk.org Subject: Re: Convenient methods to build switch and try/finally in Classfile API As I mentioned to Chen Liang, I would really like to place a moratorium on "new convenience methods" for a while, and I'd like to stop doing them one at a time. Instead, let's gather a list of areas that potentially need improvement, and come back in a more organized way for a subsequent preview. I have no objection to eventually improving the API for both switches and try, but both of these areas have some risk in them, so I don't want to treat these as the same sort of convenience as `aload_0`. Switches have messy scoping (it's all one big scope, which means we don't create a new locals context the way we do with if/else) and control flow (break, continue). We have an existing trying() construct but it doesn't handle finally. Having two constructs that look like they correspond to the same language construct may be confusing. (Also, I'm not convinced that our trying() builder handles local variable slot allocation correctly either -- I think the try block does, but I am pretty sure the cache handler doesn't. But we don't have tests that cover the interaction of the various convenience builders with local allocation. So let's put this on the list of things that need improvement, and meanwhile, continue to shore up basics. On 5/19/2023 5:12 AM, Adam Sotona wrote: Hi, Classfile API provides convenient methods for if/then/else or try/catch, however similar methods to build switch or try/finally are missing. Here I would like to propose new CodeBuilder conveniences: 1. lookupswitch(Consumer switchHandler) 2. tableswitch(Consumer switchHandler) 3. tryWithFinalizer(Consumer tryHandler, Consumer finalizerHandler, Label... externalLabels) 4. tryWithFinalizer(Consumer tryHandler, Consumer finalizerHandler, boolean compactForm, Label... externalLabels) and new CodeBuilder.SwitchBuilder interface with methods: 1. switchCase(int caseValue, Consumer caseHandler) 2. switchCase(List caseValues, Consumer caseHandler) 3. defaultCase(Consumer defaultHandler) Sample use case (more can be seen in the tests at https://github.com/openjdk/jdk/pull/14056/files): cob.lookupswitch(swb -> swb .switchCase(1, b -> b.iinc(0,1).goto_(b.breakLabel())) .switchCase(2, b -> b.iinc(0,2).goto_(b.breakLabel())) .switchCase(3, b -> b.iinc(0,3).goto_(b.breakLabel())) .defaultCase(b -> b.iinc(0,100).goto_(b.breakLabel())) .switchCase(4, b -> b.iinc(0,4).goto_(b.breakLabel())) .switchCase(5, b -> b.iinc(0,5).goto_(b.breakLabel())) .switchCase(6, b -> b.iinc(0,6).goto_(b.breakLabel()))) cob.tryWithFinalizer( tryb -> tryb.goto_(externalLabel1), finb -> finb.constantInstruction(42).ireturn(), externalLabel1); The implementation work is in progress. I?m now fiddling with precise placing of finalizers in all possible situations. Trying to do not miss any exit from try block, do not produce dead code, do not fall to double finalizer execution and all that with appropriate test coverage. The implementation of finalizers now offers two possible modes: 1. expanded (as javac does now), where the code flow is unaffected and finalizer block is copied before each exit from the try block 2. compact, where identical exits are joined to common copy of the finalizer block (for example all returns are merged into one finalizer and return). Please let me know your comments and suggestions. Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Fri May 19 15:40:56 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Fri, 19 May 2023 15:40:56 +0000 Subject: Planned features of a classfile context object In-Reply-To: References: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> Message-ID: I like the idea when the whole ?context? is out of the ClassModel, so using it with a different context will do the job differently. However doesn?t it mean they are rather mutable, when you get different content based on context? Some of the options already behave that way and are quite flexible (processDebug, generateStackmap, constantPoolSharing, fixShortJumps, patchDeadCode, classHierarchyResolver, filterDeadLabels) and some can be fixed to do so (processLineNumbers). However for example attributeMapper is closely tied with the parsed model and it would require to re-parse when. Also the last used context would have to be stored with the model to know if any reset of already expanded model is needed (for example did we skipped parsing line numbers or are they just missing?) Another confusion we are already facing is when the actual option become effective. For example generateStackmap(false) does not implicitly mean they will be dropped by the transformation (unless we expand each Code attribute). The same for processDebug, processLineNumbers and patchDeadCode. While DROP_DEBUG_ON_READ is clearly an option, DROP_DEBUG_ON_WRITE suppose to be rather a CodeTransform, because by setting the option we do not say it will be effective. All attributes attached to code have very specific position, as we must expand the whole code when we want just filter them out. To your example I?m not quite sure what to expect from: ClassModel cm = CC.of(DROP_DEBUG).parse(bytes); CC.of(do not DROP_DEBUG).transform(cm, transform); From: Brian Goetz Date: Friday, 19 May 2023 16:15 To: Adam Sotona , liangchenblue at gmail.com , classfile-api-dev Subject: Re: Planned features of a classfile context object I agree that much of what such a context would manage is options. But I think we can also agree that the current treatment, where we stash Options in the CPBuilder, isn't right either. I can buy that sometimes we want to parse with different options than we generate with. We can add this in as as goal for rationalizing the classfile context. I suspect also that part of the problem is that our options are too granular: "process debug" is just a boolean, which could be broken down further. As a not-entirely-serious suggestion: enum ProcessDebugOptions implements Option { DROP_DEBUG, DROP_DEBUG_ON_READ, DROP_DEBUG_ON_WRITE; } Originally all our options were booleans, and we've moved past that structurally, but haven't really reexamined all the options we defined. I think the problem with "can't transform with different options" stems from the fact that transform lives only on the model, not on the context. This would let you do what you want as: ClassModel cm = CC.of(opts1).parse(bytes); CC.of(opts2).transform(cm, transform); CC.of(opts3).transform(cm, transform); This derives from the principle that all classfile entities are immutable and can be used in any context; no need to re-parse. On 5/19/2023 3:52 AM, Adam Sotona wrote: I think our current Classfile.Option is already not so far from the proposed global context holder approach. The most significant use cases to persist (class hierarchy resolution and custom attributes) can be very well cached by keeping the Classfile.Option instances and re-using them as many times as user needs. var options = List.of(Option.classHierarchyResolver(resolver), Option.attributeMapper(mapper)); Classfile.build(clsDesc, options, handler); Classfile.parse(bytes, options); //here we have only vararg now What I?m missing much much more than ?global options holder? is per-transformation options (or even option change for transformed method). For example I would like to use one class model to transform into multiple classes with different options, and I have to write: Classfile.parse(bytes, options1).transform(classTransform1); Classfile.parse(bytes, options2).transform(classTransform2); Instead of simple: var clm = Classfile.parse(bytes); clm.transform(classTransform1, options1); clm.transform(classTransform2, options2); Or maybe: var clm = Classfile.parse(bytes, options1); clm.transform(classTransform1); clm.setOptions(options2); clm.transform(classTransform2); However it raises the main question about what options are really tied to model parsing (I think it is only attributeMapper). And what options can be safely changed later. For example can I change processDebug, processLineNumbers or processUnknownAttributes on already parsed class model to get different results of the following model traversal? And there are majority of options unrelated to the parsed class model but affecting only building and transformations (generateStackmap, constantPoolSharing, fixShortJumps, patchDeadCode, classHierarchyResolver, filterDeadLabels). I think we should make more clear which options affect what actions (parsing, building, transformation) and give more flexibility to change them on the fly. Thanks, Adam From: classfile-api-dev on behalf of Brian Goetz Date: Thursday, 18 May 2023 16:16 To: liangchenblue at gmail.com , classfile-api-dev Subject: Re: Planned features of a classfile context object Currently the ConstantPoolBuilder is another place we we attach options. This was more of a sin of convenience than anything else; the theory was that if we are transforming a class, we'll parse it with a set of options, and then generate a new class which shares its CP, and the options came along for the ride. Searching for a better name that ClassfileContext. The name Classfile is too good to waste, but it doesn't represent a classfile (that's ClassModel), so calling the new thing Classfile would likely be weird. Also ClassfileContext would make it harder to discover the key entry points (build/parse). Classfile.Context is better in that it is more discoverable at least. Classfile{Parser,Generator} are more discoverable but only tell half the story, and there's no obvious "first half" of the story. ClassfileReaderWriter is discoverable and honest but long. Classfile{Broker,Manager,Mediator} sound like parodies of the Design Patterns era. A slightly cheeky but possibly viable option is "Classfiles"; while not a collection of classfiles, it is a collection of behaviors _about_ classfiles. Classfiles.of(options).parse(bytes) Classfiles.of(options).generate(handler) On 5/17/2023 11:21 AM, Brian Goetz wrote: These are basically what is on my list, though I had in mind to move _all_ option information to the context, and leave it out of individual processing decisions. On 5/17/2023 9:59 AM, - wrote: Hi, In the discussions a few weeks ago, we envisioned a Classfile context object shared across multiple Classfile processing scenarios, mainly for Class hierarchy information caching. I want to confirm that these are contents of the object, mainly promoted from individual options: - Class Hierarchy information caching - Custom attribute processing - A set of default options, propagated to individual processing (may be overridden classfile-wise) What else is planned for the context object currently? Please don't hesitate to propose. Thanks! Chen Liang -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Fri May 19 16:07:32 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 19 May 2023 12:07:32 -0400 Subject: Planned features of a classfile context object In-Reply-To: References: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> Message-ID: <4c8e8b9a-2691-2a2a-fbd3-7d15324c6ac4@oracle.com> > I like the idea when the whole ?context? is out of the ClassModel, so > using it with a different context will do the job differently. However > doesn?t it mean they are rather mutable, when you get different > content based on context? > It means certain operations are, well, contextual.? If you ask a context to generate a class, it will do so with the options of that context.? So if you generate the same class twice with different contexts, you get different results. It also means some entities might drag parts of their creating context with them.? For example, suppose we parse a classfile using context A, which has a mapper for attribute "Foo".? Then we adapt it using context B, that doens't know about Foo.? Well, B may not know about Foo, but the FooAttribute will have a reference to its creating AttributeMapper.? That's fine too (well, if different contexts have radically different ideas of how a FooAttribute is laid out, maybe not, but in that case we have bigger problems.) > Some of the options already behave that way and are quite flexible > (processDebug, generateStackmap, constantPoolSharing, fixShortJumps, > patchDeadCode, classHierarchyResolver, filterDeadLabels) and some can > be fixed to do so (processLineNumbers). > I think its been a while since we took a good look at our options and asked "is this the way we want it"; now is a good time to review this. > However for example attributeMapper is closely tied with the parsed > model and it would require to re-parse when. Also the last used > context would have to be stored with the model to know if any reset of > already expanded model is needed (for example did we skipped parsing > line numbers or are they just missing?) > I think this is a place where we have to lean on existing design constraints of backward compatibility.? We can parse the attribute with the mapper found at parse time, and we can write it with either that mapper or the one we find at generation time. > Another confusion we are already facing is when the actual option > become effective. > > For example generateStackmap(false) does not implicitly mean they will > be dropped by the transformation (unless we expand each Code > attribute). The same for processDebug, processLineNumbers and > patchDeadCode. > Correct, this is part of what I mean by the "let's review our options and make sure they mean what we think, and write that down" above. > While DROP_DEBUG_ON_READis clearly an option, > DROP_DEBUG_ON_WRITEsuppose to be rather a CodeTransform, because by > setting the option we do not say it will be effective. > > All attributes attached to code have very specific position, as we > must expand the whole code when we want just filter them out. > > To your example I?m not quite sure what to expect from: > > ??? ClassModel cm = CC.of(DROP_DEBUG).parse(bytes); > ??? CC.of(do not DROP_DEBUG).transform(cm, transform); > So, what happens here is that on reading, we drop the debug information, so the ClassModel has no debug information.? When we use that as a source, there's no debug information in the source but any debug information added by the transform is not dropped. > *From: *Brian Goetz > *Date: *Friday, 19 May 2023 16:15 > *To: *Adam Sotona , liangchenblue at gmail.com > , classfile-api-dev > > *Subject: *Re: Planned features of a classfile context object > > I agree that much of what such a context would manage is options.? But > I think we can also agree that the current treatment, where we stash > Options in the CPBuilder, isn't right either. > > I can buy that sometimes we want to parse with different options than > we generate with.? We can add this in as as goal for rationalizing the > classfile context.? I suspect also that part of the problem is that > our options are too granular: "process debug" is just a boolean, which > could be broken down further.? As a not-entirely-serious suggestion: > > ??? enum ProcessDebugOptions implements Option { > ??????? DROP_DEBUG, DROP_DEBUG_ON_READ, DROP_DEBUG_ON_WRITE; > ??? } > > Originally all our options were booleans, and we've moved past that > structurally, but haven't really reexamined all the options we defined. > > I think the problem with "can't transform with different options" > stems from the fact that transform lives only on the model, not on the > context.? This would let you do what you want as: > > ??? ClassModel cm = CC.of(opts1).parse(bytes); > ??? CC.of(opts2).transform(cm, transform); > ??? CC.of(opts3).transform(cm, transform); > > This derives from the principle that all classfile entities are > immutable and can be used in any context; no need to re-parse. > > > > On 5/19/2023 3:52 AM, Adam Sotona wrote: > > I think our current Classfile.Option is already not so far from > the proposed global context holder approach. > > The most significant use cases to persist (class hierarchy > resolution and custom attributes) can be very well cached by > keeping the Classfile.Option instances and re-using them as many > times as user needs. > > var options = List.of(Option.classHierarchyResolver(resolver), > > ??????????????????????Option.attributeMapper(mapper)); > > Classfile.build(clsDesc, options, handler); > > Classfile.parse(bytes, options); //here we have only vararg now > > What I?m missing much much more than ?global options holder? is > per-transformation options (or even option change for transformed > method). > > For example I would like to use one class model to transform into > multiple classes with different options, and I have to write: > > Classfile.parse(bytes, options1).transform(classTransform1); > > Classfile.parse(bytes, options2).transform(classTransform2); > > Instead of simple: > > var clm = Classfile.parse(bytes); > > clm.transform(classTransform1, options1); > > clm.transform(classTransform2, options2); > > Or maybe: > > var clm = Classfile.parse(bytes, options1); > > clm.transform(classTransform1); > > clm.setOptions(options2); > > clm.transform(classTransform2); > > However it raises the main question about what options are really > tied to model parsing (I think it is only attributeMapper). > > And what options can be safely changed later. For example can I > change processDebug, processLineNumbers or > processUnknownAttributes on already parsed class model to get > different results of the following model traversal? > > And there are majority of options unrelated to the parsed class > model but affecting only building and transformations > (generateStackmap, constantPoolSharing, fixShortJumps, > patchDeadCode, classHierarchyResolver, filterDeadLabels). > > I think we should make more clear which options affect what > actions (parsing, building, transformation) and give more > flexibility to change them on the fly. > > Thanks, > > Adam > > *From: *classfile-api-dev > on behalf of Brian > Goetz > *Date: *Thursday, 18 May 2023 16:16 > *To: *liangchenblue at gmail.com > > , classfile-api-dev > > *Subject: *Re: Planned features of a classfile context object > > Currently the ConstantPoolBuilder is another place we we attach > options.? This was more of a sin of convenience than anything > else; the theory was that if we are transforming a class, we'll > parse it with a set of options, and then generate a new class > which shares its CP, and the options came along for the ride. > > Searching for a better name that ClassfileContext.? The name > Classfile is too good to waste, but it doesn't represent a > classfile (that's ClassModel), so calling the new thing Classfile > would likely be weird.? Also ClassfileContext would make it harder > to discover the key entry points (build/parse).? Classfile.Context > is better in that it is more discoverable at least. > > Classfile{Parser,Generator} are more discoverable but only tell > half the story, and there's no obvious "first half" of the story.? > ClassfileReaderWriter is discoverable and honest but long.? > Classfile{Broker,Manager,Mediator} sound like parodies of the > Design Patterns era. > > A slightly cheeky but possibly viable option is "Classfiles"; > while not a collection of classfiles, it is a collection of > behaviors _about_ classfiles. > > ??? Classfiles.of(options).parse(bytes) > ??? Classfiles.of(options).generate(handler) > > > > > > On 5/17/2023 11:21 AM, Brian Goetz wrote: > > These are basically what is on my list, though I had in mind > to move _all_ option information to the context, and leave it > out of individual processing decisions. > > On 5/17/2023 9:59 AM, - wrote: > > Hi, > > In the discussions a few weeks ago, we envisioned a Classfile context > > object shared across multiple Classfile processing scenarios, mainly > > for Class hierarchy information caching. > > > > I want to confirm that these are contents of the object, mainly > > promoted from individual options: > > - Class Hierarchy information caching > > - Custom attribute processing > > - A set of default options, propagated to individual processing (may > > be overridden classfile-wise) > > > > What else is planned for the context object currently? Please don't > > hesitate to propose. Thanks! > > > > Chen Liang > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Fri May 19 16:41:08 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 19 May 2023 12:41:08 -0400 Subject: Planned features of a classfile context object In-Reply-To: References: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> Message-ID: <093f8cc9-72b6-d23e-2e92-202ec83eb3ed@oracle.com> A few more words on this suggestion.? As of Java 21, we have integrated treatment of enums and sealed types in switch, so you can freely mix: ??? sealed interface Option { } ??? enum FooOption implements Option { FOO_YES, FOO_NO; } ??? record BarOption(int maxBars) implements Option { } so that options representing "constants" could be handled as enums rather than records, and options representing data holders (e.g., attribute mappers) are records, and the two can be mixed. The use site can do: ??? ClassfileContext cc = CC.of(FooOption.FOO_YES, new BarOption(7)) and a switch over Option can mix enums and record patterns. It is probably worth considering recasting some boolean-holder options as enums. On 5/19/2023 11:40 AM, Adam Sotona wrote: > > ??? enum ProcessDebugOptions implements Option { > ??????? DROP_DEBUG, DROP_DEBUG_ON_READ, DROP_DEBUG_ON_WRITE; > ??? } -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.sandoz at oracle.com Fri May 19 23:16:44 2023 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 19 May 2023 23:16:44 +0000 Subject: Convenient methods to build switch and try/finally in Classfile API In-Reply-To: References: Message-ID: On May 19, 2023, at 7:07 AM, Brian Goetz wrote: As I mentioned to Chen Liang, I would really like to place a moratorium on "new convenience methods" for a while, and I'd like to stop doing them one at a time. Instead, let's gather a list of areas that potentially need improvement, and come back in a more organized way for a subsequent preview. I have no objection to eventually improving the API for both switches and try, but both of these areas have some risk in them, so I don't want to treat these as the same sort of convenience as `aload_0`. Switches have messy scoping (it's all one big scope, which means we don't create a new locals context the way we do with if/else) and control flow (break, continue). We have an existing trying() construct but it doesn't handle finally. Having two constructs that look like they correspond to the same language construct may be confusing. (Also, I'm not convinced that our trying() builder handles local variable slot allocation correctly either -- I think the try block does, but I am pretty sure the cache handler doesn't. But we don't have tests that cover the interaction of the various convenience builders with local allocation. Also, it can get complex for nested try with finally, where we need to keep track of the layering and know where exit points ?break?/?continue? to within that layering so as to know what finalizer code to inline in the required order. It?s probably the most complex lowering step that the java compiler does. Paul. So let's put this on the list of things that need improvement, and meanwhile, continue to shore up basics. On 5/19/2023 5:12 AM, Adam Sotona wrote: Hi, Classfile API provides convenient methods for if/then/else or try/catch, however similar methods to build switch or try/finally are missing. Here I would like to propose new CodeBuilder conveniences: * lookupswitch(Consumer switchHandler) * tableswitch(Consumer switchHandler) * tryWithFinalizer(Consumer tryHandler, Consumer finalizerHandler, Label... externalLabels) * tryWithFinalizer(Consumer tryHandler, Consumer finalizerHandler, boolean compactForm, Label... externalLabels) and new CodeBuilder.SwitchBuilder interface with methods: * switchCase(int caseValue, Consumer caseHandler) * switchCase(List caseValues, Consumer caseHandler) * defaultCase(Consumer defaultHandler) Sample use case (more can be seen in the tests at https://github.com/openjdk/jdk/pull/14056/files): cob.lookupswitch(swb -> swb .switchCase(1, b -> b.iinc(0,1).goto_(b.breakLabel())) .switchCase(2, b -> b.iinc(0,2).goto_(b.breakLabel())) .switchCase(3, b -> b.iinc(0,3).goto_(b.breakLabel())) .defaultCase(b -> b.iinc(0,100).goto_(b.breakLabel())) .switchCase(4, b -> b.iinc(0,4).goto_(b.breakLabel())) .switchCase(5, b -> b.iinc(0,5).goto_(b.breakLabel())) .switchCase(6, b -> b.iinc(0,6).goto_(b.breakLabel()))) cob.tryWithFinalizer( tryb -> tryb.goto_(externalLabel1), finb -> finb.constantInstruction(42).ireturn(), externalLabel1); The implementation work is in progress. I?m now fiddling with precise placing of finalizers in all possible situations. Trying to do not miss any exit from try block, do not produce dead code, do not fall to double finalizer execution and all that with appropriate test coverage. The implementation of finalizers now offers two possible modes: ? expanded (as javac does now), where the code flow is unaffected and finalizer block is copied before each exit from the try block ? compact, where identical exits are joined to common copy of the finalizer block (for example all returns are merged into one finalizer and return). Please let me know your comments and suggestions. Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Sun May 21 04:55:38 2023 From: liangchenblue at gmail.com (-) Date: Sat, 20 May 2023 23:55:38 -0500 Subject: Convenient methods to build switch and try/finally in Classfile API In-Reply-To: References: Message-ID: These high-level utilities aren't likely used by the JDK itself (more startup dependencies) or javac (most likely last candidate to migrate to Classfile API) in the short run. The first goal of Classfile API is to replace ASM, and these detailed improvements are indeed out-of-scope. And to match up the behavior and specification of a Classfile-API try-finally with a javac one would probably involve extra work from the compiler team as well. I think we might focus on more primitive works first, since we can see if core libraries are eligible to migrate after the recent performance improvements. On Fri, May 19, 2023 at 6:16?PM Paul Sandoz wrote: > > > > On May 19, 2023, at 7:07 AM, Brian Goetz wrote: > > As I mentioned to Chen Liang, I would really like to place a moratorium on "new convenience methods" for a while, and I'd like to stop doing them one at a time. Instead, let's gather a list of areas that potentially need improvement, and come back in a more organized way for a subsequent preview. > > I have no objection to eventually improving the API for both switches and try, but both of these areas have some risk in them, so I don't want to treat these as the same sort of convenience as `aload_0`. Switches have messy scoping (it's all one big scope, which means we don't create a new locals context the way we do with if/else) and control flow (break, continue). > > We have an existing trying() construct but it doesn't handle finally. Having two constructs that look like they correspond to the same language construct may be confusing. (Also, I'm not convinced that our trying() builder handles local variable slot allocation correctly either -- I think the try block does, but I am pretty sure the cache handler doesn't. But we don't have tests that cover the interaction of the various convenience builders with local allocation. > > > Also, it can get complex for nested try with finally, where we need to keep track of the layering and know where exit points ?break?/?continue? to within that layering so as to know what finalizer code to inline in the required order. It?s probably the most complex lowering step that the java compiler does. > > Paul. > > > So let's put this on the list of things that need improvement, and meanwhile, continue to shore up basics. > > On 5/19/2023 5:12 AM, Adam Sotona wrote: > > Hi, > > Classfile API provides convenient methods for if/then/else or try/catch, however similar methods to build switch or try/finally are missing. > > Here I would like to propose new CodeBuilder conveniences: > > lookupswitch(Consumer switchHandler) > tableswitch(Consumer switchHandler) > tryWithFinalizer(Consumer tryHandler, Consumer finalizerHandler, Label... externalLabels) > tryWithFinalizer(Consumer tryHandler, Consumer finalizerHandler, boolean compactForm, Label... externalLabels) > > > and new CodeBuilder.SwitchBuilder interface with methods: > > switchCase(int caseValue, Consumer caseHandler) > switchCase(List caseValues, Consumer caseHandler) > defaultCase(Consumer defaultHandler) > > > Sample use case (more can be seen in the tests at https://github.com/openjdk/jdk/pull/14056/files): > > cob.lookupswitch(swb -> swb > .switchCase(1, b -> b.iinc(0,1).goto_(b.breakLabel())) > .switchCase(2, b -> b.iinc(0,2).goto_(b.breakLabel())) > .switchCase(3, b -> b.iinc(0,3).goto_(b.breakLabel())) > .defaultCase(b -> b.iinc(0,100).goto_(b.breakLabel())) > .switchCase(4, b -> b.iinc(0,4).goto_(b.breakLabel())) > .switchCase(5, b -> b.iinc(0,5).goto_(b.breakLabel())) > .switchCase(6, b -> b.iinc(0,6).goto_(b.breakLabel()))) > > > > cob.tryWithFinalizer( > > tryb -> tryb.goto_(externalLabel1), > > finb -> finb.constantInstruction(42).ireturn(), > > externalLabel1); > > > > > The implementation work is in progress. I?m now fiddling with precise placing of finalizers in all possible situations. Trying to do not miss any exit from try block, do not produce dead code, do not fall to double finalizer execution and all that with appropriate test coverage. > The implementation of finalizers now offers two possible modes: > ? expanded (as javac does now), where the code flow is unaffected and finalizer block is copied before each exit from the try block > ? compact, where identical exits are joined to common copy of the finalizer block (for example all returns are merged into one finalizer and return). > > > Please let me know your comments and suggestions. > > Thanks, > Adam > > From adam.sotona at oracle.com Mon May 22 07:57:01 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Mon, 22 May 2023 07:57:01 +0000 Subject: Convenient methods to build switch and try/finally in Classfile API In-Reply-To: References: Message-ID: From: Paul Sandoz Also, it can get complex for nested try with finally, where we need to keep track of the layering and know where exit points ?break?/?continue? to within that layering so as to know what finalizer code to inline in the required order. It?s probably the most complex lowering step that the java compiler does. Paul. Yes, I?m aware of the complexity and layering. In this working prototype I tried to test the idea that all returns and exceptions thrown in try block are finalized automatically, while all break labels must be explicitly declared by user to be recognized and all related branch instructions pre-pended by appropriate finalizer. Implementation by code transformation of the try block is chainable, so any outer try/finally block transforms it again to correctly pre-pend its own finalizer block. I?m trying to find and test appropriate Classfile API solution to make future javac conversion easier. Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Mon May 22 14:09:31 2023 From: liangchenblue at gmail.com (-) Date: Mon, 22 May 2023 09:09:31 -0500 Subject: Bug in attribute writing In-Reply-To: References: Message-ID: For writing, I don't think unknown attributes are ever created when they are presented in the first place; so we can assume "support unknown attributes" is always true. A tricky part of unknown attribute handling is that AttributedElement::attributes() return value may contain null if unknown attributes aren't supported. This should be explicitly specified. The bound type annotation attributes indeed may not properly be rewritten when its dependencies (exception handlers, labels, parameter indices, etc.) are updated. Should users be advised to track these changes to attributes on their own? I think we can alleviate such dependencies by exposing pointers in the model: for instance, we will use Label instead of int for startPc for LocalVariableTable; the catch table index in type annotation targets can refer to ExceptionCatch. Then, in the writeDirect checks, we propagate checks to the components: if the dependencies of the Labels or ExceptionCatchs are the same (much like how a constant pool is still the same), we will perform direct write; otherwise we perform a clean write. Chen Liang On Thu, May 18, 2023 at 9:34?AM Brian Goetz wrote: > > I just noticed that in BoundAttribute.BoundUnknownAttribute: > > private void checkWriteSupported(Function condition) { > if (!condition.apply(classReader)) > throw new UnsupportedOperationException("Write of unknown attribute " + attributeName() + " not supported to alien constant pool"); > } > > we should at least be checking the option "support unknown attributes" before we say OK. > > Overall, I am not sure we have a fully consistent story for unsupported attributes here. The basic problem with attributes for which we don't have an AttributeMapper is that we can't be sure about validity. We don't know where the CP indexes are (though if we are sharing CPs, this is OK), and we don't know what other data might be off. > > (We also have a bigger problem with the type annotations attributes, which are known to have all sorts of non-CP offsets (nth bound of mth type variable, nth exception, indexes into code array, etc) which could easily but thrown off by transformation and there is no practical way to detect whether the original RVTA is still valid.) From brian.goetz at oracle.com Mon May 22 14:32:24 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 22 May 2023 10:32:24 -0400 Subject: Bug in attribute writing In-Reply-To: References: Message-ID: <27f1e876-0a58-353e-5fca-6a0f20303ef6@oracle.com> On 5/22/2023 10:09 AM, - wrote: > For writing, I don't think unknown attributes are ever created when > they are presented in the first place; so we can assume "support > unknown attributes" is always true. Not quite sure what you're saying here? > > A tricky part of unknown attribute handling is that > AttributedElement::attributes() return value may contain null if > unknown attributes aren't supported. This should be explicitly > specified. Or perhaps unknown attributes should be removed from the list and the size adjusted, as if the attribute were not present.? Having a null in this list likely helps no one. > The bound type annotation attributes indeed may not properly be > rewritten when its dependencies (exception handlers, labels, parameter > indices, etc.) are updated. Should users be advised to track these > changes to attributes on their own? Sigh, type annotations are the bane of my existence.? There are really no good answers here. Most attributes contain indexes into the CP, but express little other dependency on the structure of the classfile.? As a result, we can use "is there a shared CP" as a proxy for "is it OK to write an attribute from one classfile to another".? This is true for all the standard attributes except the type annotation ones, which contain indexes into the code array, exception table, signature attribute ("mth bound of nth type parameter"), etc.? Tracking whether a transform would undermine these indexes is probably nearly impossible, and even if it were, the cost would be orders of magnitude out of line.? So we have a few bad choices: ?- silently drop TA attributes on the floor when adapting; ?- faithfully write presented TA attributes even though they might be invalid; ?- insist the user unpack and repack the attribute. The "unpack" option is just make-work that won't do much to improve the accuracy of the result; much of what's in the TA attributes is of interest to static compilers, not bytecode processors.? Silently dropping the attribute seems rude.? So the least-bad alternative is to just believe the user that this attribute is valid enough for use at runtime. I'd like a fourth option, which maybe involves some sort of classfile processing option to give us better guidance, but so far we are not sure what is best here. > I think we can alleviate such dependencies by exposing pointers in the > model: for instance, we will use Label instead of int for startPc for > LocalVariableTable; the catch table index in type annotation targets > can refer to ExceptionCatch. Then, in the writeDirect checks, we > propagate checks to the components: if the dependencies of the Labels > or ExceptionCatchs are the same (much like how a constant pool is > still the same), we will perform direct write; otherwise we perform a > clean write. There are two kinds of references that are virtualized by the library: CP indexes and offsets (labels).? The underlying goal is that a classfile element from one classfile can be correctly interpreted in another.? For CP indexes, if the two classfile share a CP, then CP indexes are already good; if not, they contain enough information to be copied at write time.? For labels, labels have a meaning specific to the containing Code attribute / CodeBuilder, so taht the same label can be correctly interpreted in both contexts. We do not virtualize anything else. We could try to do the same for the exception table; this is worth exploring, since we've definitely encountered bugs related to the table being reordered before.? But for TA attributes, this eventually runs into the absurd, because it contains all sorts of exotic pointers like "mth bound of nth type parameter". Further, virtualization of more quantities starts to exhibit poor returns, because we want to be able to minimize the number of "unpack/repack" operations when doing a trivial transform.? We can't help but do so for labels, but if we had to do so for everything else, the costs would likely spiral out of control. Zooming out, there's a real lesson here: the type annotation attributes violated an implicit design principle for attributes, and now everyone downstream is paying the cost.? But I am diffident about imposing more of these costs on every user of every bytecode-processing framework for the sake of incrementally better TA fidelity. Where we are now is a compromise; do you see a better compromise? > > Chen Liang > > On Thu, May 18, 2023 at 9:34?AM Brian Goetz wrote: >> I just noticed that in BoundAttribute.BoundUnknownAttribute: >> >> private void checkWriteSupported(Function condition) { >> if (!condition.apply(classReader)) >> throw new UnsupportedOperationException("Write of unknown attribute " + attributeName() + " not supported to alien constant pool"); >> } >> >> we should at least be checking the option "support unknown attributes" before we say OK. >> >> Overall, I am not sure we have a fully consistent story for unsupported attributes here. The basic problem with attributes for which we don't have an AttributeMapper is that we can't be sure about validity. We don't know where the CP indexes are (though if we are sharing CPs, this is OK), and we don't know what other data might be off. >> >> (We also have a bigger problem with the type annotations attributes, which are known to have all sorts of non-CP offsets (nth bound of mth type variable, nth exception, indexes into code array, etc) which could easily but thrown off by transformation and there is no practical way to detect whether the original RVTA is still valid.) From liangchenblue at gmail.com Mon May 22 16:43:03 2023 From: liangchenblue at gmail.com (-) Date: Mon, 22 May 2023 11:43:03 -0500 Subject: Bug in attribute writing In-Reply-To: <27f1e876-0a58-353e-5fca-6a0f20303ef6@oracle.com> References: <27f1e876-0a58-353e-5fca-6a0f20303ef6@oracle.com> Message-ID: On Mon, May 22, 2023 at 9:32?AM Brian Goetz wrote: > > > > On 5/22/2023 10:09 AM, - wrote: > > For writing, I don't think unknown attributes are ever created when > > they are presented in the first place; so we can assume "support > > unknown attributes" is always true. > > Not quite sure what you're saying here? > I mean, the check for whether unknown attributes are enabled is redundant, as unknown attributes won't be created if it's not enabled, and if it's provided to the wrong class builder, the current check always rejects it, too. > > Sigh, type annotations are the bane of my existence. There are really > no good answers here. > > Most attributes contain indexes into the CP, but express little other > dependency on the structure of the classfile. As a result, we can use > "is there a shared CP" as a proxy for "is it OK to write an attribute > from one classfile to another". This is true for all the standard > attributes except the type annotation ones, which contain indexes into > the code array, exception table, signature attribute ("mth bound of nth > type parameter"), etc. Tracking whether a transform would undermine > these indexes is probably nearly impossible, and even if it were, the > cost would be orders of magnitude out of line. So we have a few bad > choices: > > - silently drop TA attributes on the floor when adapting; > - faithfully write presented TA attributes even though they might be > invalid; > - insist the user unpack and repack the attribute. > > The "unpack" option is just make-work that won't do much to improve the > accuracy of the result; much of what's in the TA attributes is of > interest to static compilers, not bytecode processors. Silently > dropping the attribute seems rude. So the least-bad alternative is to > just believe the user that this attribute is valid enough for use at > runtime. > > I'd like a fourth option, which maybe involves some sort of classfile > processing option to give us better guidance, but so far we are not sure > what is best here. A fourth option already exists in the Classfile API: we already process some attributes closely tied to others as an integral unit instead of separately, such as Code-StackMapTable, constant pool-BootstrapMethods. Maybe we can try singling out each element of a type annotation structure and bind it to its respective parent (exception catch, local variable, signature etc.) during processing. If a piece of information the annotation is trying to apply to does not exist, we can simply leave it and not transform or drop it (configured by classfile options), since it wouldn't be useful in the beginning. > > > I think we can alleviate such dependencies by exposing pointers in the > > model: for instance, we will use Label instead of int for startPc for > > LocalVariableTable; the catch table index in type annotation targets > > can refer to ExceptionCatch. Then, in the writeDirect checks, we > > propagate checks to the components: if the dependencies of the Labels > > or ExceptionCatchs are the same (much like how a constant pool is > > still the same), we will perform direct write; otherwise we perform a > > clean write. > > There are two kinds of references that are virtualized by the library: > CP indexes and offsets (labels). The underlying goal is that a > classfile element from one classfile can be correctly interpreted in > another. For CP indexes, if the two classfile share a CP, then CP > indexes are already good; if not, they contain enough information to be > copied at write time. For labels, labels have a meaning specific to the > containing Code attribute / CodeBuilder, so taht the same label can be > correctly interpreted in both contexts. We do not virtualize anything else. > > We could try to do the same for the exception table; this is worth > exploring, since we've definitely encountered bugs related to the table > being reordered before. But for TA attributes, this eventually runs > into the absurd, because it contains all sorts of exotic pointers like > "mth bound of nth type parameter". > > Further, virtualization of more quantities starts to exhibit poor > returns, because we want to be able to minimize the number of > "unpack/repack" operations when doing a trivial transform. We can't > help but do so for labels, but if we had to do so for everything else, > the costs would likely spiral out of control. Binding type annotation entries to the application sites would be another type of virtualization, which I think is more acceptable. > > Zooming out, there's a real lesson here: the type annotation attributes > violated an implicit design principle for attributes, and now everyone > downstream is paying the cost. But I am diffident about imposing more > of these costs on every user of every bytecode-processing framework for > the sake of incrementally better TA fidelity. > > Where we are now is a compromise; do you see a better compromise? Thinking about custom attributes, I think such a dependency (linking with indices) would, in fact, be fairly common: 1. Class-files are constant, indice links are reliable; 2. Attribute formats are predetermined, changing preexisting attributes' format is almost impossible. We thus might need some sort of attribute-linking features, where one attribute's updates can be propagated to a downstream attribute. I would envision that to be defined by the attribute mapper, which probably has already expanded to become a handler by that point. But this whole thing looks like an overkill; I am looking for a more simple solution as well. > > > > Chen Liang > > > > On Thu, May 18, 2023 at 9:34?AM Brian Goetz wrote: > >> I just noticed that in BoundAttribute.BoundUnknownAttribute: > >> > >> private void checkWriteSupported(Function condition) { > >> if (!condition.apply(classReader)) > >> throw new UnsupportedOperationException("Write of unknown attribute " + attributeName() + " not supported to alien constant pool"); > >> } > >> > >> we should at least be checking the option "support unknown attributes" before we say OK. > >> > >> Overall, I am not sure we have a fully consistent story for unsupported attributes here. The basic problem with attributes for which we don't have an AttributeMapper is that we can't be sure about validity. We don't know where the CP indexes are (though if we are sharing CPs, this is OK), and we don't know what other data might be off. > >> > >> (We also have a bigger problem with the type annotations attributes, which are known to have all sorts of non-CP offsets (nth bound of mth type variable, nth exception, indexes into code array, etc) which could easily but thrown off by transformation and there is no practical way to detect whether the original RVTA is still valid.) > From brian.goetz at oracle.com Mon May 22 16:50:29 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 22 May 2023 12:50:29 -0400 Subject: Bug in attribute writing In-Reply-To: References: <27f1e876-0a58-353e-5fca-6a0f20303ef6@oracle.com> Message-ID: <9d9d6538-306f-b65d-516a-7c37862e2c94@oracle.com> >> I'd like a fourth option, which maybe involves some sort of classfile >> processing option to give us better guidance, but so far we are not sure >> what is best here. > A fourth option already exists in the Classfile API: we already > process some attributes closely tied to others as an integral unit > instead of separately, such as Code-StackMapTable, constant > pool-BootstrapMethods. Maybe we can try singling out each element of a > type annotation structure and bind it to its respective parent > (exception catch, local variable, signature etc.) during processing. > If a piece of information the annotation is trying to apply to does > not exist, we can simply leave it and not transform or drop it > (configured by classfile options), since it wouldn't be useful in the > beginning. So, this is viable (at some cost) but only for the subset of indexes that are routinely touched by classfile processing -- code array offsets and exception table offsets.? Others, such as "mth bound of nth type parameter" are strictly about signature attributes, which are (currently) opaque to the API.? So if we did this, we'd be trading incremental fidelity against incremental cost, but getting to full fidelity would be a heavy lift. One of the costs of the model we've chosen is that it is somewhat hard to detect whether a transformation (or a delimited portion of it, such as on a method of a class) is the identity operation.? We do so with the Code attribute (if the code array is unchanged, we can reuse the stack map) but this is ad-hoc and expensive (it just happens to be less expensive than recomputing stack maps.)? But doing this more generally would have a significant cost that we have not yet paid. From liangchenblue at gmail.com Tue May 23 01:49:41 2023 From: liangchenblue at gmail.com (-) Date: Mon, 22 May 2023 20:49:41 -0500 Subject: Benchmarks to check migration impact to j.l.invoke and j.l.reflect.Proxy performance Message-ID: Hello Adam, Last time, you said that a major obstacle to the migration of Proxy and MethodHandle ASM code to Classfile API is the performance degradation. I wish you can share how you measured the impact of migration, so that I can try for myself and see the performance bottlenecks. Chen Liang From adam.sotona at oracle.com Tue May 23 13:38:36 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Tue, 23 May 2023 13:38:36 +0000 Subject: Benchmarks to check migration impact to j.l.invoke and j.l.reflect.Proxy performance In-Reply-To: References: Message-ID: Hi, For benchmarking Proxies I?ve used https://github.com/openjdk/jdk/tree/master/test/micro/org/openjdk/bench/java/lang/reflect/Proxy Adam From: liangchenblue at gmail.com Date: Tuesday, 23 May 2023 3:49 To: classfile-api-dev , asotona at openjdk.org Subject: Benchmarks to check migration impact to j.l.invoke and j.l.reflect.Proxy performance Hello Adam, Last time, you said that a major obstacle to the migration of Proxy and MethodHandle ASM code to Classfile API is the performance degradation. I wish you can share how you measured the impact of migration, so that I can try for myself and see the performance bottlenecks. Chen Liang -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Wed May 24 04:39:06 2023 From: liangchenblue at gmail.com (-) Date: Tue, 23 May 2023 23:39:06 -0500 Subject: Benchmarks to check migration impact to j.l.invoke and j.l.reflect.Proxy performance In-Reply-To: References: Message-ID: Thank you for the info. I've run the ProxyPerf after edits (removing obsolete code) and the result doesn't look quite promising. It appears that the exception handler might be time-consuming. I've just found Claes Redestad's talk about lambda and bytecode generation performance: https://youtu.be/3r_tHGtpU7A?t=1921 I think I should perform a debug build one day and discover the impact of string-related operations. My current assumption is that to find a corresponding ClassEntry in a constant pool, a ClassDesc's descriptor string must undergo substring and hashCode, which may be costly, so we might change ClassEntry's hashing strategy so we can just reuse the cached hashCode of a ClassDesc's descriptorString easily. On Tue, May 23, 2023 at 8:38?AM Adam Sotona wrote: > > Hi, > > For benchmarking Proxies I?ve used https://github.com/openjdk/jdk/tree/master/test/micro/org/openjdk/bench/java/lang/reflect/Proxy > > > > Adam > > > > From: liangchenblue at gmail.com > Date: Tuesday, 23 May 2023 3:49 > To: classfile-api-dev , asotona at openjdk.org > Subject: Benchmarks to check migration impact to j.l.invoke and j.l.reflect.Proxy performance > > Hello Adam, > Last time, you said that a major obstacle to the migration of Proxy > and MethodHandle ASM code to Classfile API is the performance > degradation. I wish you can share how you measured the impact of > migration, so that I can try for myself and see the performance > bottlenecks. > > Chen Liang From adam.sotona at oracle.com Wed May 24 11:37:47 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Wed, 24 May 2023 11:37:47 +0000 Subject: Benchmarks to check migration impact to j.l.invoke and j.l.reflect.Proxy performance In-Reply-To: References: Message-ID: Yes, calculation of hash of class internal name from its descriptor without sub-stringing might help to save some CPU, however it must be benchmarked. Another (a bit more radical) approach in this specific case is to use CP entries as constants instead of the ClassDescs and MethodTypeDescs. Even more CPU can be saved if the constant CP entries are a part of a constant ?template? ClassModel and specific proxy classes are generated as the ?template? class transformation. From: liangchenblue at gmail.com Date: Wednesday, 24 May 2023 6:39 To: Adam Sotona Cc: classfile-api-dev , asotona at openjdk.org Subject: Re: Benchmarks to check migration impact to j.l.invoke and j.l.reflect.Proxy performance Thank you for the info. I've run the ProxyPerf after edits (removing obsolete code) and the result doesn't look quite promising. It appears that the exception handler might be time-consuming. I've just found Claes Redestad's talk about lambda and bytecode generation performance: https://youtu.be/3r_tHGtpU7A?t=1921 I think I should perform a debug build one day and discover the impact of string-related operations. My current assumption is that to find a corresponding ClassEntry in a constant pool, a ClassDesc's descriptor string must undergo substring and hashCode, which may be costly, so we might change ClassEntry's hashing strategy so we can just reuse the cached hashCode of a ClassDesc's descriptorString easily. On Tue, May 23, 2023 at 8:38?AM Adam Sotona wrote: > > Hi, > > For benchmarking Proxies I?ve used https://github.com/openjdk/jdk/tree/master/test/micro/org/openjdk/bench/java/lang/reflect/Proxy > > > > Adam > > > > From: liangchenblue at gmail.com > Date: Tuesday, 23 May 2023 3:49 > To: classfile-api-dev , asotona at openjdk.org > Subject: Benchmarks to check migration impact to j.l.invoke and j.l.reflect.Proxy performance > > Hello Adam, > Last time, you said that a major obstacle to the migration of Proxy > and MethodHandle ASM code to Classfile API is the performance > degradation. I wish you can share how you measured the impact of > migration, so that I can try for myself and see the performance > bottlenecks. > > Chen Liang -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Wed May 24 12:18:20 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Wed, 24 May 2023 12:18:20 +0000 Subject: CSR for Class-File API (Preview) Message-ID: Hi, I?ve just kicked off CSR JDK-8308754 Class-File API (Preview) to have enough time for discussions and improvements. Javadoc at https://cr.openjdk.org/~asotona/8308754/classfile-api/javadoc/java.base/java/lang/classfile/package-summary.html is altered to show target java.lang.classfile package and will be updated according to our current main codebase. No branch nor fork is created for the transition and all API or javadoc changes are expected to go to the current codebase in the jdk main under jdk.internal.classfile package. Please let me know your comments. Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Wed May 24 13:05:27 2023 From: liangchenblue at gmail.com (-) Date: Wed, 24 May 2023 08:05:27 -0500 Subject: CSR for Class-File API (Preview) In-Reply-To: References: Message-ID: First, congratulations on the progress of Classfile API! There are two outstanding issues that I see for now. 1. The design of Classfiles (context) object, where options are shared; and some options might be localized to individual codebuilder/classbuilder etc. 2. Many of the API endpoints are missing specifications, such as missing @sealedGraph tag. I would be glad to provide or improve documentation for the Classfile API. In addition, what's the rationale for putting all these new contents under java.lang instead of java, like java.classfile vs java.lang.classfile? Chen Liang On Wed, May 24, 2023 at 7:18?AM Adam Sotona wrote: > > Hi, > > I?ve just kicked off CSR JDK-8308754 Class-File API (Preview) to have enough time for discussions and improvements. > > Javadoc at https://cr.openjdk.org/~asotona/8308754/classfile-api/javadoc/java.base/java/lang/classfile/package-summary.html is altered to show target java.lang.classfile package and will be updated according to our current main codebase. > > No branch nor fork is created for the transition and all API or javadoc changes are expected to go to the current codebase in the jdk main under jdk.internal.classfile package. > > > > Please let me know your comments. > > > > Thanks, > > Adam > > > > > > > > > > > > From adam.sotona at oracle.com Wed May 24 13:42:21 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Wed, 24 May 2023 13:42:21 +0000 Subject: [External] : Re: CSR for Class-File API (Preview) In-Reply-To: References: Message-ID: Changes in the API and Javadoc are expected, that is why the Javadoc is not attached to the CSR. I propose all API and Javadoc changes to track as subtasks of (or link to) the CSR for transparency. Target package java.lang.classfile is also subject for discussion. Thanks, Adam From: liangchenblue at gmail.com Date: Wednesday, 24 May 2023 15:06 To: Adam Sotona Cc: classfile-api-dev at openjdk.org Subject: [External] : Re: CSR for Class-File API (Preview) First, congratulations on the progress of Classfile API! There are two outstanding issues that I see for now. 1. The design of Classfiles (context) object, where options are shared; and some options might be localized to individual codebuilder/classbuilder etc. 2. Many of the API endpoints are missing specifications, such as missing @sealedGraph tag. I would be glad to provide or improve documentation for the Classfile API. In addition, what's the rationale for putting all these new contents under java.lang instead of java, like java.classfile vs java.lang.classfile? Chen Liang On Wed, May 24, 2023 at 7:18?AM Adam Sotona wrote: > > Hi, > > I?ve just kicked off CSR JDK-8308754 Class-File API (Preview) to have enough time for discussions and improvements. > > Javadoc at https://cr.openjdk.org/~asotona/8308754/classfile-api/javadoc/java.base/java/lang/classfile/package-summary.html is altered to show target java.lang.classfile package and will be updated according to our current main codebase. > > No branch nor fork is created for the transition and all API or javadoc changes are expected to go to the current codebase in the jdk main under jdk.internal.classfile package. > > > > Please let me know your comments. > > > > Thanks, > > Adam > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Wed May 24 15:57:34 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 24 May 2023 11:57:34 -0400 Subject: CSR for Class-File API (Preview) In-Reply-To: References: Message-ID: > In addition, what's the rationale for putting all these new contents > under java.lang instead of java, like java.classfile vs > java.lang.classfile? Largely historical.? Imagine "lang" meant "platform"; this is where low-level platform classes have historically gone (e.g., java.lang.invoke, java.lang.constant, neither of which have anything to do with the language.) From liangchenblue at gmail.com Wed May 24 16:19:38 2023 From: liangchenblue at gmail.com (-) Date: Wed, 24 May 2023 11:19:38 -0500 Subject: CSR for Class-File API (Preview) In-Reply-To: References: Message-ID: I think lang package makes sense then. In addition, foreign API is also in lang, while its interactions with the rest of lang package is less obvious. Classfile API is already used by reflect and invoke content, so I think putting them in parallel makes sense. Also, after the move, will the old jdk.internal.classfile.impl package simply become jdk.internal.classfile, taking place of the original API package? On Wed, May 24, 2023, 10:57 AM Brian Goetz wrote: > > > In addition, what's the rationale for putting all these new contents > > under java.lang instead of java, like java.classfile vs > > java.lang.classfile? > > Largely historical. Imagine "lang" meant "platform"; this is where > low-level platform classes have historically gone (e.g., > java.lang.invoke, java.lang.constant, neither of which have anything to > do with the language.) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ebruneton at free.fr Wed May 24 16:30:40 2023 From: ebruneton at free.fr (ebruneton at free.fr) Date: Wed, 24 May 2023 18:30:40 +0200 Subject: CSR for Class-File API (Preview) In-Reply-To: References: Message-ID: The current formulation of the CSR seems to imply that the API will fully solve the "Version skew between frameworks and the running JDK" problem. I think it will help mitigate this problem, but that it won't fully solve it. A new JDK might a) change the semantics of existing elements (e.g. ACC_SUPER), b) remove elements which were previously valid (e.g. JSR/RET), c) add new elements (e.g. bootstrap methods, new descriptor syntax, etc), Hence a framework written for a previous JDK version, using the ClassFile API, might a) produce code which has an unexpected semantics in the new version, b) crash or produce code which is invalid for the new version, c) fail or crash due to unexpected inputs (either right away if the user code is "clean" and checks for every possible unexpected input case, or later and in possibly hard to debug ways otherwise), when run without any change on a newer JDK with its "up-to-date classfile library". See also the issues raised by Brian in https://mail.openjdk.org/pipermail/classfile-api-dev/2022-July/000053.html. E.g. "you should not be parsing (let alone instrumenting) Java 19 classfiles if you don?t understand the semantics of the attributes defined for Java 19 classfiles.": I think this applies to a framework using the ClassFile API as well, if this API existed in Java 18. In other words, I think that even with the ClassFile API, frameworks which are not explicitly updated to work with a new JDK version (which can take some time) will result "in runtime errors or, worse, frameworks trying to [transform] class files from the future and engaging in leaps of faith that nothing too serious has changed." If this analysis is correct, is there a way to reformulate this "Version skew between frameworks and the running JDK" section to make it clear that the proposed API will not fully solve this problem? Eric > Hi, > > I've just kicked off CSR JDK-8308754 Class-File API (Preview) to have > enough time for discussions and improvements. > > Javadoc at > https://cr.openjdk.org/~asotona/8308754/classfile-api/javadoc/java.base/java/lang/classfile/package-summary.html > is altered to show target java.lang.classfile package and will be > updated according to our current main codebase. > > No branch nor fork is created for the transition and all API or javadoc > changes are expected to go to the current codebase in the jdk main > under jdk.internal.classfile package. > > Please let me know your comments. > > Thanks, > > Adam From liangchenblue at gmail.com Wed May 24 16:55:04 2023 From: liangchenblue at gmail.com (-) Date: Wed, 24 May 2023 11:55:04 -0500 Subject: CSR for Class-File API (Preview) In-Reply-To: References: Message-ID: I think the Classfile API has a few approaches to mitigate your issue, with the latest API on latest JDK model: On Wed, May 24, 2023, 11:30 AM wrote: > The current formulation of the CSR seems to imply that the API will > fully solve the "Version skew between frameworks and the running JDK" > problem. I think it will help mitigate this problem, but that it won't > fully solve it. A new JDK might > a) change the semantics of existing elements (e.g. ACC_SUPER), > AccessFlag takes care of it, and can potentially ignore/fail on invalid flags. > b) remove elements which were previously valid (e.g. JSR/RET), > A specification of major version can be used to reject invalid entries > c) add new elements (e.g. bootstrap methods, new descriptor syntax, > etc), > These are parsed by classfile API properly; if users encounter them, they fail like enum switch encountering new enums. > Hence a framework written for a previous JDK version, using the > ClassFile API, might > a) produce code which has an unexpected semantics in the new version, > Only if you don't configure your major version > b) crash or produce code which is invalid for the new version, > Same as above. > c) fail or crash due to unexpected inputs (either right away if the > user code is "clean" and checks for every possible unexpected input > case, or later and in possibly hard to debug ways otherwise), > It may, but the Classfile API can drastically decrease the failure chance if, say, it's just adding an annotation. > when run without any change on a newer JDK with its "up-to-date > classfile library". > > See also the issues raised by Brian in > https://mail.openjdk.org/pipermail/classfile-api-dev/2022-July/000053.html. > > E.g. "you should not be parsing (let alone instrumenting) Java 19 > classfiles if you don?t understand the semantics of the attributes > defined for Java 19 classfiles.": I think this applies to a framework > using the ClassFile API as well, if this API existed in Java 18. > > In other words, I think that even with the ClassFile API, frameworks > which are not explicitly updated to work with a new JDK version (which > can take some time) will result "in runtime errors or, worse, frameworks > trying to [transform] class files from the future and engaging in leaps > of faith that nothing too serious has changed." > The classfile API still can address most of these problems, such as handling new CP entries (otherwise unknown size) when everything else doesn't change. It parses Classfile into something more understandable to users and more resistant to format updates. And it allows older code to fail fast on incompatibilities of new class files, which is a significant improvement from the existing situation in terms of compatibility and correctness. > > If this analysis is correct, is there a way to reformulate this "Version > skew between frameworks and the running JDK" section to make it clear > that the proposed API will not fully solve this problem? > > Eric > > > Hi, > > > > I've just kicked off CSR JDK-8308754 Class-File API (Preview) to have > > enough time for discussions and improvements. > > > > Javadoc at > > > https://cr.openjdk.org/~asotona/8308754/classfile-api/javadoc/java.base/java/lang/classfile/package-summary.html > > is altered to show target java.lang.classfile package and will be > > updated according to our current main codebase. > > > > No branch nor fork is created for the transition and all API or javadoc > > changes are expected to go to the current codebase in the jdk main > > under jdk.internal.classfile package. > > > > Please let me know your comments. > > > > Thanks, > > > > Adam > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Wed May 24 20:07:51 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 24 May 2023 16:07:51 -0400 Subject: CSR for Class-File API (Preview) In-Reply-To: References: Message-ID: > The current formulation of the CSR seems to imply that the API will > fully solve the "Version skew between frameworks and the running JDK" > problem. I think it will help mitigate this problem, but that it won't > fully solve it. A new JDK might > ?a) change the semantics of existing elements (e.g. ACC_SUPER), > ?b) remove elements which were previously valid (e.g. JSR/RET), > ?c) add new elements (e.g. bootstrap methods, new descriptor syntax, > etc), > Hence a framework written for a previous JDK version, using the > ClassFile API, might > ?a) produce code which has an unexpected semantics in the new version, > ?b) crash or produce code which is invalid for the new version, > ?c) fail or crash due to unexpected inputs (either right away if the > user code is "clean" and checks for every possible unexpected input > case, or later and in possibly hard to debug ways otherwise), > when run without any change on a newer JDK with its "up-to-date > classfile library". While I can't characterize all future changes to the classfile specification, and clearly all bets would be off if the `aaload` bytecode went the way of jsr/ret, there is an approach that frameworks can use to mitigate the problem you outline.? The key choice a framework here is whether to preserve the classfile version when transforming, or whether to use a version the framework knows about.? There are cases when the sensible thing to do is preserve the version, such as if you're only dropping the LineNumberTable. (And yes, if at a later version of the spec made LNT required, this would be a problem.)? There are also cases where the sensible thing to do is stick to a known version of the classfile format.? What this does is let the framework author trade off between "fail at instrumentation time" and "fail at class load time". > > See also the issues raised by Brian in > https://mail.openjdk.org/pipermail/classfile-api-dev/2022-July/000053.html. > E.g. "you should not be parsing (let alone instrumenting) Java 19 > classfiles if you don?t understand the semantics of the attributes > defined for Java 19 classfiles.": I think this applies to a framework > using the ClassFile API as well, if this API existed in Java 18. I agree that without a characterization of all possible future JVMS Ch4 changes, we can't claim the problem is solved.? What we can add to the story, over and above the previous scenario, is that there is now an intermediary (the JDK) that understands the newer classfile format, and can translate it into an object model that is at least partially understood by the framework. From ebruneton at free.fr Thu May 25 16:32:38 2023 From: ebruneton at free.fr (ebruneton at free.fr) Date: Thu, 25 May 2023 18:32:38 +0200 Subject: CSR for Class-File API (Preview) In-Reply-To: References: Message-ID: liangchenblue at gmail.com and brian.goetz at oracle.com wrote: - "I think the Classfile API has a few approaches to mitigate your issue, with the latest API on latest JDK model:" - "A specification of major version can be used to reject invalid entries" - "if users encounter them, they fail like enum switch encountering new enums." - "Only if you don't configure your major version" - "The classfile API still can address most of these problems," - "I agree that without a characterization of all possible future JVMS Ch4 changes, we can't claim the problem is solved." As I said "I think [the ClassFile API] will help mitigate this problem ["Version skew between frameworks and the running JDK"], but that it won't fully solve it.". So its seems we agree on this. To reflect this, can the end of this CSR section be rewritten to something like "... that nothing too serious has changed. Providing a class-file library as part of the JDK, always up-to-date with the new JDK features, can mitigate most of these version skew problems."? Eric PS: failing fast by checking the major version is what ASM currently does. If frameworks do this on top of the ClassFile API, they don't gain much with respect to this "version skew" problem (i.e. a possibly long delay between a JDK release and the point where frameworks and applications work with the new class file format). >> The current formulation of the CSR seems to imply that the API will >> fully solve the "Version skew between frameworks and the running JDK" >> problem. I think it will help mitigate this problem, but that it won't >> fully solve it. A new JDK might >> ?a) change the semantics of existing elements (e.g. ACC_SUPER), >> ?b) remove elements which were previously valid (e.g. JSR/RET), >> ?c) add new elements (e.g. bootstrap methods, new descriptor syntax, >> etc), >> Hence a framework written for a previous JDK version, using the >> ClassFile API, might >> ?a) produce code which has an unexpected semantics in the new version, >> ?b) crash or produce code which is invalid for the new version, >> ?c) fail or crash due to unexpected inputs (either right away if the >> user code is "clean" and checks for every possible unexpected input >> case, or later and in possibly hard to debug ways otherwise), >> when run without any change on a newer JDK with its "up-to-date >> classfile library". > > While I can't characterize all future changes to the classfile > specification, and clearly all bets would be off if the `aaload` > bytecode went the way of jsr/ret, there is an approach that frameworks > can use to mitigate the problem you outline.? The key choice a > framework here is whether to preserve the classfile version when > transforming, or whether to use a version the framework knows about.? > There are cases when the sensible thing to do is preserve the version, > such as if you're only dropping the LineNumberTable. (And yes, if at a > later version of the spec made LNT required, this would be a problem.)? > There are also cases where the sensible thing to do is stick to a known > version of the classfile format.? What this does is let the framework > author trade off between "fail at instrumentation time" and "fail at > class load time". > >> >> See also the issues raised by Brian in >> https://mail.openjdk.org/pipermail/classfile-api-dev/2022-July/000053.html. >> E.g. "you should not be parsing (let alone instrumenting) Java 19 >> classfiles if you don?t understand the semantics of the attributes >> defined for Java 19 classfiles.": I think this applies to a framework >> using the ClassFile API as well, if this API existed in Java 18. > > I agree that without a characterization of all possible future JVMS Ch4 > changes, we can't claim the problem is solved.? What we can add to the > story, over and above the previous scenario, is that there is now an > intermediary (the JDK) that understands the newer classfile format, and > can translate it into an object model that is at least partially > understood by the framework. From brian.goetz at oracle.com Thu May 25 16:39:12 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 25 May 2023 12:39:12 -0400 Subject: CSR for Class-File API (Preview) In-Reply-To: References: Message-ID: > > To reflect this, can the end of this CSR section be rewritten to > something like "... that nothing too serious has changed. Providing a > class-file library as part of the JDK, always up-to-date with the new > JDK features, can mitigate most of these version skew problems."? I don't have any objection to refining the message like this.? (This should happen in the JEP, though, not the CSR, because that's what most people will read.) From adam.sotona at oracle.com Thu May 25 18:02:05 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Thu, 25 May 2023 18:02:05 +0000 Subject: Planned features of a classfile context object In-Reply-To: <093f8cc9-72b6-d23e-2e92-202ec83eb3ed@oracle.com> References: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> <093f8cc9-72b6-d23e-2e92-202ec83eb3ed@oracle.com> Message-ID: I?m trying to figure out how the context will be passed down to the models (if it suppose to be detached from the models). Do we plan to enforce another CC argument to all models methods requiring context? Or do we plan to strip down all context-sensitive methods from all models and attach them to a kind of context-wrappers? For example codeModel.forEach(Classfile.Context.of(?), ?) or Classfile.Context.of(?).forEach(codeModel, ?) ? Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Thu May 25 18:31:32 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 25 May 2023 14:31:32 -0400 Subject: Planned features of a classfile context object In-Reply-To: References: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> <093f8cc9-72b6-d23e-2e92-202ec83eb3ed@oracle.com> Message-ID: A model created within a context can retain a reference to its creating context.? (We do a lot of this sort of thing already, where MethodModel retains a reference to ClassModel, and where CPBuilder retains a reference to Options.)? So for operations that require a context, a model derived from a classfile can use the context that created it. Methods like parse/build/transform will be moved to the classfile context object. On 5/25/2023 2:02 PM, Adam Sotona wrote: > > I?m trying to figure out how the context will be passed down to the > models (if it suppose to be detached from the models). > > Do we plan to enforce another CC argument to all models methods > requiring context? > > Or do we plan to strip down all context-sensitive methods from all > models and attach them to a kind of context-wrappers? > > For example codeModel.forEach(Classfile.Context.of(?), ?) or > Classfile.Context.of(?).forEach(codeModel, ?) ? > > Thanks, > > Adam > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Fri May 26 15:18:18 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Fri, 26 May 2023 15:18:18 +0000 Subject: Planned features of a classfile context object In-Reply-To: References: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> <093f8cc9-72b6-d23e-2e92-202ec83eb3ed@oracle.com> Message-ID: This is a raw prototype of Classfile.Context implementation and the new options: https://github.com/openjdk/jdk/pull/14180/files https://github.com/openjdk/jdk/blob/4456ca78bc67138930dc313f71aba58ef1cf5f13/src/java.base/share/classes/jdk/internal/classfile/Classfile.java It is far from finished, however before going further I would appreciate your comments (mainly on options naming). Thanks, Adam From: Brian Goetz Date: Thursday, 25 May 2023 20:31 To: Adam Sotona , liangchenblue at gmail.com , classfile-api-dev Subject: Re: Planned features of a classfile context object A model created within a context can retain a reference to its creating context. (We do a lot of this sort of thing already, where MethodModel retains a reference to ClassModel, and where CPBuilder retains a reference to Options.) So for operations that require a context, a model derived from a classfile can use the context that created it. Methods like parse/build/transform will be moved to the classfile context object. On 5/25/2023 2:02 PM, Adam Sotona wrote: I?m trying to figure out how the context will be passed down to the models (if it suppose to be detached from the models). Do we plan to enforce another CC argument to all models methods requiring context? Or do we plan to strip down all context-sensitive methods from all models and attach them to a kind of context-wrappers? For example codeModel.forEach(Classfile.Context.of(?), ?) or Classfile.Context.of(?).forEach(codeModel, ?) ? Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Fri May 26 15:21:22 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 26 May 2023 11:21:22 -0400 Subject: Planned features of a classfile context object In-Reply-To: References: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> <093f8cc9-72b6-d23e-2e92-202ec83eb3ed@oracle.com> Message-ID: Thanks Adam.? I'll have a look. As a quick observation, for purposes of discussion and review, I might consider separating out the option refactor from the Classfile context refactor; the two seem almost completely orthogonal. On 5/26/2023 11:18 AM, Adam Sotona wrote: > > This is a raw prototype of Classfile.Context implementation and the > new options: > > https://github.com/openjdk/jdk/pull/14180/files > > https://github.com/openjdk/jdk/blob/4456ca78bc67138930dc313f71aba58ef1cf5f13/src/java.base/share/classes/jdk/internal/classfile/Classfile.java > > It is far from finished, however before going further I would > appreciate your comments (mainly on options naming). > > Thanks, > > Adam > > *From: *Brian Goetz > *Date: *Thursday, 25 May 2023 20:31 > *To: *Adam Sotona , liangchenblue at gmail.com > , classfile-api-dev > > *Subject: *Re: Planned features of a classfile context object > > A model created within a context can retain a reference to its > creating context.? (We do a lot of this sort of thing already, where > MethodModel retains a reference to ClassModel, and where CPBuilder > retains a reference to Options.)? So for operations that require a > context, a model derived from a classfile can use the context that > created it. > > Methods like parse/build/transform will be moved to the classfile > context object. > > On 5/25/2023 2:02 PM, Adam Sotona wrote: > > I?m trying to figure out how the context will be passed down to > the models (if it suppose to be detached from the models). > > Do we plan to enforce another CC argument to all models methods > requiring context? > > Or do we plan to strip down all context-sensitive methods from all > models and attach them to a kind of context-wrappers? > > For example codeModel.forEach(Classfile.Context.of(?), ?) or > Classfile.Context.of(?).forEach(codeModel, ?) ? > > Thanks, > > Adam > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Fri May 26 15:30:32 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Fri, 26 May 2023 15:30:32 +0000 Subject: Planned features of a classfile context object In-Reply-To: References: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> <093f8cc9-72b6-d23e-2e92-202ec83eb3ed@oracle.com> Message-ID: Yes, I was considering to split it, however both require refactoring across the whole codebase and they conflict on the same source lines. It is near impossible to work on them in parallel PRs. From: Brian Goetz Date: Friday, 26 May 2023 17:21 To: Adam Sotona , liangchenblue at gmail.com , classfile-api-dev Subject: Re: Planned features of a classfile context object Thanks Adam. I'll have a look. As a quick observation, for purposes of discussion and review, I might consider separating out the option refactor from the Classfile context refactor; the two seem almost completely orthogonal. On 5/26/2023 11:18 AM, Adam Sotona wrote: This is a raw prototype of Classfile.Context implementation and the new options: https://github.com/openjdk/jdk/pull/14180/files https://github.com/openjdk/jdk/blob/4456ca78bc67138930dc313f71aba58ef1cf5f13/src/java.base/share/classes/jdk/internal/classfile/Classfile.java It is far from finished, however before going further I would appreciate your comments (mainly on options naming). Thanks, Adam From: Brian Goetz Date: Thursday, 25 May 2023 20:31 To: Adam Sotona , liangchenblue at gmail.com , classfile-api-dev Subject: Re: Planned features of a classfile context object A model created within a context can retain a reference to its creating context. (We do a lot of this sort of thing already, where MethodModel retains a reference to ClassModel, and where CPBuilder retains a reference to Options.) So for operations that require a context, a model derived from a classfile can use the context that created it. Methods like parse/build/transform will be moved to the classfile context object. On 5/25/2023 2:02 PM, Adam Sotona wrote: I?m trying to figure out how the context will be passed down to the models (if it suppose to be detached from the models). Do we plan to enforce another CC argument to all models methods requiring context? Or do we plan to strip down all context-sensitive methods from all models and attach them to a kind of context-wrappers? For example codeModel.forEach(Classfile.Context.of(?), ?) or Classfile.Context.of(?).forEach(codeModel, ?) ? Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Fri May 26 21:15:57 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 26 May 2023 17:15:57 -0400 Subject: Planned features of a classfile context object In-Reply-To: References: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> <093f8cc9-72b6-d23e-2e92-202ec83eb3ed@oracle.com> Message-ID: <69bf8797-1c16-2d20-1464-4c725e30dec9@oracle.com> Some review comments on your patch: ?- I'd prefer for all of the static methods in Classfile to go away, though I can imagine choosing to leave the most basic version (no options) of build/parse around as conveniences.? If we leave these, the Javadoc should make it clear that these are equivalent to Context.of().parse(...). ?- I am OK with calling the context class "Classfile" for the time being; this makes the diff easier to follow as it makes it clear that the static methods are being promoted to instance methods, and we can discuss name later. ?- I see you cache the Options in BufWriterImpl/BufferedCodeBuilder/etc, but we should probably be caching the full context (impl) object instead, and fetching options out of the context. ?- (Syntax nit) Some enum options are fully descriptive of what they are (DROP_DEBUG_ELEMENTS), and some are implicit (ALWAYS_GENERATE -- generate what?).? I think I would prefer to err on the side of fully descriptive (and people can use static imports to drop the qualifier if they like), but we should be consistent.? There should be a convention of what the default is (like "first is always default") for options that are enums. On 5/26/2023 11:18 AM, Adam Sotona wrote: > > This is a raw prototype of Classfile.Context implementation and the > new options: > > https://github.com/openjdk/jdk/pull/14180/files > > https://github.com/openjdk/jdk/blob/4456ca78bc67138930dc313f71aba58ef1cf5f13/src/java.base/share/classes/jdk/internal/classfile/Classfile.java > > It is far from finished, however before going further I would > appreciate your comments (mainly on options naming). > > Thanks, > > Adam > > *From: *Brian Goetz > *Date: *Thursday, 25 May 2023 20:31 > *To: *Adam Sotona , liangchenblue at gmail.com > , classfile-api-dev > > *Subject: *Re: Planned features of a classfile context object > > A model created within a context can retain a reference to its > creating context.? (We do a lot of this sort of thing already, where > MethodModel retains a reference to ClassModel, and where CPBuilder > retains a reference to Options.)? So for operations that require a > context, a model derived from a classfile can use the context that > created it. > > Methods like parse/build/transform will be moved to the classfile > context object. > > On 5/25/2023 2:02 PM, Adam Sotona wrote: > > I?m trying to figure out how the context will be passed down to > the models (if it suppose to be detached from the models). > > Do we plan to enforce another CC argument to all models methods > requiring context? > > Or do we plan to strip down all context-sensitive methods from all > models and attach them to a kind of context-wrappers? > > For example codeModel.forEach(Classfile.Context.of(?), ?) or > Classfile.Context.of(?).forEach(codeModel, ?) ? > > Thanks, > > Adam > -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Fri May 26 21:53:22 2023 From: liangchenblue at gmail.com (-) Date: Fri, 26 May 2023 16:53:22 -0500 Subject: Planned features of a classfile context object In-Reply-To: <69bf8797-1c16-2d20-1464-4c725e30dec9@oracle.com> References: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> <093f8cc9-72b6-d23e-2e92-202ec83eb3ed@oracle.com> <69bf8797-1c16-2d20-1464-4c725e30dec9@oracle.com> Message-ID: The Classfile context in the patch looks somewhat like a Classfile element to me :) With that in mind, I wish we can allow users to transform/fork these immutable contexts, such as skipping line numbers for particular files, skip constantpool sharing for select transformations, etc. with the same code pattern in Classfile Transformers. Of course, a transformation will always read the default values of options if not explicitly set, and the transformed context will have dropped options replaced with their default values. In addition, ClassModel.transform can probably be kept, which means using the creation context of the ClassModel to transform this model, if we make it clear that all models have their lifetime bound to a Classfile object. On Fri, May 26, 2023 at 4:16?PM Brian Goetz wrote: > > Some review comments on your patch: > > - I'd prefer for all of the static methods in Classfile to go away, though I can imagine choosing to leave the most basic version (no options) of build/parse around as conveniences. If we leave these, the Javadoc should make it clear that these are equivalent to Context.of().parse(...). > - I am OK with calling the context class "Classfile" for the time being; this makes the diff easier to follow as it makes it clear that the static methods are being promoted to instance methods, and we can discuss name later. > - I see you cache the Options in BufWriterImpl/BufferedCodeBuilder/etc, but we should probably be caching the full context (impl) object instead, and fetching options out of the context. > - (Syntax nit) Some enum options are fully descriptive of what they are (DROP_DEBUG_ELEMENTS), and some are implicit (ALWAYS_GENERATE -- generate what?). I think I would prefer to err on the side of fully descriptive (and people can use static imports to drop the qualifier if they like), but we should be consistent. There should be a convention of what the default is (like "first is always default") for options that are enums. > > > On 5/26/2023 11:18 AM, Adam Sotona wrote: > > This is a raw prototype of Classfile.Context implementation and the new options: > > https://github.com/openjdk/jdk/pull/14180/files > > https://github.com/openjdk/jdk/blob/4456ca78bc67138930dc313f71aba58ef1cf5f13/src/java.base/share/classes/jdk/internal/classfile/Classfile.java > > > > It is far from finished, however before going further I would appreciate your comments (mainly on options naming). > > > > Thanks, > > Adam > > > > > > From: Brian Goetz > Date: Thursday, 25 May 2023 20:31 > To: Adam Sotona , liangchenblue at gmail.com , classfile-api-dev > Subject: Re: Planned features of a classfile context object > > A model created within a context can retain a reference to its creating context. (We do a lot of this sort of thing already, where MethodModel retains a reference to ClassModel, and where CPBuilder retains a reference to Options.) So for operations that require a context, a model derived from a classfile can use the context that created it. > > Methods like parse/build/transform will be moved to the classfile context object. > > On 5/25/2023 2:02 PM, Adam Sotona wrote: > > I?m trying to figure out how the context will be passed down to the models (if it suppose to be detached from the models). > > Do we plan to enforce another CC argument to all models methods requiring context? > > Or do we plan to strip down all context-sensitive methods from all models and attach them to a kind of context-wrappers? > > For example codeModel.forEach(Classfile.Context.of(?), ?) or Classfile.Context.of(?).forEach(codeModel, ?) ? > > > > Thanks, > > Adam > > > > From adam.sotona at oracle.com Mon May 29 07:27:10 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Mon, 29 May 2023 07:27:10 +0000 Subject: Planned features of a classfile context object In-Reply-To: <69bf8797-1c16-2d20-1464-4c725e30dec9@oracle.com> References: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> <093f8cc9-72b6-d23e-2e92-202ec83eb3ed@oracle.com> <69bf8797-1c16-2d20-1464-4c725e30dec9@oracle.com> Message-ID: From: Brian Goetz Date: Friday, 26 May 2023 23:16 To: Adam Sotona , liangchenblue at gmail.com , classfile-api-dev Subject: Re: Planned features of a classfile context object Some review comments on your patch: - I'd prefer for all of the static methods in Classfile to go away, though I can imagine choosing to leave the most basic version (no options) of build/parse around as conveniences. If we leave these, the Javadoc should make it clear that these are equivalent to Context.of().parse(...). - I am OK with calling the context class "Classfile" for the time being; this makes the diff easier to follow as it makes it clear that the static methods are being promoted to instance methods, and we can discuss name later. OK, Classfile.of().parse(?) is much easier to find than Classfile.Context.of().parse(?), so I?ll remove the static methods. - I see you cache the Options in BufWriterImpl/BufferedCodeBuilder/etc, but we should probably be caching the full context (impl) object instead, and fetching options out of the context. impl.Options was the context class and now it only become implementation of Classfile.Context, I?ll rename it to ClassfileImpl. - (Syntax nit) Some enum options are fully descriptive of what they are (DROP_DEBUG_ELEMENTS), and some are implicit (ALWAYS_GENERATE -- generate what?). I think I would prefer to err on the side of fully descriptive (and people can use static imports to drop the qualifier if they like), but we should be consistent. There should be a convention of what the default is (like "first is always default") for options that are enums. I?ll fix it, thanks. Adam On 5/26/2023 11:18 AM, Adam Sotona wrote: This is a raw prototype of Classfile.Context implementation and the new options: https://github.com/openjdk/jdk/pull/14180/files https://github.com/openjdk/jdk/blob/4456ca78bc67138930dc313f71aba58ef1cf5f13/src/java.base/share/classes/jdk/internal/classfile/Classfile.java It is far from finished, however before going further I would appreciate your comments (mainly on options naming). Thanks, Adam From: Brian Goetz Date: Thursday, 25 May 2023 20:31 To: Adam Sotona , liangchenblue at gmail.com , classfile-api-dev Subject: Re: Planned features of a classfile context object A model created within a context can retain a reference to its creating context. (We do a lot of this sort of thing already, where MethodModel retains a reference to ClassModel, and where CPBuilder retains a reference to Options.) So for operations that require a context, a model derived from a classfile can use the context that created it. Methods like parse/build/transform will be moved to the classfile context object. On 5/25/2023 2:02 PM, Adam Sotona wrote: I?m trying to figure out how the context will be passed down to the models (if it suppose to be detached from the models). Do we plan to enforce another CC argument to all models methods requiring context? Or do we plan to strip down all context-sensitive methods from all models and attach them to a kind of context-wrappers? For example codeModel.forEach(Classfile.Context.of(?), ?) or Classfile.Context.of(?).forEach(codeModel, ?) ? Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Mon May 29 07:52:21 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Mon, 29 May 2023 07:52:21 +0000 Subject: [External] : Re: Planned features of a classfile context object In-Reply-To: References: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> <093f8cc9-72b6-d23e-2e92-202ec83eb3ed@oracle.com> <69bf8797-1c16-2d20-1464-4c725e30dec9@oracle.com> Message-ID: From: liangchenblue at gmail.com Date: Friday, 26 May 2023 23:53 To: Brian Goetz Cc: Adam Sotona , classfile-api-dev Subject: [External] : Re: Planned features of a classfile context object The Classfile context in the patch looks somewhat like a Classfile element to me :) With that in mind, I wish we can allow users to transform/fork these immutable contexts, such as skipping line numbers for particular files, skip constantpool sharing for select transformations, etc. with the same code pattern in Classfile Transformers. Of course, a transformation will always read the default values of options if not explicitly set, and the transformed context will have dropped options replaced with their default values. I can imagine use for context factory method: Classfile.Context.of(Classfile.Context parentContext, Option? optionsOverride) or after merge of Classfile.Context to Classfile: Classfile.of(Classfile parentContext, Option? optionsOverride) Is it what you think about? In addition, ClassModel.transform can probably be kept, which means using the creation context of the ClassModel to transform this model, if we make it clear that all models have their lifetime bound to a Classfile object. I found the rule of all new classes created from the context only as positive move. Active class creation directly from ClassModel is very confusing with respect to all the options that must be defined before the model is created. Explicit call of cc.transform(classModel, transform) is more clear about what build/transform/write options are applied. Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Tue May 30 23:32:48 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 30 May 2023 19:32:48 -0400 Subject: [External] : Re: Planned features of a classfile context object In-Reply-To: References: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> <093f8cc9-72b6-d23e-2e92-202ec83eb3ed@oracle.com> <69bf8797-1c16-2d20-1464-4c725e30dec9@oracle.com> Message-ID: I think having an instance method on Classfile that produces a modified context: ??? Classfile withOptions(Option...) would let you "refine" the options while keeping the base options / cache config / etc. I agree with Adam that moving all the "make me a class" stuff in one place is a benefit. On 5/29/2023 3:52 AM, Adam Sotona wrote: > > *From: *liangchenblue at gmail.com > *Date: *Friday, 26 May 2023 23:53 > *To: *Brian Goetz > *Cc: *Adam Sotona , classfile-api-dev > > *Subject: *[External] : Re: Planned features of a classfile context object > > /The Classfile context in the patch looks somewhat like a Classfile > element to me :) With that in mind, I wish we can allow users to > transform/fork these immutable contexts, such as skipping line numbers > for particular files, skip constantpool sharing for select > transformations, etc. with the same code pattern in Classfile > Transformers. Of course, a transformation will always read the default > values of options if not explicitly set, and the transformed context > will have dropped options replaced with their default values./ > > I can imagine use for context factory method: > > Classfile.Context.of(Classfile.Context parentContext, Option? > optionsOverride) > > or after merge of Classfile.Context to Classfile: > > Classfile.of(Classfile parentContext, Option? optionsOverride) > > Is it what you think about? > > > > /In addition, ClassModel.transform can probably be kept, which means > using the creation context of the ClassModel to transform this model, > if we make it clear that all models have their lifetime bound to a > Classfile object. > > / > > I found the rule of all new classes created from the context only as > positive move. > > Active class creation directly from ClassModel is very confusing with > respect to all the options that must be defined before the model is > created. > > Explicit call of? cc.transform(classModel, transform) is more clear > about what build/transform/write options are applied. > > Thanks, > > Adam > -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Wed May 31 01:47:46 2023 From: liangchenblue at gmail.com (-) Date: Wed, 31 May 2023 09:47:46 +0800 Subject: [External] : Re: Planned features of a classfile context object In-Reply-To: References: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> <093f8cc9-72b6-d23e-2e92-202ec83eb3ed@oracle.com> <69bf8797-1c16-2d20-1464-4c725e30dec9@oracle.com> Message-ID: I like this withOptions() factory too. Looks more convenient than the static of() parent-based forking; similar API designs exist in ModuleLayer, but I think we can just keep the instance version as we don't have multiple parents for a Classfile. On Wed, May 31, 2023 at 7:32?AM Brian Goetz wrote: > > I think having an instance method on Classfile that produces a modified context: > > Classfile withOptions(Option...) > > would let you "refine" the options while keeping the base options / cache config / etc. > > I agree with Adam that moving all the "make me a class" stuff in one place is a benefit. > > On 5/29/2023 3:52 AM, Adam Sotona wrote: > > > > > > From: liangchenblue at gmail.com > Date: Friday, 26 May 2023 23:53 > To: Brian Goetz > Cc: Adam Sotona , classfile-api-dev > Subject: [External] : Re: Planned features of a classfile context object > > The Classfile context in the patch looks somewhat like a Classfile > element to me :) With that in mind, I wish we can allow users to > transform/fork these immutable contexts, such as skipping line numbers > for particular files, skip constantpool sharing for select > transformations, etc. with the same code pattern in Classfile > Transformers. Of course, a transformation will always read the default > values of options if not explicitly set, and the transformed context > will have dropped options replaced with their default values. > > > > I can imagine use for context factory method: > > Classfile.Context.of(Classfile.Context parentContext, Option? optionsOverride) > > or after merge of Classfile.Context to Classfile: > > Classfile.of(Classfile parentContext, Option? optionsOverride) > > Is it what you think about? > > > > In addition, ClassModel.transform can probably be kept, which means > using the creation context of the ClassModel to transform this model, > if we make it clear that all models have their lifetime bound to a > Classfile object. > > I found the rule of all new classes created from the context only as positive move. > > Active class creation directly from ClassModel is very confusing with respect to all the options that must be defined before the model is created. > > Explicit call of cc.transform(classModel, transform) is more clear about what build/transform/write options are applied. > > > > Thanks, > > Adam > > From liangchenblue at gmail.com Wed May 31 06:26:38 2023 From: liangchenblue at gmail.com (-) Date: Wed, 31 May 2023 14:26:38 +0800 Subject: ClassHierarchyResolver changes for the Classfile object Message-ID: Hello, Since we are migrating to the Classfile object (8308899), which was initiated by our observation that Class hierarchy resolvers shouldn't always cache their results, I wonder what changes I should adapt to my current patch (8304425, PR #13082). We have already decided a few API changes for the resolver: 1. Remove the default hierarchy resolver (at least change to static factories), caching is bound to Classfile lifetime 2. Two new resolver factories: Bytecode/resource parsing and classloading/reflection from a given classloader (The goal of this patch) 3. The resolution result ClassHierarchyInfo will be converted into an interface; record is an implementation detail (TBD) What should I do with my patch? Should I drop the changes to default resolver (which adds security manager suppression) and move on with the current content, or should I expand it to address all 3 planned changes (despite some of them still in drafting phase) and accomodate to the Classfile object? (We still have to decide about the default resolver fo a Classfile object, esp. whether to have one or not) Chen Liang From adam.sotona at oracle.com Wed May 31 09:55:46 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Wed, 31 May 2023 09:55:46 +0000 Subject: [External] : Re: Planned features of a classfile context object In-Reply-To: References: <771c9153-9235-f349-d239-6ca6cafb8f7b@oracle.com> <83a0a243-c8f8-dce3-42c5-dbec0be5ff9d@oracle.com> <093f8cc9-72b6-d23e-2e92-202ec83eb3ed@oracle.com> <69bf8797-1c16-2d20-1464-4c725e30dec9@oracle.com> Message-ID: OK, I?ve updated the PR accordingly: https://github.com/openjdk/jdk/pull/14180/files https://github.com/openjdk/jdk/blob/b7e42aa60e8ef6440ffe09577ab2de695dbc7513/src/java.base/share/classes/jdk/internal/classfile/Classfile.java If there are no objections I?ll proceed to update all doc and snippets to reflect the changes. Thanks, Adam From: liangchenblue at gmail.com Date: Wednesday, 31 May 2023 3:48 To: Brian Goetz Cc: Adam Sotona , classfile-api-dev Subject: Re: [External] : Re: Planned features of a classfile context object I like this withOptions() factory too. Looks more convenient than the static of() parent-based forking; similar API designs exist in ModuleLayer, but I think we can just keep the instance version as we don't have multiple parents for a Classfile. On Wed, May 31, 2023 at 7:32?AM Brian Goetz wrote: > > I think having an instance method on Classfile that produces a modified context: > > Classfile withOptions(Option...) > > would let you "refine" the options while keeping the base options / cache config / etc. > > I agree with Adam that moving all the "make me a class" stuff in one place is a benefit. > > On 5/29/2023 3:52 AM, Adam Sotona wrote: > > > > > > From: liangchenblue at gmail.com > Date: Friday, 26 May 2023 23:53 > To: Brian Goetz > Cc: Adam Sotona , classfile-api-dev > Subject: [External] : Re: Planned features of a classfile context object > > The Classfile context in the patch looks somewhat like a Classfile > element to me :) With that in mind, I wish we can allow users to > transform/fork these immutable contexts, such as skipping line numbers > for particular files, skip constantpool sharing for select > transformations, etc. with the same code pattern in Classfile > Transformers. Of course, a transformation will always read the default > values of options if not explicitly set, and the transformed context > will have dropped options replaced with their default values. > > > > I can imagine use for context factory method: > > Classfile.Context.of(Classfile.Context parentContext, Option? optionsOverride) > > or after merge of Classfile.Context to Classfile: > > Classfile.of(Classfile parentContext, Option? optionsOverride) > > Is it what you think about? > > > > In addition, ClassModel.transform can probably be kept, which means > using the creation context of the ClassModel to transform this model, > if we make it clear that all models have their lifetime bound to a > Classfile object. > > I found the rule of all new classes created from the context only as positive move. > > Active class creation directly from ClassModel is very confusing with respect to all the options that must be defined before the model is created. > > Explicit call of cc.transform(classModel, transform) is more clear about what build/transform/write options are applied. > > > > Thanks, > > Adam > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Wed May 31 15:08:51 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 31 May 2023 11:08:51 -0400 Subject: [External] : Re: Planned features of a classfile context object In-Reply-To: References: <093f8cc9-72b6-d23e-2e92-202ec83eb3ed@oracle.com> <69bf8797-1c16-2d20-1464-4c725e30dec9@oracle.com> Message-ID: <37aba51e-33f7-90de-6c4b-aff0f1bee6a4@oracle.com> Small nits: ?- should add in buildTo() overload that mirrors build() overloads ?- Not sure doing ClassFileImpl as a record is a win The rest of it seems nicely minimal. On 5/31/2023 5:55 AM, Adam Sotona wrote: > > OK, > > I?ve updated the PR accordingly: > > https://github.com/openjdk/jdk/pull/14180/files > > > https://github.com/openjdk/jdk/blob/b7e42aa60e8ef6440ffe09577ab2de695dbc7513/src/java.base/share/classes/jdk/internal/classfile/Classfile.java > > If there are no objections I?ll proceed to update all doc and snippets > to reflect the changes. > > Thanks, > > Adam > > *From: *liangchenblue at gmail.com > *Date: *Wednesday, 31 May 2023 3:48 > *To: *Brian Goetz > *Cc: *Adam Sotona , classfile-api-dev > > *Subject: *Re: [External] : Re: Planned features of a classfile > context object > > I like this withOptions() factory too. Looks more convenient than the > static of() parent-based forking; similar API designs exist in > ModuleLayer, but I think we can just keep the instance version as we > don't have multiple parents for a Classfile. > > On Wed, May 31, 2023 at 7:32?AM Brian Goetz > wrote: > > > > I think having an instance method on Classfile that produces a > modified context: > > > >???? Classfile withOptions(Option...) > > > > would let you "refine" the options while keeping the base options / > cache config / etc. > > > > I agree with Adam that moving all the "make me a class" stuff in one > place is a benefit. > > > > On 5/29/2023 3:52 AM, Adam Sotona wrote: > > > > > > > > > > > > From: liangchenblue at gmail.com > > Date: Friday, 26 May 2023 23:53 > > To: Brian Goetz > > Cc: Adam Sotona , classfile-api-dev > > > Subject: [External] : Re: Planned features of a classfile context object > > > > The Classfile context in the patch looks somewhat like a Classfile > > element to me :) With that in mind, I wish we can allow users to > > transform/fork these immutable contexts, such as skipping line numbers > > for particular files, skip constantpool sharing for select > > transformations, etc. with the same code pattern in Classfile > > Transformers. Of course, a transformation will always read the default > > values of options if not explicitly set, and the transformed context > > will have dropped options replaced with their default values. > > > > > > > > I can imagine use for context factory method: > > > > Classfile.Context.of(Classfile.Context parentContext, Option? > optionsOverride) > > > > or after merge of Classfile.Context to Classfile: > > > > Classfile.of(Classfile parentContext, Option? optionsOverride) > > > > Is it what you think about? > > > > > > > > In addition, ClassModel.transform can probably be kept, which means > > using the creation context of the ClassModel to transform this model, > > if we make it clear that all models have their lifetime bound to a > > Classfile object. > > > > I found the rule of all new classes created from the context only as > positive move. > > > > Active class creation directly from ClassModel is very confusing > with respect to all the options that must be defined before the model > is created. > > > > Explicit call of? cc.transform(classModel, transform) is more clear > about what build/transform/write options are applied. > > > > > > > > Thanks, > > > > Adam > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Wed May 31 17:33:57 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Wed, 31 May 2023 17:33:57 +0000 Subject: ClassHierarchyResolver changes for the Classfile object In-Reply-To: References: Message-ID: From: classfile-api-dev on behalf of liangchenblue at gmail.com Date: Wednesday, 31 May 2023 8:26 To: classfile-api-dev Subject: ClassHierarchyResolver changes for the Classfile object Hello, Since we are migrating to the Classfile object (8308899), which was initiated by our observation that Class hierarchy resolvers shouldn't always cache their results, I wonder what changes I should adapt to my current patch (8304425, PR #13082). We have already decided a few API changes for the resolver: 1. Remove the default hierarchy resolver (at least change to static factories), caching is bound to Classfile lifetime Yes, but we need to check performance impact on existing use cases (where to extra hold Classfile instance with cache). 2. Two new resolver factories: Bytecode/resource parsing and classloading/reflection from a given classloader (The goal of this patch) Yes. 3. The resolution result ClassHierarchyInfo will be converted into an interface; record is an implementation detail (TBD) This can go even further. To save footprint ClassHierarchyInfo does not need to expose anything, it does not need to carry thisClass and all interface infos can point to a singleton object, something like this: /** * Information about a resolved class or interface. */ sealed interface ClassHierarchyInfo permits ClassHierarchyImpl.ClassHierarchyInfoImpl { /** * {@return the {@link ClassHierarchyInfo} of an interface} * no other information about interface is required */ static ClassHierarchyInfo ofInterface() { return ClassHierarchyImpl.INTERFACE_INFO_INSTANCE; } /** * @return the {@link ClassHierarchyInfo} of an interface * @param superClass information about super of the class is required */ static ClassHierarchyInfo ofClass(ClassDesc superClass) { return new ClassHierarchyImpl.ClassHierarchyInfoImpl(superClass); } } What should I do with my patch? Should I drop the changes to default resolver (which adds security manager suppression) and move on with the current content, or should I expand it to address all 3 planned changes (despite some of them still in drafting phase) and accomodate to the Classfile object? (We still have to decide about the default resolver fo a Classfile object, esp. whether to have one or not) I?m not sure why the default class hierarchy resolver should call AccessController.doPrivileged? Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: