From adam.sotona at oracle.com Fri Nov 1 13:16:13 2024 From: adam.sotona at oracle.com (Adam Sotona) Date: Fri, 1 Nov 2024 13:16:13 +0000 Subject: [External] : Re: Stack size miscalculated with try-catch blocks In-Reply-To: References: Message-ID: ? ? You are right, it is a recent regression and the JDK I tested it on was outdated, my apologies. The regression is here: https://github.com/openjdk/jdk/blob/f77a5144a12fc31bad8b672a3cc9caa688d78e72/src/java.base/share/classes/jdk/internal/classfile/impl/StackMapGenerator.java#L1117 introduced by https://github.com/openjdk/jdk/pull/20756 I?ve created a bug https://bugs.openjdk.org/browse/JDK-8343436 and working on the fix. Thanks for the report, Adam From: Rafael Winterhalter Date: Wednesday, 30 October 2024 at 11:41 To: Adam Sotona Cc: classfile-api-dev Subject: [External] : Re: Stack size miscalculated with try-catch blocks Is this pushed to master? I still experience these errors on multiple test cases. Am Mi., 30. Okt. 2024 um 10:19 Uhr schrieb Adam Sotona >: Hi Rafael, Please update to the latest codebase, maxstack of the provided test case is correctly calculated now. Thanks, Adam From: classfile-api-dev > on behalf of Rafael Winterhalter > Date: Thursday, 24 October 2024 at 8:33 To: classfile-api-dev > Subject: Stack size miscalculated with try-catch blocks Hello, when testing a recent build of the JDK with my bridge to ASM, I encountered some test failures. This is related to the stack size of a method being calculated with an additional, unnecessary slot. This can be reproduced using https://github.com/raphw/asm-jdk-bridge This can also be reproduced by inspecting the computed stack for: void c() { try { throw new RuntimeException(); } catch (RuntimeException e) { } } which will be computed as 3 and not 2. Best regards, Rafael -------------- next part -------------- An HTML attachment was scrubbed... URL: From rafael.wth at gmail.com Sat Nov 9 08:32:44 2024 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Sat, 9 Nov 2024 09:32:44 +0100 Subject: Mapping byte code offsets in custom code attributes Message-ID: Hello, I mentioned before that I think there is still a missing bit when it comes to custom code attributes, and mapping them using an AttributeMapper. Many code attributes point to a location, or area, within a method. This typically implies specifying one or two byte code offsets within an attribute. To implement this, I would need a possibility to translate offsets to labels within an attribute mapper, and vice versa. For example, if I created byte code from a different source, where a method consists of multiple files, I would want to mark what part of the method comes from what file. I do not currently know of a way to do this in the Class File API. To resolve this, I would have hoped that the attribute reader method would be able to issue labels that are later issued when the code instructions are supplied. Similarly, I would have hoped that the attribute writer method would be capable of translating labels to byte code offsets. In the same way, I would have hoped that all attributes such as LineNumberInfo of the LineNumberTableAttribute would expose bytecode offsets as labels and not as integers, as the latter does not really give any useful information by itself. This way, I could also write a custom LineNumberTableAttribute after a method is already created by issuing relevant labels, but by only writing the attribute when the entire method was visited. Sometimes this is necessary when generating code, when the information about the actual line numbers is only retrieved at a later stage that requires knowledge of the entire method. As I understand it, the ClassFile API is supposed to be released in Java 24. Are there still plans to integrate such a change? After that it would require incompatible changes, therefore I had hoped this would make it into the final version. Thanks! Rafael -------------- next part -------------- An HTML attachment was scrubbed... URL: From rafael.wth at gmail.com Sat Nov 9 21:09:39 2024 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Sat, 9 Nov 2024 22:09:39 +0100 Subject: Offset error when writing type annotations on NEW instructions Message-ID: With the most recent build of JDK 24, I discover one test failure in my test suite when comparing with ASM (https://github.com/raphw/asm-jdk-bridge): Any type annotation that is added to a "new" instruction is added to the subsequent instruction instead. This might irritate parsers as the label will point to an instruction that cannot normally be annotated, as NEW is normally followed by DUP. I created a reproducers that compares a javac compiled class and one that is created with the class file API. The error is likely in the attribute mapper's write method as I cannot reproduce it when testing a class file reader using the class file API compared to ASM. The reproducer is as follows and is visualized in the first line of each of the two representations where the correct byte code offset is 0, not 3 as the annotation is added to the first instruction in the method: import java.io.InputStream; import java.lang.annotation.ElementType; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; import java.lang.classfile.*; import java.lang.classfile.attribute.RuntimeVisibleTypeAnnotationsAttribute; import java.lang.classfile.instruction.LineNumber; import java.lang.classfile.instruction.LocalVariable; import java.lang.constant.ClassDesc; import java.lang.constant.MethodTypeDesc; import java.util.List; public class TypeAnnotationSample { void m() { new @A Object(); } public static void main(String[] args) throws Exception { byte[] compiled; try (InputStream inputStream = TypeAnnotationSample.class.getClassLoader().getResourceAsStream( TypeAnnotationSample.class.getName().replace('.', '/') + ".class" )) { compiled = inputStream.readAllBytes(); } print(compiled); System.out.println("-----"); byte[] created = ClassFile.of().build(ClassDesc.of("Synthetic"), classBuilder -> { classBuilder.withMethod("m", MethodTypeDesc.of(void.class.describeConstable().orElseThrow()), 0, methodBuilder -> { methodBuilder.withCode(codeBuilder -> { codeBuilder.new_(Object.class.describeConstable().orElseThrow()); Label label = codeBuilder.newBoundLabel(); codeBuilder.with(RuntimeVisibleTypeAnnotationsAttribute.of(TypeAnnotation.of( TypeAnnotation.TargetInfo.ofNewExpr(label), List.of(), Annotation.of(A.class.describeConstable().orElseThrow()) ))); codeBuilder.dup() .invokespecial(Object.class.describeConstable().orElseThrow(), "", MethodTypeDesc.of(void.class.describeConstable().orElseThrow())) .pop() .return_(); }); }); }); print(created); } private static void print(byte[] classFile) { ClassModel classModel = ClassFile.of().parse(classFile); MethodModel methodModel = classModel.methods().stream() .filter(element -> element.methodName().equalsString("m")) .findFirst() .orElseThrow(); CodeModel codeModel = methodModel.code().orElseThrow(); codeModel.elementStream().forEach(element -> { switch (element) { case LocalVariable _ -> { } case LineNumber _ -> { } case RuntimeVisibleTypeAnnotationsAttribute a -> System.out.println(a.annotations().stream() .map(x -> x.annotation() + "@" + x.targetPath() + "/" + ((TypeAnnotation.OffsetTarget) x.targetInfo()).target()) .toList()); default -> System.out.println(element); } }); } @Retention(RetentionPolicy.RUNTIME) @Target(ElementType.TYPE_USE) @interface A { } } I think that the attribute mapper must subtract the bytes of the instruction as it is currently adding the bytes at the end of the instruction, not at its beginning. ASM and javac set the annotation offset to the beginning of the statement, so I understand that this should be the right approach. Thanks, Rafael -------------- next part -------------- An HTML attachment was scrubbed... URL: From chen.l.liang at oracle.com Sun Nov 10 23:12:16 2024 From: chen.l.liang at oracle.com (Chen Liang) Date: Sun, 10 Nov 2024 23:12:16 +0000 Subject: Offset error when writing type annotations on NEW instructions In-Reply-To: References: Message-ID: Hi Rafael, I believe this is due to a misunderstanding with how ClassFile API labels work. A ClassFile API label models an insertion point represented by a BCI; so in your example, the bound label should be located right before the new instruction, so that the next immediately executed instruction would be the new instruction. This is like how the labels for control flows work. Unfortunately, I understand this is distinct from the ASM model - ASM visitor model treats this as if this is a tiny tree structure and considers a type annotation as a subsidiary of an instruction, and this brings challenges to porting the implementations. Here's some context: we ClassFile API developers long believed that type annotations interact poorly with class file transformations because of hard-to-track arbitrary references such as interface indices, and thus we did not integrate it as a part of the ClassFile API code streaming model, and we drops type annotation attributes in transformations by the attribute stability settings. Regards, Chen Liang ________________________________ From: classfile-api-dev on behalf of Rafael Winterhalter Sent: Saturday, November 9, 2024 3:09 PM To: classfile-api-dev Subject: Offset error when writing type annotations on NEW instructions With the most recent build of JDK 24, I discover one test failure in my test suite when comparing with ASM (https://github.com/raphw/asm-jdk-bridge): Any type annotation that is added to a "new" instruction is added to the subsequent instruction instead. This might irritate parsers as the label will point to an instruction that cannot normally be annotated, as NEW is normally followed by DUP. I created a reproducers that compares a javac compiled class and one that is created with the class file API. The error is likely in the attribute mapper's write method as I cannot reproduce it when testing a class file reader using the class file API compared to ASM. The reproducer is as follows and is visualized in the first line of each of the two representations where the correct byte code offset is 0, not 3 as the annotation is added to the first instruction in the method: import java.io.InputStream; import java.lang.annotation.ElementType; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; import java.lang.classfile.*; import java.lang.classfile.attribute.RuntimeVisibleTypeAnnotationsAttribute; import java.lang.classfile.instruction.LineNumber; import java.lang.classfile.instruction.LocalVariable; import java.lang.constant.ClassDesc; import java.lang.constant.MethodTypeDesc; import java.util.List; public class TypeAnnotationSample { void m() { new @A Object(); } public static void main(String[] args) throws Exception { byte[] compiled; try (InputStream inputStream = TypeAnnotationSample.class.getClassLoader().getResourceAsStream( TypeAnnotationSample.class.getName().replace('.', '/') + ".class" )) { compiled = inputStream.readAllBytes(); } print(compiled); System.out.println("-----"); byte[] created = ClassFile.of().build(ClassDesc.of("Synthetic"), classBuilder -> { classBuilder.withMethod("m", MethodTypeDesc.of(void.class.describeConstable().orElseThrow()), 0, methodBuilder -> { methodBuilder.withCode(codeBuilder -> { codeBuilder.new_(Object.class.describeConstable().orElseThrow()); Label label = codeBuilder.newBoundLabel(); codeBuilder.with(RuntimeVisibleTypeAnnotationsAttribute.of(TypeAnnotation.of( TypeAnnotation.TargetInfo.ofNewExpr(label), List.of(), Annotation.of(A.class.describeConstable().orElseThrow()) ))); codeBuilder.dup() .invokespecial(Object.class.describeConstable().orElseThrow(), "", MethodTypeDesc.of(void.class.describeConstable().orElseThrow())) .pop() .return_(); }); }); }); print(created); } private static void print(byte[] classFile) { ClassModel classModel = ClassFile.of().parse(classFile); MethodModel methodModel = classModel.methods().stream() .filter(element -> element.methodName().equalsString("m")) .findFirst() .orElseThrow(); CodeModel codeModel = methodModel.code().orElseThrow(); codeModel.elementStream().forEach(element -> { switch (element) { case LocalVariable _ -> { } case LineNumber _ -> { } case RuntimeVisibleTypeAnnotationsAttribute a -> System.out.println(a.annotations().stream() .map(x -> x.annotation() + "@" + x.targetPath() + "/" + ((TypeAnnotation.OffsetTarget) x.targetInfo()).target()) .toList()); default -> System.out.println(element); } }); } @Retention(RetentionPolicy.RUNTIME) @Target(ElementType.TYPE_USE) @interface A { } } I think that the attribute mapper must subtract the bytes of the instruction as it is currently adding the bytes at the end of the instruction, not at its beginning. ASM and javac set the annotation offset to the beginning of the statement, so I understand that this should be the right approach. Thanks, Rafael -------------- next part -------------- An HTML attachment was scrubbed... URL: From chen.l.liang at oracle.com Sun Nov 10 23:28:38 2024 From: chen.l.liang at oracle.com (Chen Liang) Date: Sun, 10 Nov 2024 23:28:38 +0000 Subject: Mapping byte code offsets in custom code attributes In-Reply-To: References: Message-ID: Hi Rafael, You are correct. I had previously created an issue https://bugs.openjdk.org/browse/JDK-8341274 to track this demand. Since this scenario has no way to bypass, maybe we can raise this issue to P3 and work on this feature for JDK 24. Also I wonder how frequently are custom attributes storing BCIs? For LineNumberInfo: I believe the Label-based version, LineNumber, is already streamed in class file reading, and can be sent to the builder also via CodeBuilder::lineNumber(int). Your particular use case can probably be covered by the RFE too, once we can convert labels to BCIs in non-Code attributes. Regards, Chen Liang ________________________________ From: classfile-api-dev on behalf of Rafael Winterhalter Sent: Saturday, November 9, 2024 2:32 AM To: classfile-api-dev Subject: Mapping byte code offsets in custom code attributes Hello, I mentioned before that I think there is still a missing bit when it comes to custom code attributes, and mapping them using an AttributeMapper. Many code attributes point to a location, or area, within a method. This typically implies specifying one or two byte code offsets within an attribute. To implement this, I would need a possibility to translate offsets to labels within an attribute mapper, and vice versa. For example, if I created byte code from a different source, where a method consists of multiple files, I would want to mark what part of the method comes from what file. I do not currently know of a way to do this in the Class File API. To resolve this, I would have hoped that the attribute reader method would be able to issue labels that are later issued when the code instructions are supplied. Similarly, I would have hoped that the attribute writer method would be capable of translating labels to byte code offsets. In the same way, I would have hoped that all attributes such as LineNumberInfo of the LineNumberTableAttribute would expose bytecode offsets as labels and not as integers, as the latter does not really give any useful information by itself. This way, I could also write a custom LineNumberTableAttribute after a method is already created by issuing relevant labels, but by only writing the attribute when the entire method was visited. Sometimes this is necessary when generating code, when the information about the actual line numbers is only retrieved at a later stage that requires knowledge of the entire method. As I understand it, the ClassFile API is supposed to be released in Java 24. Are there still plans to integrate such a change? After that it would require incompatible changes, therefore I had hoped this would make it into the final version. Thanks! Rafael -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Mon Nov 11 08:17:16 2024 From: adam.sotona at oracle.com (Adam Sotona) Date: Mon, 11 Nov 2024 08:17:16 +0000 Subject: Mapping byte code offsets in custom code attributes In-Reply-To: References: Message-ID: Hi Rafael, LineNumber, LocalVariable and LocalVariableType (as CodeElements) are integral parts of the Class-File API and they should serve well to 99% of user use cases. In addition, LineNumberTableAttribute, LocalVariableTableAttribute, LocalVariableTypeTableAttribute and StackMapTableAttribute are available in ?raw? forms to support strongly requested specific corner cases. This is clear request to support custom code-level attributes with references to bytecode offsets. I agree with Chen that JDK-8341274 should be sufficient to resolve it. Thank you, Adam From: classfile-api-dev on behalf of Chen Liang Date: Monday, 11 November 2024 at 0:28 To: Rafael Winterhalter , classfile-api-dev Subject: Re: Mapping byte code offsets in custom code attributes Hi Rafael, You are correct. I had previously created an issue https://bugs.openjdk.org/browse/JDK-8341274 to track this demand. Since this scenario has no way to bypass, maybe we can raise this issue to P3 and work on this feature for JDK 24. Also I wonder how frequently are custom attributes storing BCIs? For LineNumberInfo: I believe the Label-based version, LineNumber, is already streamed in class file reading, and can be sent to the builder also via CodeBuilder::lineNumber(int). Your particular use case can probably be covered by the RFE too, once we can convert labels to BCIs in non-Code attributes. Regards, Chen Liang ________________________________ From: classfile-api-dev on behalf of Rafael Winterhalter Sent: Saturday, November 9, 2024 2:32 AM To: classfile-api-dev Subject: Mapping byte code offsets in custom code attributes Hello, I mentioned before that I think there is still a missing bit when it comes to custom code attributes, and mapping them using an AttributeMapper. Many code attributes point to a location, or area, within a method. This typically implies specifying one or two byte code offsets within an attribute. To implement this, I would need a possibility to translate offsets to labels within an attribute mapper, and vice versa. For example, if I created byte code from a different source, where a method consists of multiple files, I would want to mark what part of the method comes from what file. I do not currently know of a way to do this in the Class File API. To resolve this, I would have hoped that the attribute reader method would be able to issue labels that are later issued when the code instructions are supplied. Similarly, I would have hoped that the attribute writer method would be capable of translating labels to byte code offsets. In the same way, I would have hoped that all attributes such as LineNumberInfo of the LineNumberTableAttribute would expose bytecode offsets as labels and not as integers, as the latter does not really give any useful information by itself. This way, I could also write a custom LineNumberTableAttribute after a method is already created by issuing relevant labels, but by only writing the attribute when the entire method was visited. Sometimes this is necessary when generating code, when the information about the actual line numbers is only retrieved at a later stage that requires knowledge of the entire method. As I understand it, the ClassFile API is supposed to be released in Java 24. Are there still plans to integrate such a change? After that it would require incompatible changes, therefore I had hoped this would make it into the final version. Thanks! Rafael -------------- next part -------------- An HTML attachment was scrubbed... URL: From rafael.wth at gmail.com Mon Nov 11 08:17:56 2024 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Mon, 11 Nov 2024 09:17:56 +0100 Subject: Mapping byte code offsets in custom code attributes In-Reply-To: References: Message-ID: Hei Chen, I would argue that code attributes are the most common type of custom attributes, as attributes on other elements often can be expressed by annotations. Even compilers for other languages on the JVM use annotations where they can as of today. While code (type) annotations are supported, many code attributes try to mark a segment of byte code, whereas code annotations target a specific instruction. This is what makes them so common. As a matter of fact, I thought about this recently, that Java could likely make the need for custom code attributes obsolete if one could annotate blocks of code such as in: @MyAnnotation { ... }, but of course this is not an option today, and custom attributes will not disappear even if this option was supported by the class file format. With this background, almost all custom attributes I ever encountered were code attributes that mark segments of code. Without an option to resolve a bci to a label and vice versa, I already see a range of byte code power users which will struggle to take the Class File API into use. At the same time: ASM's supports this, but the API is complicated, so there is also a good opportunity to offer a better user experience, if done right. This is why I was hoping that bci could be avoided in the public API to begin with. I already see some needed deprecation if one would needs to adjust this later. Thanks, Rafael Am Mo., 11. Nov. 2024 um 00:28 Uhr schrieb Chen Liang < chen.l.liang at oracle.com>: > Hi Rafael, > You are correct. I had previously created an issue > https://bugs.openjdk.org/browse/JDK-8341274 to track this demand. Since > this scenario has no way to bypass, maybe we can raise this issue to P3 and > work on this feature for JDK 24. Also I wonder how frequently are custom > attributes storing BCIs? > > For LineNumberInfo: I believe the Label-based version, LineNumber, is > already streamed in class file reading, and can be sent to the builder also > via CodeBuilder::lineNumber(int). Your particular use case can probably be > covered by the RFE too, once we can convert labels to BCIs in non-Code > attributes. > > Regards, > Chen Liang > ------------------------------ > *From:* classfile-api-dev on behalf > of Rafael Winterhalter > *Sent:* Saturday, November 9, 2024 2:32 AM > *To:* classfile-api-dev > *Subject:* Mapping byte code offsets in custom code attributes > > Hello, > > I mentioned before that I think there is still a missing bit when it comes > to custom code attributes, and mapping them using an AttributeMapper. Many > code attributes point to a location, or area, within a method. This > typically implies specifying one or two byte code offsets within an > attribute. > > To implement this, I would need a possibility to translate offsets to > labels within an attribute mapper, and vice versa. For example, if I > created byte code from a different source, where a method consists of > multiple files, I would want to mark what part of the method comes from > what file. > > I do not currently know of a way to do this in the Class File API. To > resolve this, I would have hoped that the attribute reader method would be > able to issue labels that are later issued when the code instructions are > supplied. Similarly, I would have hoped that the attribute writer method > would be capable of translating labels to byte code offsets. > > In the same way, I would have hoped that all attributes such as > LineNumberInfo of the LineNumberTableAttribute would expose bytecode > offsets as labels and not as integers, as the latter does not really give > any useful information by itself. This way, I could also write a custom > LineNumberTableAttribute after a method is already created by issuing > relevant labels, but by only writing the attribute when the entire method > was visited. Sometimes this is necessary when generating code, when the > information about the actual line numbers is only retrieved at a later > stage that requires knowledge of the entire method. > > As I understand it, the ClassFile API is supposed to be released in Java > 24. Are there still plans to integrate such a change? After that it would > require incompatible changes, therefore I had hoped this would make it into > the final version. > > Thanks! Rafael > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rafael.wth at gmail.com Mon Nov 11 08:42:28 2024 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Mon, 11 Nov 2024 09:42:28 +0100 Subject: Mapping byte code offsets in custom code attributes In-Reply-To: References: Message-ID: Hei Adam, I agree that most use cases should be covered by today's API. I already use the former in my bridge implementation to use ASM with the Class File API. When it comes to LineNumberTableAttribute, I do not understand how it can be useful to resolve the remaining 1%, though. LineNumberInfo is built around a "startPc". Without an opportunity to resolve a BCI for a generated method, I do not understand how I can create a custom line number table attribute using the Class File API. The CodeBuilder interface does not expose the created CodeAttribute, which would be needed to resolve a label position to the needed BCI that the LineNumberInfo class requires. At a minimum, I would need CodeBuilder to resolve the current byte code index, but in the end this would only be used to build a custom label mechanism. I argue that this should already be handled by the API. The same goes for reading. I can of course count the size of each encountered instruction and keep a custom mapping of attributed BCIs to invoke a custom callback whenever a relevant offset is discovered, but this is also what the Label API is used for, and I think the API should handle this mapping. I find the same limitation with LocalVariableTypeInfo, LocalVariableInfo and CharacterRangeInfo. I also believe this was considered for type annotations where TypeAnnotation.TargetInfo.OffsetTarget resolves a label, and not a BCI. I expected consistent handling for all attributes other than type annotations, including custom attributes. Best regards, Rafael Am Mo., 11. Nov. 2024 um 09:17 Uhr schrieb Adam Sotona < adam.sotona at oracle.com>: > Hi Rafael, > > > > LineNumber, LocalVariable and LocalVariableType (as CodeElements) are > integral parts of the Class-File API and they should serve well to 99% of > user use cases. > > > > In addition, LineNumberTableAttribute, LocalVariableTableAttribute, > LocalVariableTypeTableAttribute and StackMapTableAttribute are available in > ?raw? forms to support strongly requested specific corner cases. > > > > This is clear request to support custom code-level attributes with > references to bytecode offsets. > > I agree with Chen that JDK-8341274 should be sufficient to resolve it. > > > > Thank you, > > Adam > > > > > > > > > > *From: *classfile-api-dev on behalf > of Chen Liang > *Date: *Monday, 11 November 2024 at 0:28 > *To: *Rafael Winterhalter , classfile-api-dev < > classfile-api-dev at openjdk.org> > *Subject: *Re: Mapping byte code offsets in custom code attributes > > Hi Rafael, > > You are correct. I had previously created an issue > https://bugs.openjdk.org/browse/JDK-8341274 to track this demand. Since > this scenario has no way to bypass, maybe we can raise this issue to P3 and > work on this feature for JDK 24. Also I wonder how frequently are custom > attributes storing BCIs? > > > > For LineNumberInfo: I believe the Label-based version, LineNumber, is > already streamed in class file reading, and can be sent to the builder also > via CodeBuilder::lineNumber(int). Your particular use case can probably be > covered by the RFE too, once we can convert labels to BCIs in non-Code > attributes. > > > > Regards, > > Chen Liang > ------------------------------ > > *From:* classfile-api-dev on behalf > of Rafael Winterhalter > *Sent:* Saturday, November 9, 2024 2:32 AM > *To:* classfile-api-dev > *Subject:* Mapping byte code offsets in custom code attributes > > > > Hello, > > > > I mentioned before that I think there is still a missing bit when it comes > to custom code attributes, and mapping them using an AttributeMapper. Many > code attributes point to a location, or area, within a method. This > typically implies specifying one or two byte code offsets within an > attribute. > > > > To implement this, I would need a possibility to translate offsets to > labels within an attribute mapper, and vice versa. For example, if I > created byte code from a different source, where a method consists of > multiple files, I would want to mark what part of the method comes from > what file. > > > > I do not currently know of a way to do this in the Class File API. To > resolve this, I would have hoped that the attribute reader method would be > able to issue labels that are later issued when the code instructions are > supplied. Similarly, I would have hoped that the attribute writer method > would be capable of translating labels to byte code offsets. > > > > In the same way, I would have hoped that all attributes such as > LineNumberInfo of the LineNumberTableAttribute would expose bytecode > offsets as labels and not as integers, as the latter does not really give > any useful information by itself. This way, I could also write a custom > LineNumberTableAttribute after a method is already created by issuing > relevant labels, but by only writing the attribute when the entire method > was visited. Sometimes this is necessary when generating code, when the > information about the actual line numbers is only retrieved at a later > stage that requires knowledge of the entire method. > > > > As I understand it, the ClassFile API is supposed to be released in Java > 24. Are there still plans to integrate such a change? After that it would > require incompatible changes, therefore I had hoped this would make it into > the final version. > > > > Thanks! Rafael > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rafael.wth at gmail.com Mon Nov 11 09:17:53 2024 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Mon, 11 Nov 2024 10:17:53 +0100 Subject: Offset error when writing type annotations on NEW instructions In-Reply-To: References: Message-ID: Hei Chen, I think I understand the problem then. With ASM, type annotations on elements are processed just after the element was encountered. From the javadoc of ASM: "Visits an annotation on an instruction. This method must be called just after the annotated instruction." In contrast, the Class File API requires a label to be inserted "just before" the instruction is added. Ideally, CodeBuilder would allow me to call something like "newBoundLabelOnPreviousInstruction", but I understand if you do not want to add something like it. If the Class File API allowed to add labels both prior or after encountering an instruction, one could work with both approaches of inserting type annotations prior or after an instruction is added. This would make migrating from ASM more straight-forward, too, as I assume that this is the prevalent model today with ASM being so dominant. Otherwise, I will need to work around and delay any ASM invocation for my bridge, to consider if a subsequent call represents a type annotation and apply a reordering this way. This is not a big deal, but of course it would be much easier solved if I could bind a label with a BCI offset further up the stack. Possibly, I could even add a label before any annotatable instruction, just in case. Would this be overly expensive to do? Cheers, Rafael Am Mo., 11. Nov. 2024 um 00:12 Uhr schrieb Chen Liang < chen.l.liang at oracle.com>: > Hi Rafael, > I believe this is due to a misunderstanding with how ClassFile API labels > work. A ClassFile API label models an insertion point represented by a > BCI; so in your example, the bound label should be located right before the > new instruction, so that the next immediately executed instruction would be > the new instruction. This is like how the labels for control flows work. > > Unfortunately, I understand this is distinct from the ASM model - ASM > visitor model treats this as if this is a tiny tree structure and considers > a type annotation as a subsidiary of an instruction, and this brings > challenges to porting the implementations. Here's some context: we > ClassFile API developers long believed that type annotations interact > poorly with class file transformations because of hard-to-track arbitrary > references such as interface indices, and thus we did not integrate it as a > part of the ClassFile API code streaming model, and we drops type > annotation attributes in transformations by the attribute stability > settings. > > Regards, > Chen Liang > ------------------------------ > *From:* classfile-api-dev on behalf > of Rafael Winterhalter > *Sent:* Saturday, November 9, 2024 3:09 PM > *To:* classfile-api-dev > *Subject:* Offset error when writing type annotations on NEW instructions > > With the most recent build of JDK 24, I discover one test failure in my > test suite when comparing with ASM ( > https://github.com/raphw/asm-jdk-bridge): Any type annotation that is > added to a "new" instruction is added to the subsequent instruction > instead. This might irritate parsers as the label will point to an > instruction that cannot normally be annotated, as NEW is normally followed > by DUP. > > I created a reproducers that compares a javac compiled class and one that > is created with the class file API. The error is likely in the attribute > mapper's write method as I cannot reproduce it when testing a class file > reader using the class file API compared to ASM. The reproducer is as > follows and is visualized in the first line of each of the two > representations where the correct byte code offset is 0, not 3 as the > annotation is added to the first instruction in the method: > > > import java.io.InputStream; > import java.lang.annotation.ElementType; > import java.lang.annotation.Retention; > import java.lang.annotation.RetentionPolicy; > import java.lang.annotation.Target; > import java.lang.classfile.*; > import java.lang.classfile.attribute.RuntimeVisibleTypeAnnotationsAttribute; > import java.lang.classfile.instruction.LineNumber; > import java.lang.classfile.instruction.LocalVariable; > import java.lang.constant.ClassDesc; > import java.lang.constant.MethodTypeDesc; > import java.util.List; > > public class TypeAnnotationSample { > > void m() { > new @A Object(); > } > > public static void main(String[] args) throws Exception { > byte[] compiled; > try (InputStream inputStream = TypeAnnotationSample.class.getClassLoader().getResourceAsStream( > TypeAnnotationSample.class.getName().replace('.', '/') + ".class" > )) { > compiled = inputStream.readAllBytes(); > } > print(compiled); > System.out.println("-----"); > byte[] created = ClassFile.of().build(ClassDesc.of("Synthetic"), classBuilder -> { > classBuilder.withMethod("m", MethodTypeDesc.of(void.class.describeConstable().orElseThrow()), 0, methodBuilder -> { > methodBuilder.withCode(codeBuilder -> { > codeBuilder.new_(Object.class.describeConstable().orElseThrow()); > Label label = codeBuilder.newBoundLabel(); > codeBuilder.with(RuntimeVisibleTypeAnnotationsAttribute.of(TypeAnnotation.of( > TypeAnnotation.TargetInfo.ofNewExpr(label), > List.of(), > Annotation.of(A.class.describeConstable().orElseThrow()) > ))); > codeBuilder.dup() > .invokespecial(Object.class.describeConstable().orElseThrow(), "", MethodTypeDesc.of(void.class.describeConstable().orElseThrow())) > .pop() > .return_(); > }); > }); > }); > print(created); > } > > private static void print(byte[] classFile) { > ClassModel classModel = ClassFile.of().parse(classFile); > MethodModel methodModel = classModel.methods().stream() > .filter(element -> element.methodName().equalsString("m")) > .findFirst() > .orElseThrow(); > CodeModel codeModel = methodModel.code().orElseThrow(); > codeModel.elementStream().forEach(element -> { > switch (element) { > case LocalVariable _ -> { } > case LineNumber _ -> { } > case RuntimeVisibleTypeAnnotationsAttribute a -> System.out.println(a.annotations().stream() > .map(x -> x.annotation() > + "@" + x.targetPath() > + "/" + ((TypeAnnotation.OffsetTarget) x.targetInfo()).target()) > .toList()); > default -> System.out.println(element); > } > }); > } > > @Retention(RetentionPolicy.RUNTIME) > @Target(ElementType.TYPE_USE) > @interface A { > } > } > > > I think that the attribute mapper must subtract the bytes of the > instruction as it is currently adding the bytes at the end of the > instruction, not at its beginning. ASM and javac set the annotation offset > to the beginning of the statement, so I understand that this should be the > right approach. > > Thanks, Rafael > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Mon Nov 11 09:35:31 2024 From: adam.sotona at oracle.com (Adam Sotona) Date: Mon, 11 Nov 2024 09:35:31 +0000 Subject: [External] : Re: Mapping byte code offsets in custom code attributes In-Reply-To: References: Message-ID: Hi Rafael, Code-level attributes are decomposed to code elements, streamed/transformed as pseudo-instructions together with the bytecode instructions and composed back to the relevant attributes by Class-File API. To insert a new line number, you can call CodeBuilder::lineNumber at required position of code building or transformation. This is the way covering 99% of use cases. Javap is an example of the ?raw? attributes consumers minority. After JDK-8341274 you will be able to receive a Label for a valid bytecode offset in your CustomAttribute mapper. Each Label position is identified by relevant LabelTarget in the CodeModel stream. All targeted labels are resolved to target bytecode offsets automatically for known code elements and attributes. All targeted labels will be also resolvable to the target bytecode offsets in custom attribute mappers after JDK-8341274. Unfortunately, class file transformation is so complex process with many different use cases, aspects, and contradicting priorities that we cannot always provide equal level of support for everything with shining perfect API symmetry. I think custom code attributes operating with bytecode offsets should be provided with at least basic level of Class-File API support. However I would not expect the same seamless integration as for the JVMS-defined attributes. Adam From: Rafael Winterhalter Date: Monday, 11 November 2024 at 9:42 To: Adam Sotona Cc: Chen Liang , classfile-api-dev Subject: [External] : Re: Mapping byte code offsets in custom code attributes Hei Adam, I agree that most use cases should be covered by today's API. I already use the former in my bridge implementation to use ASM with the Class File API. When it comes to LineNumberTableAttribute, I do not understand how it can be useful to resolve the remaining 1%, though. LineNumberInfo is built around a "startPc". Without an opportunity to resolve a BCI for a generated method, I do not understand how I can create a custom line number table attribute using the Class File API. The CodeBuilder interface does not expose the created CodeAttribute, which would be needed to resolve a label position to the needed BCI that the LineNumberInfo class requires. At a minimum, I would need CodeBuilder to resolve the current byte code index, but in the end this would only be used to build a custom label mechanism. I argue that this should already be handled by the API. The same goes for reading. I can of course count the size of each encountered instruction and keep a custom mapping of attributed BCIs to invoke a custom callback whenever a relevant offset is discovered, but this is also what the Label API is used for, and I think the API should handle this mapping. I find the same limitation with LocalVariableTypeInfo, LocalVariableInfo and CharacterRangeInfo. I also believe this was considered for type annotations where TypeAnnotation.TargetInfo.OffsetTarget resolves a label, and not a BCI. I expected consistent handling for all attributes other than type annotations, including custom attributes. Best regards, Rafael Am Mo., 11. Nov. 2024 um 09:17 Uhr schrieb Adam Sotona >: Hi Rafael, LineNumber, LocalVariable and LocalVariableType (as CodeElements) are integral parts of the Class-File API and they should serve well to 99% of user use cases. In addition, LineNumberTableAttribute, LocalVariableTableAttribute, LocalVariableTypeTableAttribute and StackMapTableAttribute are available in ?raw? forms to support strongly requested specific corner cases. This is clear request to support custom code-level attributes with references to bytecode offsets. I agree with Chen that JDK-8341274 should be sufficient to resolve it. Thank you, Adam From: classfile-api-dev > on behalf of Chen Liang > Date: Monday, 11 November 2024 at 0:28 To: Rafael Winterhalter >, classfile-api-dev > Subject: Re: Mapping byte code offsets in custom code attributes Hi Rafael, You are correct. I had previously created an issue https://bugs.openjdk.org/browse/JDK-8341274 to track this demand. Since this scenario has no way to bypass, maybe we can raise this issue to P3 and work on this feature for JDK 24. Also I wonder how frequently are custom attributes storing BCIs? For LineNumberInfo: I believe the Label-based version, LineNumber, is already streamed in class file reading, and can be sent to the builder also via CodeBuilder::lineNumber(int). Your particular use case can probably be covered by the RFE too, once we can convert labels to BCIs in non-Code attributes. Regards, Chen Liang ________________________________ From: classfile-api-dev > on behalf of Rafael Winterhalter > Sent: Saturday, November 9, 2024 2:32 AM To: classfile-api-dev > Subject: Mapping byte code offsets in custom code attributes Hello, I mentioned before that I think there is still a missing bit when it comes to custom code attributes, and mapping them using an AttributeMapper. Many code attributes point to a location, or area, within a method. This typically implies specifying one or two byte code offsets within an attribute. To implement this, I would need a possibility to translate offsets to labels within an attribute mapper, and vice versa. For example, if I created byte code from a different source, where a method consists of multiple files, I would want to mark what part of the method comes from what file. I do not currently know of a way to do this in the Class File API. To resolve this, I would have hoped that the attribute reader method would be able to issue labels that are later issued when the code instructions are supplied. Similarly, I would have hoped that the attribute writer method would be capable of translating labels to byte code offsets. In the same way, I would have hoped that all attributes such as LineNumberInfo of the LineNumberTableAttribute would expose bytecode offsets as labels and not as integers, as the latter does not really give any useful information by itself. This way, I could also write a custom LineNumberTableAttribute after a method is already created by issuing relevant labels, but by only writing the attribute when the entire method was visited. Sometimes this is necessary when generating code, when the information about the actual line numbers is only retrieved at a later stage that requires knowledge of the entire method. As I understand it, the ClassFile API is supposed to be released in Java 24. Are there still plans to integrate such a change? After that it would require incompatible changes, therefore I had hoped this would make it into the final version. Thanks! Rafael -------------- next part -------------- An HTML attachment was scrubbed... URL: From rafael.wth at gmail.com Mon Nov 11 09:43:51 2024 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Mon, 11 Nov 2024 10:43:51 +0100 Subject: Offset error when writing type annotations on NEW instructions In-Reply-To: References: Message-ID: Hei Chen, just to let you know: I found an easy way to implement this by simply creating a buffer of one Instruction where I can add instructions either before or after the buffer where the former is now applied for type-annotations. This way I do not need to create a label for each BCI and my britch now yields the correct results. Best regards, Rafael Am Mo., 11. Nov. 2024 um 10:17 Uhr schrieb Rafael Winterhalter < rafael.wth at gmail.com>: > Hei Chen, > > I think I understand the problem then. With ASM, type annotations on > elements are processed just after the element was encountered. From the > javadoc of ASM: "Visits an annotation on an instruction. This method must > be called just after the annotated instruction." In contrast, the Class > File API requires a label to be inserted "just before" the instruction is > added. > > Ideally, CodeBuilder would allow me to call something like > "newBoundLabelOnPreviousInstruction", but I understand if you do not want > to add something like it. If the Class File API allowed to add labels both > prior or after encountering an instruction, one could work with both > approaches of inserting type annotations prior or after an instruction is > added. This would make migrating from ASM more straight-forward, too, as I > assume that this is the prevalent model today with ASM being so dominant. > > Otherwise, I will need to work around and delay any ASM invocation for my > bridge, to consider if a subsequent call represents a type annotation and > apply a reordering this way. This is not a big deal, but of course it would > be much easier solved if I could bind a label with a BCI offset further up > the stack. Possibly, I could even add a label before any annotatable > instruction, just in case. Would this be overly expensive to do? > > Cheers, Rafael > > Am Mo., 11. Nov. 2024 um 00:12 Uhr schrieb Chen Liang < > chen.l.liang at oracle.com>: > >> Hi Rafael, >> I believe this is due to a misunderstanding with how ClassFile API labels >> work. A ClassFile API label models an insertion point represented by a >> BCI; so in your example, the bound label should be located right before the >> new instruction, so that the next immediately executed instruction would be >> the new instruction. This is like how the labels for control flows work. >> >> Unfortunately, I understand this is distinct from the ASM model - ASM >> visitor model treats this as if this is a tiny tree structure and considers >> a type annotation as a subsidiary of an instruction, and this brings >> challenges to porting the implementations. Here's some context: we >> ClassFile API developers long believed that type annotations interact >> poorly with class file transformations because of hard-to-track arbitrary >> references such as interface indices, and thus we did not integrate it as a >> part of the ClassFile API code streaming model, and we drops type >> annotation attributes in transformations by the attribute stability >> settings. >> >> Regards, >> Chen Liang >> ------------------------------ >> *From:* classfile-api-dev on behalf >> of Rafael Winterhalter >> *Sent:* Saturday, November 9, 2024 3:09 PM >> *To:* classfile-api-dev >> *Subject:* Offset error when writing type annotations on NEW instructions >> >> With the most recent build of JDK 24, I discover one test failure in my >> test suite when comparing with ASM ( >> https://github.com/raphw/asm-jdk-bridge): Any type annotation that is >> added to a "new" instruction is added to the subsequent instruction >> instead. This might irritate parsers as the label will point to an >> instruction that cannot normally be annotated, as NEW is normally followed >> by DUP. >> >> I created a reproducers that compares a javac compiled class and one that >> is created with the class file API. The error is likely in the attribute >> mapper's write method as I cannot reproduce it when testing a class file >> reader using the class file API compared to ASM. The reproducer is as >> follows and is visualized in the first line of each of the two >> representations where the correct byte code offset is 0, not 3 as the >> annotation is added to the first instruction in the method: >> >> >> import java.io.InputStream; >> import java.lang.annotation.ElementType; >> import java.lang.annotation.Retention; >> import java.lang.annotation.RetentionPolicy; >> import java.lang.annotation.Target; >> import java.lang.classfile.*; >> import java.lang.classfile.attribute.RuntimeVisibleTypeAnnotationsAttribute; >> import java.lang.classfile.instruction.LineNumber; >> import java.lang.classfile.instruction.LocalVariable; >> import java.lang.constant.ClassDesc; >> import java.lang.constant.MethodTypeDesc; >> import java.util.List; >> >> public class TypeAnnotationSample { >> >> void m() { >> new @A Object(); >> } >> >> public static void main(String[] args) throws Exception { >> byte[] compiled; >> try (InputStream inputStream = TypeAnnotationSample.class.getClassLoader().getResourceAsStream( >> TypeAnnotationSample.class.getName().replace('.', '/') + ".class" >> )) { >> compiled = inputStream.readAllBytes(); >> } >> print(compiled); >> System.out.println("-----"); >> byte[] created = ClassFile.of().build(ClassDesc.of("Synthetic"), classBuilder -> { >> classBuilder.withMethod("m", MethodTypeDesc.of(void.class.describeConstable().orElseThrow()), 0, methodBuilder -> { >> methodBuilder.withCode(codeBuilder -> { >> codeBuilder.new_(Object.class.describeConstable().orElseThrow()); >> Label label = codeBuilder.newBoundLabel(); >> codeBuilder.with(RuntimeVisibleTypeAnnotationsAttribute.of(TypeAnnotation.of( >> TypeAnnotation.TargetInfo.ofNewExpr(label), >> List.of(), >> Annotation.of(A.class.describeConstable().orElseThrow()) >> ))); >> codeBuilder.dup() >> .invokespecial(Object.class.describeConstable().orElseThrow(), "", MethodTypeDesc.of(void.class.describeConstable().orElseThrow())) >> .pop() >> .return_(); >> }); >> }); >> }); >> print(created); >> } >> >> private static void print(byte[] classFile) { >> ClassModel classModel = ClassFile.of().parse(classFile); >> MethodModel methodModel = classModel.methods().stream() >> .filter(element -> element.methodName().equalsString("m")) >> .findFirst() >> .orElseThrow(); >> CodeModel codeModel = methodModel.code().orElseThrow(); >> codeModel.elementStream().forEach(element -> { >> switch (element) { >> case LocalVariable _ -> { } >> case LineNumber _ -> { } >> case RuntimeVisibleTypeAnnotationsAttribute a -> System.out.println(a.annotations().stream() >> .map(x -> x.annotation() >> + "@" + x.targetPath() >> + "/" + ((TypeAnnotation.OffsetTarget) x.targetInfo()).target()) >> .toList()); >> default -> System.out.println(element); >> } >> }); >> } >> >> @Retention(RetentionPolicy.RUNTIME) >> @Target(ElementType.TYPE_USE) >> @interface A { >> } >> } >> >> >> I think that the attribute mapper must subtract the bytes of the >> instruction as it is currently adding the bytes at the end of the >> instruction, not at its beginning. ASM and javac set the annotation offset >> to the beginning of the statement, so I understand that this should be the >> right approach. >> >> Thanks, Rafael >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rafael.wth at gmail.com Mon Nov 11 17:59:34 2024 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Mon, 11 Nov 2024 18:59:34 +0100 Subject: [External] : Re: Mapping byte code offsets in custom code attributes In-Reply-To: References: Message-ID: (I realized that I did not reply-all and wanted to add the mailing list again. Some of the discussion can be found in the quotation below, if anyone is reading.) I cannot describe a use case that goes beyond mere printing where I could work with BCIs in the context of a Class File API represented list of instructions. If I wanted to create a CharacterRangeTableAttribute, I would always have to resolve BCIs to labels. Why not let the AttributeMapper handle this translation consistently but require BCI as input to the factory method? To me, this seems more error prone and less type-safe. Maybe I should ask the question differently: what led to a choice of using labels over BCIs when dealing with stack map frames, and for the runtime (in-)visible type annotations? I think the answer is that those are more convenient, despite that they are inlined. I cannot make sense that two groups of attributes are represented in two different ways, when labels are obviously more expressive. I get that type annotations and stack map frames might be more central attributes, where the others are more often used by power users. But also power users would benefit from the better abstraction, so I think this should be the target? If the answer is that there is no time to adjust the API, I wonder if one should avoid exposing those four other attributes until there is time, especially with Java 24 likely being less adopted compared to Java 25 where Oracle plans to offer LTS. I feel like exposing BCIs is premature, if labels could be used, and it would be a shame to lose the opportunity to use labels within these four info attributes. I already see the BCI factories and methods being deprecated in a future release, so why not aim for the best solution already today? I understand that this API change is out of scope for JDK-8341274, but I would love to see a follow up change that adjusts those four interfaces to use labels and not BCIs, such that the Class File API is consistent. If possible within the release of Java 24, or Java 25 with the BCI methods not exposed before that. Am Mo., 11. Nov. 2024 um 16:19 Uhr schrieb Adam Sotona < adam.sotona at oracle.com>: > Purpose of JDK-8341274 is to enable labels translation for custom > attributes, it means other attributes than LineNumberTable, > LocalVariableTable, LocalVariableTypeTable or CharacterRangeTable. It does > not include any plan to remodel existing architecture. > > > > What exact use case related to the above-mentioned attributes are you > struggling with? > > > > Thanks, > > Adam > > > > *From: *Rafael Winterhalter > *Date: *Monday, 11 November 2024 at 15:44 > *To: *Adam Sotona > *Subject: *Re: [External] : Re: Mapping byte code offsets in custom code > attributes > > Hello, > > > > As a goal of this conversation, I was hoping to give feedback on where I > see limitations of the current API, something that resulted in JDK-8341274. > As I wrote in previous emails on this list, I am evaluating it against a > number of use cases, all of which use ASM as of today. When I speak of > migration, I mean that users of ASM would take the class file API into use > instead. This is why I hoped that JDK-8341274 could be concluded by using > this API to remodel these four info attributes, to be consistent with > TypeAnnotation and UninitializedVerificationTypeInfo. Clearly, if those > attributes are using labels instead of BCIs, there should be a benefit of > using labels exclusively? I had hoped that the Class File API could become > as consistent as ASM in this regard. > > > > I also understand that the mentioned info elements are streamlined with > the code instruction API. But as the use of code generation has evolved > upon ASM over many years, where attributes are not always generated at the > same time as the attributed code, but later. Rather, one notes down BCIs as > labels in the generated code and writes the attribute upon completion. In > some cases, such logic can be rewritten to fit the streamlined model. But > sometimes this is really hard. Neither the class file format, nor the class > file API require that attributes on code are created simultaneously, so I > was hoping that explicit BCIs could be removed from the public API > altogether. Today, they only appear at four locations (LocalVariable, > LocalVariableType, CharacterRange and LineNumber) throughout the entire > Class File API. Anywhere else, BCIs are resolved as labels. I think > changing that would add to the Class File APIs quality, consistency and > usability, when working with these attributes, similar to how it is > suggested to work with custom ones. I hoped I could argue that case before > the API becomes final and this can no longer be changed. > > > > Patching line numbers is indeed supported, if one reiterates over the > code, but I would not call this a seamless solution in contrast to just > writing that attribute after completing the code attribute, if this is what > is desired. > > > > Thanks, Rafael > > > > Am Mo., 11. Nov. 2024 um 14:38 Uhr schrieb Adam Sotona < > adam.sotona at oracle.com>: > > Hi Rafael, > > I?m not sure what is goal of this discussion. Let me answer your questions > below. > > > > *From: *Rafael Winterhalter > > > > - What I do not understand is why this model should not be applied for > LineNumberInfo, LocalVariableTypeInfo, LocalVariableInfo and > CharacterRangeInfo? > > > > LocalVariable, LocalVariableType, CharacterRange and LineNumber are pseudo > instructions (code elements). As such they are integral part of the > CodeModel and they are streamed and transformed with the instructions. LineNumberInfo, > LocalVariableTypeInfo, LocalVariableInfo are just inner structure data > holders of the corresponding attributes and they serve only to specific use > cases requesting the raw values. > > > > > > - Once a translation mechanism is in place, it should be > straightforward to migrate? > > > > I?m not sure what migration do you have in mind. > > > > > > - What is the point of offering these attributes in a (public) API > then, if not as ready for consumption? > > > > There are regular consumers of the API, like for example javap. > > > > > > - As I see it: Without labels, I would need to implement a custom > mapper, for example for writing a custom LineNumberTableAttribute, > > > > There are two aspects: > > 1. A custom LineNumberTable attribute mapper overriding the default > one is not accepted by the API. > 2. Implementation of a custom attribute similar to LineNumberTable > attribute is supported and after JDK-8341274 the support will improve. > > > > > > - if I wanted to patch line numbers after creating a method. > > > > Patching line numbers is seamlessly supported. CodeTransform can freely > change, add, and remove line numbers. > > > > > > From the context I understand you call for indirect translation of each > (currently inline) LocalVariable, LocalVariableType, CharacterRange and > LineNumber into a Label-parametrized pseudo instructions + corresponding > LabelTargets. Such model has been rejected in the early Class-File API > prototypes for significant negative performance impact. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Mon Nov 11 20:21:11 2024 From: adam.sotona at oracle.com (Adam Sotona) Date: Mon, 11 Nov 2024 20:21:11 +0000 Subject: [External] : Re: Mapping byte code offsets in custom code attributes In-Reply-To: References: Message-ID: Hi Rafael, Here is an example method: static void m() { int a = 1; int b = 2; a = 3; b = 4; } This is a sample transformation manipulating line numbers and local variables: ClassFile.of().transformClass(clm, ClassTransform.transformingMethodBodies((cob, coe) -> { switch (coe) { case LineNumber ln -> cob.lineNumber(ln.line() + 100); // change line numers case LocalVariable lv when lv.name().equalsString("a") -> cob.localVariable(lv.slot(), "x", lv.typeSymbol(), cob.startLabel(), lv.endScope()); // rename var ?a? to ?x? and change its start scope case LocalVariable _ -> {} // drop var ?b? default -> cob.with(coe); } } ).andThen(ClassTransform.transformingMethodBodies(CodeTransform.endHandler(cob -> cob .localVariable(1, "y", ConstantDescs.CD_int, cob.startLabel(), cob.endLabel()) // add var ?y? .lineNumber(999))))); // add line number And as you can see there are no custom attributes nor bytecode offset conversions necessary. Yes, Class-File API contains several ?raw? points where bytecode offsets are intentionally exposed to support specific use cases, however no one is obliged to use them. I hope this finally addressed your questions. Adam From: Rafael Winterhalter Date: Monday, 11 November 2024 at 18:59 To: Adam Sotona , classfile-api-dev , Chen Liang Subject: Re: [External] : Re: Mapping byte code offsets in custom code attributes (I realized that I did not reply-all and wanted to add the mailing list again. Some of the discussion can be found in the quotation below, if anyone is reading.) I cannot describe a use case that goes beyond mere printing where I could work with BCIs in the context of a Class File API represented list of instructions. If I wanted to create a CharacterRangeTableAttribute, I would always have to resolve BCIs to labels. Why not let the AttributeMapper handle this translation consistently but require BCI as input to the factory method? To me, this seems more error prone and less type-safe. Maybe I should ask the question differently: what led to a choice of using labels over BCIs when dealing with stack map frames, and for the runtime (in-)visible type annotations? I think the answer is that those are more convenient, despite that they are inlined. I cannot make sense that two groups of attributes are represented in two different ways, when labels are obviously more expressive. I get that type annotations and stack map frames might be more central attributes, where the others are more often used by power users. But also power users would benefit from the better abstraction, so I think this should be the target? If the answer is that there is no time to adjust the API, I wonder if one should avoid exposing those four other attributes until there is time, especially with Java 24 likely being less adopted compared to Java 25 where Oracle plans to offer LTS. I feel like exposing BCIs is premature, if labels could be used, and it would be a shame to lose the opportunity to use labels within these four info attributes. I already see the BCI factories and methods being deprecated in a future release, so why not aim for the best solution already today? I understand that this API change is out of scope for JDK-8341274, but I would love to see a follow up change that adjusts those four interfaces to use labels and not BCIs, such that the Class File API is consistent. If possible within the release of Java 24, or Java 25 with the BCI methods not exposed before that. Am Mo., 11. Nov. 2024 um 16:19 Uhr schrieb Adam Sotona >: Purpose of JDK-8341274 is to enable labels translation for custom attributes, it means other attributes than LineNumberTable, LocalVariableTable, LocalVariableTypeTable or CharacterRangeTable. It does not include any plan to remodel existing architecture. What exact use case related to the above-mentioned attributes are you struggling with? Thanks, Adam From: Rafael Winterhalter > Date: Monday, 11 November 2024 at 15:44 To: Adam Sotona > Subject: Re: [External] : Re: Mapping byte code offsets in custom code attributes Hello, As a goal of this conversation, I was hoping to give feedback on where I see limitations of the current API, something that resulted in JDK-8341274. As I wrote in previous emails on this list, I am evaluating it against a number of use cases, all of which use ASM as of today. When I speak of migration, I mean that users of ASM would take the class file API into use instead. This is why I hoped that JDK-8341274 could be concluded by using this API to remodel these four info attributes, to be consistent with TypeAnnotation and UninitializedVerificationTypeInfo. Clearly, if those attributes are using labels instead of BCIs, there should be a benefit of using labels exclusively? I had hoped that the Class File API could become as consistent as ASM in this regard. I also understand that the mentioned info elements are streamlined with the code instruction API. But as the use of code generation has evolved upon ASM over many years, where attributes are not always generated at the same time as the attributed code, but later. Rather, one notes down BCIs as labels in the generated code and writes the attribute upon completion. In some cases, such logic can be rewritten to fit the streamlined model. But sometimes this is really hard. Neither the class file format, nor the class file API require that attributes on code are created simultaneously, so I was hoping that explicit BCIs could be removed from the public API altogether. Today, they only appear at four locations (LocalVariable, LocalVariableType, CharacterRange and LineNumber) throughout the entire Class File API. Anywhere else, BCIs are resolved as labels. I think changing that would add to the Class File APIs quality, consistency and usability, when working with these attributes, similar to how it is suggested to work with custom ones. I hoped I could argue that case before the API becomes final and this can no longer be changed. Patching line numbers is indeed supported, if one reiterates over the code, but I would not call this a seamless solution in contrast to just writing that attribute after completing the code attribute, if this is what is desired. Thanks, Rafael Am Mo., 11. Nov. 2024 um 14:38 Uhr schrieb Adam Sotona >: Hi Rafael, I?m not sure what is goal of this discussion. Let me answer your questions below. From: Rafael Winterhalter > * What I do not understand is why this model should not be applied for LineNumberInfo, LocalVariableTypeInfo, LocalVariableInfo and CharacterRangeInfo? LocalVariable, LocalVariableType, CharacterRange and LineNumber are pseudo instructions (code elements). As such they are integral part of the CodeModel and they are streamed and transformed with the instructions. LineNumberInfo, LocalVariableTypeInfo, LocalVariableInfo are just inner structure data holders of the corresponding attributes and they serve only to specific use cases requesting the raw values. * Once a translation mechanism is in place, it should be straightforward to migrate? I?m not sure what migration do you have in mind. * What is the point of offering these attributes in a (public) API then, if not as ready for consumption? There are regular consumers of the API, like for example javap. * As I see it: Without labels, I would need to implement a custom mapper, for example for writing a custom LineNumberTableAttribute, There are two aspects: 1. A custom LineNumberTable attribute mapper overriding the default one is not accepted by the API. 2. Implementation of a custom attribute similar to LineNumberTable attribute is supported and after JDK-8341274 the support will improve. * if I wanted to patch line numbers after creating a method. Patching line numbers is seamlessly supported. CodeTransform can freely change, add, and remove line numbers. >From the context I understand you call for indirect translation of each (currently inline) LocalVariable, LocalVariableType, CharacterRange and LineNumber into a Label-parametrized pseudo instructions + corresponding LabelTargets. Such model has been rejected in the early Class-File API prototypes for significant negative performance impact. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rafael.wth at gmail.com Mon Nov 11 22:57:16 2024 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Mon, 11 Nov 2024 23:57:16 +0100 Subject: [External] : Re: Mapping byte code offsets in custom code attributes In-Reply-To: References: Message-ID: I fully agree that the current API covers 99% of the use cases. However, in ASM, labels are visited before line numbers. The actual line numbers can be patched in at any time. If you are working with a large ASM-based application, and you want to migrate this application to using the Class File API, this migration is much less straight-forward. At the same time, this is an API limitation, and not inherent with the class file API. It feels like an unnecessary burden that could easily be resolved. Am Mo., 11. Nov. 2024 um 21:21 Uhr schrieb Adam Sotona < adam.sotona at oracle.com>: > Hi Rafael, > > Here is an example method: > > > > static void *m*() { > > int a = 1; > > int b = 2; > > a = 3; > > b = 4; > > } > > > > This is a sample transformation manipulating line numbers and local > variables: > > ClassFile.*of*().transformClass(clm, ClassTransform. > *transformingMethodBodies*((cob, coe) -> { > > switch (coe) { > > case LineNumber ln -> > > cob.lineNumber(ln.line() + 100); // change line numers > > case LocalVariable lv when lv.name().equalsString("a") -> > > cob.localVariable(lv.slot(), "x", lv.typeSymbol(), > cob.startLabel(), lv.endScope()); // rename var ?a? to ?x? and change its > start scope > > case LocalVariable _ -> {} // drop var ?b? > > default -> cob.with(coe); > > } > > } > > ).andThen(ClassTransform.*transformingMethodBodies*(CodeTransform. > *endHandler*(cob -> cob > > .localVariable(1, "y", ConstantDescs.*CD_int*, > cob.startLabel(), cob.endLabel()) // add var ?y? > > .lineNumber(999))))); // add line number > > > > And as you can see there are no custom attributes nor bytecode offset > conversions necessary. > > > > Yes, Class-File API contains several ?raw? points where bytecode offsets > are intentionally exposed to support specific use cases, however no one is > obliged to use them. > > > > I hope this finally addressed your questions. > > > > Adam > > > > *From: *Rafael Winterhalter > *Date: *Monday, 11 November 2024 at 18:59 > *To: *Adam Sotona , classfile-api-dev < > classfile-api-dev at openjdk.org>, Chen Liang > *Subject: *Re: [External] : Re: Mapping byte code offsets in custom code > attributes > > (I realized that I did not reply-all and wanted to add the mailing list > again. Some of the discussion can be found in the quotation below, if > anyone is reading.) > > > > I cannot describe a use case that goes beyond mere printing where I could > work with BCIs in the context of a Class File API represented list of > instructions. If I wanted to create a CharacterRangeTableAttribute, I would > always have to resolve BCIs to labels. Why not let the AttributeMapper > handle this translation consistently but require BCI as input to the > factory method? To me, this seems more error prone and less type-safe. > > > > Maybe I should ask the question differently: what led to a choice of using > labels over BCIs when dealing with stack map frames, and for the runtime > (in-)visible type annotations? I think the answer is that those are more > convenient, despite that they are inlined. I cannot make sense that two > groups of attributes are represented in two different ways, when labels are > obviously more expressive. I get that type annotations and stack map frames > might be more central attributes, where the others are more often used by > power users. But also power users would benefit from the better > abstraction, so I think this should be the target? If the answer is that > there is no time to adjust the API, I wonder if one should avoid exposing > those four other attributes until there is time, especially with Java 24 > likely being less adopted compared to Java 25 where Oracle plans to offer > LTS. I feel like exposing BCIs is premature, if labels could be used, and > it would be a shame to lose the opportunity to use labels within these four > info attributes. I already see the BCI factories and methods being > deprecated in a future release, so why not aim for the best solution > already today? > > > > I understand that this API change is out of scope for JDK-8341274, but I > would love to see a follow up change that adjusts those four interfaces to > use labels and not BCIs, such that the Class File API is consistent. If > possible within the release of Java 24, or Java 25 with the BCI methods not > exposed before that. > > > > Am Mo., 11. Nov. 2024 um 16:19 Uhr schrieb Adam Sotona < > adam.sotona at oracle.com>: > > Purpose of JDK-8341274 is to enable labels translation for custom > attributes, it means other attributes than LineNumberTable, > LocalVariableTable, LocalVariableTypeTable or CharacterRangeTable. It does > not include any plan to remodel existing architecture. > > > > What exact use case related to the above-mentioned attributes are you > struggling with? > > > > Thanks, > > Adam > > > > *From: *Rafael Winterhalter > *Date: *Monday, 11 November 2024 at 15:44 > *To: *Adam Sotona > *Subject: *Re: [External] : Re: Mapping byte code offsets in custom code > attributes > > Hello, > > > > As a goal of this conversation, I was hoping to give feedback on where I > see limitations of the current API, something that resulted in JDK-8341274. > As I wrote in previous emails on this list, I am evaluating it against a > number of use cases, all of which use ASM as of today. When I speak of > migration, I mean that users of ASM would take the class file API into use > instead. This is why I hoped that JDK-8341274 could be concluded by using > this API to remodel these four info attributes, to be consistent with > TypeAnnotation and UninitializedVerificationTypeInfo. Clearly, if those > attributes are using labels instead of BCIs, there should be a benefit of > using labels exclusively? I had hoped that the Class File API could become > as consistent as ASM in this regard. > > > > I also understand that the mentioned info elements are streamlined with > the code instruction API. But as the use of code generation has evolved > upon ASM over many years, where attributes are not always generated at the > same time as the attributed code, but later. Rather, one notes down BCIs as > labels in the generated code and writes the attribute upon completion. In > some cases, such logic can be rewritten to fit the streamlined model. But > sometimes this is really hard. Neither the class file format, nor the class > file API require that attributes on code are created simultaneously, so I > was hoping that explicit BCIs could be removed from the public API > altogether. Today, they only appear at four locations (LocalVariable, > LocalVariableType, CharacterRange and LineNumber) throughout the entire > Class File API. Anywhere else, BCIs are resolved as labels. I think > changing that would add to the Class File APIs quality, consistency and > usability, when working with these attributes, similar to how it is > suggested to work with custom ones. I hoped I could argue that case before > the API becomes final and this can no longer be changed. > > > > Patching line numbers is indeed supported, if one reiterates over the > code, but I would not call this a seamless solution in contrast to just > writing that attribute after completing the code attribute, if this is what > is desired. > > > > Thanks, Rafael > > > > Am Mo., 11. Nov. 2024 um 14:38 Uhr schrieb Adam Sotona < > adam.sotona at oracle.com>: > > Hi Rafael, > > I?m not sure what is goal of this discussion. Let me answer your questions > below. > > > > *From: *Rafael Winterhalter > > > > - What I do not understand is why this model should not be applied for > LineNumberInfo, LocalVariableTypeInfo, LocalVariableInfo and > CharacterRangeInfo? > > > > LocalVariable, LocalVariableType, CharacterRange and LineNumber are pseudo > instructions (code elements). As such they are integral part of the > CodeModel and they are streamed and transformed with the instructions. LineNumberInfo, > LocalVariableTypeInfo, LocalVariableInfo are just inner structure data > holders of the corresponding attributes and they serve only to specific use > cases requesting the raw values. > > > > > > - Once a translation mechanism is in place, it should be > straightforward to migrate? > > > > I?m not sure what migration do you have in mind. > > > > > > - What is the point of offering these attributes in a (public) API > then, if not as ready for consumption? > > > > There are regular consumers of the API, like for example javap. > > > > > > - As I see it: Without labels, I would need to implement a custom > mapper, for example for writing a custom LineNumberTableAttribute, > > > > There are two aspects: > > 1. A custom LineNumberTable attribute mapper overriding the default > one is not accepted by the API. > 2. Implementation of a custom attribute similar to LineNumberTable > attribute is supported and after JDK-8341274 the support will improve. > > > > > > - if I wanted to patch line numbers after creating a method. > > > > Patching line numbers is seamlessly supported. CodeTransform can freely > change, add, and remove line numbers. > > > > > > From the context I understand you call for indirect translation of each > (currently inline) LocalVariable, LocalVariableType, CharacterRange and > LineNumber into a Label-parametrized pseudo instructions + corresponding > LabelTargets. Such model has been rejected in the early Class-File API > prototypes for significant negative performance impact. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chen.l.liang at oracle.com Tue Nov 12 16:21:44 2024 From: chen.l.liang at oracle.com (Chen Liang) Date: Tue, 12 Nov 2024 16:21:44 +0000 Subject: [External] : Re: Mapping byte code offsets in custom code attributes In-Reply-To: References: Message-ID: Hi Rafael, I think the particular arrangement ASM has for line numbers is probably due to that line number was a later addition to a very early version of ASM: the label argument is largely redundant. If the extra label is useful, I guess that will be added for type-use annotations in code too, which ASM obviously didn't do. Also, I don't see the point of patching line numbers: this still requires you to know the labels ahead of time. If you really, really need this pattern, you can first generate methods in a class, and process with ClassBuilder::transformingMethods (which, at this stage, has a complete code view that your line number computation may need to access) to add line numbers. Regards, Chen Liang ________________________________ From: Rafael Winterhalter Sent: Monday, November 11, 2024 4:57 PM To: Adam Sotona Cc: classfile-api-dev ; Chen Liang Subject: Re: [External] : Re: Mapping byte code offsets in custom code attributes I fully agree that the current API covers 99% of the use cases. However, in ASM, labels are visited before line numbers. The actual line numbers can be patched in at any time. If you are working with a large ASM-based application, and you want to migrate this application to using the Class File API, this migration is much less straight-forward. At the same time, this is an API limitation, and not inherent with the class file API. It feels like an unnecessary burden that could easily be resolved. Am Mo., 11. Nov. 2024 um 21:21 Uhr schrieb Adam Sotona >: Hi Rafael, Here is an example method: static void m() { int a = 1; int b = 2; a = 3; b = 4; } This is a sample transformation manipulating line numbers and local variables: ClassFile.of().transformClass(clm, ClassTransform.transformingMethodBodies((cob, coe) -> { switch (coe) { case LineNumber ln -> cob.lineNumber(ln.line() + 100); // change line numers case LocalVariable lv when lv.name().equalsString("a") -> cob.localVariable(lv.slot(), "x", lv.typeSymbol(), cob.startLabel(), lv.endScope()); // rename var ?a? to ?x? and change its start scope case LocalVariable _ -> {} // drop var ?b? default -> cob.with(coe); } } ).andThen(ClassTransform.transformingMethodBodies(CodeTransform.endHandler(cob -> cob .localVariable(1, "y", ConstantDescs.CD_int, cob.startLabel(), cob.endLabel()) // add var ?y? .lineNumber(999))))); // add line number And as you can see there are no custom attributes nor bytecode offset conversions necessary. Yes, Class-File API contains several ?raw? points where bytecode offsets are intentionally exposed to support specific use cases, however no one is obliged to use them. I hope this finally addressed your questions. Adam From: Rafael Winterhalter > Date: Monday, 11 November 2024 at 18:59 To: Adam Sotona >, classfile-api-dev >, Chen Liang > Subject: Re: [External] : Re: Mapping byte code offsets in custom code attributes (I realized that I did not reply-all and wanted to add the mailing list again. Some of the discussion can be found in the quotation below, if anyone is reading.) I cannot describe a use case that goes beyond mere printing where I could work with BCIs in the context of a Class File API represented list of instructions. If I wanted to create a CharacterRangeTableAttribute, I would always have to resolve BCIs to labels. Why not let the AttributeMapper handle this translation consistently but require BCI as input to the factory method? To me, this seems more error prone and less type-safe. Maybe I should ask the question differently: what led to a choice of using labels over BCIs when dealing with stack map frames, and for the runtime (in-)visible type annotations? I think the answer is that those are more convenient, despite that they are inlined. I cannot make sense that two groups of attributes are represented in two different ways, when labels are obviously more expressive. I get that type annotations and stack map frames might be more central attributes, where the others are more often used by power users. But also power users would benefit from the better abstraction, so I think this should be the target? If the answer is that there is no time to adjust the API, I wonder if one should avoid exposing those four other attributes until there is time, especially with Java 24 likely being less adopted compared to Java 25 where Oracle plans to offer LTS. I feel like exposing BCIs is premature, if labels could be used, and it would be a shame to lose the opportunity to use labels within these four info attributes. I already see the BCI factories and methods being deprecated in a future release, so why not aim for the best solution already today? I understand that this API change is out of scope for JDK-8341274, but I would love to see a follow up change that adjusts those four interfaces to use labels and not BCIs, such that the Class File API is consistent. If possible within the release of Java 24, or Java 25 with the BCI methods not exposed before that. Am Mo., 11. Nov. 2024 um 16:19 Uhr schrieb Adam Sotona >: Purpose of JDK-8341274 is to enable labels translation for custom attributes, it means other attributes than LineNumberTable, LocalVariableTable, LocalVariableTypeTable or CharacterRangeTable. It does not include any plan to remodel existing architecture. What exact use case related to the above-mentioned attributes are you struggling with? Thanks, Adam From: Rafael Winterhalter > Date: Monday, 11 November 2024 at 15:44 To: Adam Sotona > Subject: Re: [External] : Re: Mapping byte code offsets in custom code attributes Hello, As a goal of this conversation, I was hoping to give feedback on where I see limitations of the current API, something that resulted in JDK-8341274. As I wrote in previous emails on this list, I am evaluating it against a number of use cases, all of which use ASM as of today. When I speak of migration, I mean that users of ASM would take the class file API into use instead. This is why I hoped that JDK-8341274 could be concluded by using this API to remodel these four info attributes, to be consistent with TypeAnnotation and UninitializedVerificationTypeInfo. Clearly, if those attributes are using labels instead of BCIs, there should be a benefit of using labels exclusively? I had hoped that the Class File API could become as consistent as ASM in this regard. I also understand that the mentioned info elements are streamlined with the code instruction API. But as the use of code generation has evolved upon ASM over many years, where attributes are not always generated at the same time as the attributed code, but later. Rather, one notes down BCIs as labels in the generated code and writes the attribute upon completion. In some cases, such logic can be rewritten to fit the streamlined model. But sometimes this is really hard. Neither the class file format, nor the class file API require that attributes on code are created simultaneously, so I was hoping that explicit BCIs could be removed from the public API altogether. Today, they only appear at four locations (LocalVariable, LocalVariableType, CharacterRange and LineNumber) throughout the entire Class File API. Anywhere else, BCIs are resolved as labels. I think changing that would add to the Class File APIs quality, consistency and usability, when working with these attributes, similar to how it is suggested to work with custom ones. I hoped I could argue that case before the API becomes final and this can no longer be changed. Patching line numbers is indeed supported, if one reiterates over the code, but I would not call this a seamless solution in contrast to just writing that attribute after completing the code attribute, if this is what is desired. Thanks, Rafael Am Mo., 11. Nov. 2024 um 14:38 Uhr schrieb Adam Sotona >: Hi Rafael, I?m not sure what is goal of this discussion. Let me answer your questions below. From: Rafael Winterhalter > * What I do not understand is why this model should not be applied for LineNumberInfo, LocalVariableTypeInfo, LocalVariableInfo and CharacterRangeInfo? LocalVariable, LocalVariableType, CharacterRange and LineNumber are pseudo instructions (code elements). As such they are integral part of the CodeModel and they are streamed and transformed with the instructions. LineNumberInfo, LocalVariableTypeInfo, LocalVariableInfo are just inner structure data holders of the corresponding attributes and they serve only to specific use cases requesting the raw values. * Once a translation mechanism is in place, it should be straightforward to migrate? I?m not sure what migration do you have in mind. * What is the point of offering these attributes in a (public) API then, if not as ready for consumption? There are regular consumers of the API, like for example javap. * As I see it: Without labels, I would need to implement a custom mapper, for example for writing a custom LineNumberTableAttribute, There are two aspects: 1. A custom LineNumberTable attribute mapper overriding the default one is not accepted by the API. 2. Implementation of a custom attribute similar to LineNumberTable attribute is supported and after JDK-8341274 the support will improve. * if I wanted to patch line numbers after creating a method. Patching line numbers is seamlessly supported. CodeTransform can freely change, add, and remove line numbers. >From the context I understand you call for indirect translation of each (currently inline) LocalVariable, LocalVariableType, CharacterRange and LineNumber into a Label-parametrized pseudo instructions + corresponding LabelTargets. Such model has been rejected in the early Class-File API prototypes for significant negative performance impact. -------------- next part -------------- An HTML attachment was scrubbed... URL: From markro at cs.washington.edu Tue Nov 12 23:45:35 2024 From: markro at cs.washington.edu (Mark Roberts) Date: Tue, 12 Nov 2024 15:45:35 -0800 Subject: local variable creation and manipulation Message-ID: My name is Mark Roberts and I work in the University of Washington PLSE department. We have several tools the currently use BCEL to read and modify class files. I would love to replace that with the new java.lang.classfile package ? and throw away hundreds of line of code to deal with updating and maintaining StackMaps! I have only recently started to study the package so I may be missing somethings completely and please let me know if that is the case. My concern is that I don?t see a high level interface for creating and then using new local variables. I would like to be able to create a method scoped or a blocked scoped local variable and then be able to refer to it symbolically in the various byte code builders. Thank you, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From chen.l.liang at oracle.com Tue Nov 12 23:59:53 2024 From: chen.l.liang at oracle.com (Chen Liang) Date: Tue, 12 Nov 2024 23:59:53 +0000 Subject: local variable creation and manipulation In-Reply-To: References: Message-ID: Hello Mark, In bytecode generation with ASM or ClassFile API, we usually do not use a fully-expanded "local variable." We use a much simpler form, a reference to a local variable slot. Such a slot is usually allocated with `CodeBuilder::allocateLocal(TypeKind)`. If you want to create scope-local variables, first create a scope handler with `CodeBuilder::block(Consumer)`, and the slots allocated in the block will not exist outside of the scope. If you want to create a debug-friendly local variable (type) table entry, use the `localVariable` (used when none of your local variables have generic types) and the `localVariableType` methods. You can pass in the allocated slot and the start and end labels of your blocks (CodeBuilder::startLabel/endLabel). This debug information is also the class-file representation of javac-generated local variables. For stack maps: modern bytecode generation almost always come with a stack map generator; same with the ClassFile API. You don't have to worry about stack maps as the API already takes care of it for you. Regards, Chen Liang ________________________________ From: classfile-api-dev on behalf of Mark Roberts Sent: Tuesday, November 12, 2024 5:45 PM To: classfile-api-dev at openjdk.org Subject: local variable creation and manipulation My name is Mark Roberts and I work in the University of Washington PLSE department. We have several tools the currently use BCEL to read and modify class files. I would love to replace that with the new java.lang.classfile package ? and throw away hundreds of line of code to deal with updating and maintaining StackMaps! I have only recently started to study the package so I may be missing somethings completely and please let me know if that is the case. My concern is that I don?t see a high level interface for creating and then using new local variables. I would like to be able to create a method scoped or a blocked scoped local variable and then be able to refer to it symbolically in the various byte code builders. Thank you, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From rafael.wth at gmail.com Tue Nov 12 17:41:36 2024 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Tue, 12 Nov 2024 18:41:36 +0100 Subject: [External] : Re: Mapping byte code offsets in custom code attributes In-Reply-To: References: Message-ID: Hei Adam, as for ASM, I also think that the inline visiting of code type annotations without a label is a limitation in its API design. It does not apply as much to the tree API where one can walk up the stack to insert an annotation in the correct position, but I have seen this being a hassle in the visitor API, requiring a second pass. Not so much for annotations, but for try-catch-segments, this is a particular problem and I have gone as far as writing the exception table attribute by hand, which ASM allows by implementing a custom Attribute. The class file format is flexible when it comes to setting attributes independently from the attributed code, and I therefore believe that this should be the primary abstraction. I have no problem with the concept of pseudo instructions, but personally, I would have dropped them and asked users to request if label positions represent something that is not related to other instructions. Most people who instrument code do either not care about attributes at all, or they do in what case they exactly know what they want to do, and what attribute to access for such metadata. As you say, everything can be worked around, but this is my belief based on my experience with using ASM. This is why I had hoped that all attributes could still be label-centric and always allow for manual definition after creating the code attribute. This way, power-users get the freedom they want, while others might still want to visit the inline pseudo-instructions for their convenience. Best regards, Rafael Am Di., 12. Nov. 2024 um 17:21 Uhr schrieb Chen Liang < chen.l.liang at oracle.com>: > Hi Rafael, > I think the particular arrangement ASM has for line numbers is probably > due to that line number was a later addition to a very early version of > ASM: the label argument is largely redundant. If the extra label is > useful, I guess that will be added for type-use annotations in code too, > which ASM obviously didn't do. > > Also, I don't see the point of patching line numbers: this still requires > you to know the labels ahead of time. If you really, really need this > pattern, you can first generate methods in a class, and process with > ClassBuilder::transformingMethods (which, at this stage, has a complete > code view that your line number computation may need to access) to add line > numbers. > > Regards, > Chen Liang > ------------------------------ > *From:* Rafael Winterhalter > *Sent:* Monday, November 11, 2024 4:57 PM > *To:* Adam Sotona > *Cc:* classfile-api-dev ; Chen Liang < > chen.l.liang at oracle.com> > *Subject:* Re: [External] : Re: Mapping byte code offsets in custom code > attributes > > I fully agree that the current API covers 99% of the use cases. However, > in ASM, labels are visited before line numbers. The actual line numbers can > be patched in at any time. If you are working with a large ASM-based > application, and you want to migrate this application to using the Class > File API, this migration is much less straight-forward. > > At the same time, this is an API limitation, and not inherent with the > class file API. It feels like an unnecessary burden that could easily be > resolved. > > Am Mo., 11. Nov. 2024 um 21:21 Uhr schrieb Adam Sotona < > adam.sotona at oracle.com>: > > Hi Rafael, > > Here is an example method: > > > > static void *m*() { > > int a = 1; > > int b = 2; > > a = 3; > > b = 4; > > } > > > > This is a sample transformation manipulating line numbers and local > variables: > > ClassFile.*of*().transformClass(clm, ClassTransform. > *transformingMethodBodies*((cob, coe) -> { > > switch (coe) { > > case LineNumber ln -> > > cob.lineNumber(ln.line() + 100); // change line numers > > case LocalVariable lv when lv.name > > ().equalsString("a") -> > > cob.localVariable(lv.slot(), "x", lv.typeSymbol(), > cob.startLabel(), lv.endScope()); // rename var ?a? to ?x? and change its > start scope > > case LocalVariable _ -> {} // drop var ?b? > > default -> cob.with(coe); > > } > > } > > ).andThen(ClassTransform.*transformingMethodBodies*(CodeTransform. > *endHandler*(cob -> cob > > .localVariable(1, "y", ConstantDescs.*CD_int*, > cob.startLabel(), cob.endLabel()) // add var ?y? > > .lineNumber(999))))); // add line number > > > > And as you can see there are no custom attributes nor bytecode offset > conversions necessary. > > > > Yes, Class-File API contains several ?raw? points where bytecode offsets > are intentionally exposed to support specific use cases, however no one is > obliged to use them. > > > > I hope this finally addressed your questions. > > > > Adam > > > > *From: *Rafael Winterhalter > *Date: *Monday, 11 November 2024 at 18:59 > *To: *Adam Sotona , classfile-api-dev < > classfile-api-dev at openjdk.org>, Chen Liang > *Subject: *Re: [External] : Re: Mapping byte code offsets in custom code > attributes > > (I realized that I did not reply-all and wanted to add the mailing list > again. Some of the discussion can be found in the quotation below, if > anyone is reading.) > > > > I cannot describe a use case that goes beyond mere printing where I could > work with BCIs in the context of a Class File API represented list of > instructions. If I wanted to create a CharacterRangeTableAttribute, I would > always have to resolve BCIs to labels. Why not let the AttributeMapper > handle this translation consistently but require BCI as input to the > factory method? To me, this seems more error prone and less type-safe. > > > > Maybe I should ask the question differently: what led to a choice of using > labels over BCIs when dealing with stack map frames, and for the runtime > (in-)visible type annotations? I think the answer is that those are more > convenient, despite that they are inlined. I cannot make sense that two > groups of attributes are represented in two different ways, when labels are > obviously more expressive. I get that type annotations and stack map frames > might be more central attributes, where the others are more often used by > power users. But also power users would benefit from the better > abstraction, so I think this should be the target? If the answer is that > there is no time to adjust the API, I wonder if one should avoid exposing > those four other attributes until there is time, especially with Java 24 > likely being less adopted compared to Java 25 where Oracle plans to offer > LTS. I feel like exposing BCIs is premature, if labels could be used, and > it would be a shame to lose the opportunity to use labels within these four > info attributes. I already see the BCI factories and methods being > deprecated in a future release, so why not aim for the best solution > already today? > > > > I understand that this API change is out of scope for JDK-8341274, but I > would love to see a follow up change that adjusts those four interfaces to > use labels and not BCIs, such that the Class File API is consistent. If > possible within the release of Java 24, or Java 25 with the BCI methods not > exposed before that. > > > > Am Mo., 11. Nov. 2024 um 16:19 Uhr schrieb Adam Sotona < > adam.sotona at oracle.com>: > > Purpose of JDK-8341274 is to enable labels translation for custom > attributes, it means other attributes than LineNumberTable, > LocalVariableTable, LocalVariableTypeTable or CharacterRangeTable. It does > not include any plan to remodel existing architecture. > > > > What exact use case related to the above-mentioned attributes are you > struggling with? > > > > Thanks, > > Adam > > > > *From: *Rafael Winterhalter > *Date: *Monday, 11 November 2024 at 15:44 > *To: *Adam Sotona > *Subject: *Re: [External] : Re: Mapping byte code offsets in custom code > attributes > > Hello, > > > > As a goal of this conversation, I was hoping to give feedback on where I > see limitations of the current API, something that resulted in JDK-8341274. > As I wrote in previous emails on this list, I am evaluating it against a > number of use cases, all of which use ASM as of today. When I speak of > migration, I mean that users of ASM would take the class file API into use > instead. This is why I hoped that JDK-8341274 could be concluded by using > this API to remodel these four info attributes, to be consistent with > TypeAnnotation and UninitializedVerificationTypeInfo. Clearly, if those > attributes are using labels instead of BCIs, there should be a benefit of > using labels exclusively? I had hoped that the Class File API could become > as consistent as ASM in this regard. > > > > I also understand that the mentioned info elements are streamlined with > the code instruction API. But as the use of code generation has evolved > upon ASM over many years, where attributes are not always generated at the > same time as the attributed code, but later. Rather, one notes down BCIs as > labels in the generated code and writes the attribute upon completion. In > some cases, such logic can be rewritten to fit the streamlined model. But > sometimes this is really hard. Neither the class file format, nor the class > file API require that attributes on code are created simultaneously, so I > was hoping that explicit BCIs could be removed from the public API > altogether. Today, they only appear at four locations (LocalVariable, > LocalVariableType, CharacterRange and LineNumber) throughout the entire > Class File API. Anywhere else, BCIs are resolved as labels. I think > changing that would add to the Class File APIs quality, consistency and > usability, when working with these attributes, similar to how it is > suggested to work with custom ones. I hoped I could argue that case before > the API becomes final and this can no longer be changed. > > > > Patching line numbers is indeed supported, if one reiterates over the > code, but I would not call this a seamless solution in contrast to just > writing that attribute after completing the code attribute, if this is what > is desired. > > > > Thanks, Rafael > > > > Am Mo., 11. Nov. 2024 um 14:38 Uhr schrieb Adam Sotona < > adam.sotona at oracle.com>: > > Hi Rafael, > > I?m not sure what is goal of this discussion. Let me answer your questions > below. > > > > *From: *Rafael Winterhalter > > > > - What I do not understand is why this model should not be applied for > LineNumberInfo, LocalVariableTypeInfo, LocalVariableInfo and > CharacterRangeInfo? > > > > LocalVariable, LocalVariableType, CharacterRange and LineNumber are pseudo > instructions (code elements). As such they are integral part of the > CodeModel and they are streamed and transformed with the instructions. LineNumberInfo, > LocalVariableTypeInfo, LocalVariableInfo are just inner structure data > holders of the corresponding attributes and they serve only to specific use > cases requesting the raw values. > > > > > > - Once a translation mechanism is in place, it should be > straightforward to migrate? > > > > I?m not sure what migration do you have in mind. > > > > > > - What is the point of offering these attributes in a (public) API > then, if not as ready for consumption? > > > > There are regular consumers of the API, like for example javap. > > > > > > - As I see it: Without labels, I would need to implement a custom > mapper, for example for writing a custom LineNumberTableAttribute, > > > > There are two aspects: > > 1. A custom LineNumberTable attribute mapper overriding the default > one is not accepted by the API. > 2. Implementation of a custom attribute similar to LineNumberTable > attribute is supported and after JDK-8341274 the support will improve. > > > > > > - if I wanted to patch line numbers after creating a method. > > > > Patching line numbers is seamlessly supported. CodeTransform can freely > change, add, and remove line numbers. > > > > > > From the context I understand you call for indirect translation of each > (currently inline) LocalVariable, LocalVariableType, CharacterRange and > LineNumber into a Label-parametrized pseudo instructions + corresponding > LabelTargets. Such model has been rejected in the early Class-File API > prototypes for significant negative performance impact. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Wed Nov 13 09:56:02 2024 From: adam.sotona at oracle.com (Adam Sotona) Date: Wed, 13 Nov 2024 09:56:02 +0000 Subject: [External] : Re: Mapping byte code offsets in custom code attributes In-Reply-To: References: Message-ID: Hi Rafael, Please correct me if I understand it wrong. In your email I see following requests: 1. Enable an option to override standard attribute mapper with a custom. Overriding of standard attribute mappers is not allowed in the Class-File API as it would result in nondeterministic behavior. Standard attributes parsing, transformation and generation is always handled by Class-File API. 1. Allow out-of-order construction of LineNumberTable attribute from labels. LineNumber pseudo-instructions are inlined for performance reason. Inflation of a label for each line number will multiply number of label targets and degrade performance of Class-File API and significantly degrade performance of all flow-graph construction transformations. You can always create a two-pass transformation if you insist on out-of-order line numbers insertion. There is nothing preventing you to insert labels to the line number positions (or replace the line numbers with labels) in a first pass, transform your own aggregated map and apply the changes (if any) in a second pass. Code transformations can be chained so you do not lose much on performance. Adam From: classfile-api-dev on behalf of Rafael Winterhalter Date: Wednesday, 13 November 2024 at 8:51 To: Chen Liang Cc: classfile-api-dev Subject: Re: [External] : Re: Mapping byte code offsets in custom code attributes Hei Adam, as for ASM, I also think that the inline visiting of code type annotations without a label is a limitation in its API design. It does not apply as much to the tree API where one can walk up the stack to insert an annotation in the correct position, but I have seen this being a hassle in the visitor API, requiring a second pass. Not so much for annotations, but for try-catch-segments, this is a particular problem and I have gone as far as writing the exception table attribute by hand, which ASM allows by implementing a custom Attribute. The class file format is flexible when it comes to setting attributes independently from the attributed code, and I therefore believe that this should be the primary abstraction. I have no problem with the concept of pseudo instructions, but personally, I would have dropped them and asked users to request if label positions represent something that is not related to other instructions. Most people who instrument code do either not care about attributes at all, or they do in what case they exactly know what they want to do, and what attribute to access for such metadata. As you say, everything can be worked around, but this is my belief based on my experience with using ASM. This is why I had hoped that all attributes could still be label-centric and always allow for manual definition after creating the code attribute. This way, power-users get the freedom they want, while others might still want to visit the inline pseudo-instructions for their convenience. Best regards, Rafael -------------- next part -------------- An HTML attachment was scrubbed... URL: From rafael.wth at gmail.com Wed Nov 13 19:54:36 2024 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Wed, 13 Nov 2024 20:54:36 +0100 Subject: [External] : Re: Mapping byte code offsets in custom code attributes In-Reply-To: References: Message-ID: Hei Adam, as for (1): no I would not hope that would be needed. I just wanted to point out that one could not override the attribute mappers, if one needed to work with labels and write the attribute after the fact. If the label to BCI translation happens in an AttributeMapper, one could neither patch in the BCIs from labels. As for (2), I understand the argument for performance reasons and the possible work around. In this case, I would still prefer that I could translate labels to BCIs after a method, and then at least insert the LineNumberTableAttribute using these label based BCIs. What I still do not understand here is how this allows to abstract over ranges, such as with CharacterRangeTable which contains a start and a stop BCI. The attribute is not inlined twice as I understand it? How can I get from the ending BCI in the attribute to the end of the area that is marked? Best regards, Rafael Am Mi., 13. Nov. 2024 um 10:56 Uhr schrieb Adam Sotona < adam.sotona at oracle.com>: > Hi Rafael, > > Please correct me if I understand it wrong. In your email I see following > requests: > > > > 1. Enable an option to override standard attribute mapper with a > custom. > > > > Overriding of standard attribute mappers is not allowed in the Class-File > API as it would result in nondeterministic behavior. Standard attributes > parsing, transformation and generation is always handled by Class-File API. > > > > 2. Allow out-of-order construction of LineNumberTable attribute from > labels. > > > > LineNumber pseudo-instructions are inlined for performance reason. > Inflation of a label for each line number will multiply number of label > targets and degrade performance of Class-File API and significantly degrade > performance of all flow-graph construction transformations. You can always > create a two-pass transformation if you insist on out-of-order line numbers > insertion. There is nothing preventing you to insert labels to the line > number positions (or replace the line numbers with labels) in a first pass, > transform your own aggregated map and apply the changes (if any) in a > second pass. Code transformations can be chained so you do not lose much on > performance. > > > > Adam > > > > > > *From: *classfile-api-dev on behalf > of Rafael Winterhalter > *Date: *Wednesday, 13 November 2024 at 8:51 > *To: *Chen Liang > *Cc: *classfile-api-dev > *Subject: *Re: [External] : Re: Mapping byte code offsets in custom code > attributes > > Hei Adam, > > as for ASM, I also think that the inline visiting of code type annotations > without a label is a limitation in its API design. It does not apply as > much to the tree API where one can walk up the stack to insert an > annotation in the correct position, but I have seen this being a hassle in > the visitor API, requiring a second pass. Not so much for annotations, but > for try-catch-segments, this is a particular problem and I have gone as far > as writing the exception table attribute by hand, which ASM allows by > implementing a custom Attribute. The class file format is flexible when it > comes to setting attributes independently from the attributed code, and I > therefore believe that this should be the primary abstraction. I have no > problem with the concept of pseudo instructions, but personally, I would > have dropped them and asked users to request if label positions represent > something that is not related to other instructions. Most people who > instrument code do either not care about attributes at all, or they do in > what case they exactly know what they want to do, and what attribute to > access for such metadata. > > As you say, everything can be worked around, but this is my belief based > on my experience with using ASM. This is why I had hoped that all > attributes could still be label-centric and always allow for manual > definition after creating the code attribute. This way, power-users get the > freedom they want, while others might still want to visit the inline > pseudo-instructions for their convenience. > > Best regards, Rafael > -------------- next part -------------- An HTML attachment was scrubbed... URL: From markro at cs.washington.edu Fri Nov 15 19:42:11 2024 From: markro at cs.washington.edu (Mark Roberts) Date: Fri, 15 Nov 2024 11:42:11 -0800 Subject: class file transformation questions Message-ID: <1449936c69c967ca47ea764be86d45fc@mail.gmail.com> Several of our tools use the java.lang.instrument package to instrument class files when they are loaded. We currently use BCEL to do the instrumentation, but would like to move to the new java.lang.classfile package. I have gotten some basic transforms working in this environment and I see how to modify existing instructions. However, we need to perform some larger modifications to the classes we instrument. There are three basic transforms we need to perform (all on the same class file): 1. Injecting code into an existing method 2. Adding new methods 3. Duplicating an existing method with some changes Any suggestions as to how to accomplish these tasks would be much appreciated. Thank you, Mark Roberts -------------- next part -------------- An HTML attachment was scrubbed... URL: From chen.l.liang at oracle.com Sat Nov 16 00:47:50 2024 From: chen.l.liang at oracle.com (Chen Liang) Date: Sat, 16 Nov 2024 00:47:50 +0000 Subject: class file transformation questions In-Reply-To: <1449936c69c967ca47ea764be86d45fc@mail.gmail.com> References: <1449936c69c967ca47ea764be86d45fc@mail.gmail.com> Message-ID: Hi Mark, 1. For injecting code, you can override the atStart of the CodeTransform interface. Unfortunately we don't have a utility method for you to pass a lambda ? but we can always add it later. If you wish to find an injection point, you can get the full list of code elements, and find your insertion point and add the desired instruction objects. 2. For adding new methods, you can also override the atStart/atEnd method of your ClassTransform, or do andThen(ClassTransform.endHandler(clb -> clb.withMethod(...))) to add methods. 3. To duplicate an existing method object, you can first call withMethod to create a new method, and once you have the method builder, it has a transform(otherMethod, transform) that can pipe otherMethod's contents through the transform, and finally everything goes to the method builder. Feel free to ask more or suggest. Unfortunately I am still trying to improve documentations for ClassFile API, so at this stage the documents may still be insufficient, and many of the useful methods are not covered by the JEP. Regards, Chen Liang ________________________________ From: classfile-api-dev on behalf of Mark Roberts Sent: Friday, November 15, 2024 1:42 PM To: classfile-api-dev at openjdk.org Subject: class file transformation questions Several of our tools use the java.lang.instrument package to instrument class files when they are loaded. We currently use BCEL to do the instrumentation, but would like to move to the new java.lang.classfile package. I have gotten some basic transforms working in this environment and I see how to modify existing instructions. However, we need to perform some larger modifications to the classes we instrument. There are three basic transforms we need to perform (all on the same class file): 1. Injecting code into an existing method 2. Adding new methods 3. Duplicating an existing method with some changes Any suggestions as to how to accomplish these tasks would be much appreciated. Thank you, Mark Roberts -------------- next part -------------- An HTML attachment was scrubbed... URL: From chen.l.liang at oracle.com Mon Nov 18 00:58:50 2024 From: chen.l.liang at oracle.com (Chen Liang) Date: Mon, 18 Nov 2024 00:58:50 +0000 Subject: [External] : RE: class file transformation questions In-Reply-To: References: <1449936c69c967ca47ea764be86d45fc@mail.gmail.com> Message-ID: Hi Mark, For performing a lot of modifications, you first can create a private static void check(CodeBuilder builder, CodeElement element) { // check the elements, and if it matches a pattern, call another method // as long as you pass the CodeBuilder and make changes to it, your changes will be saved } And then refer to this as a CodeTransform in a method reference. Or, if you want to do a huge overhaul to a CodeModel, you can get a MethodTransform, and have a specific method like: if (me instanceof CodeModel code) { methodBuilder.withCode(cob -> rebuildCode(cob, code)); } private static void rebuildCode(CodeBuilder builder, CodeModel model) {} This gives you a more holistic view of the code, and should be more friendly for massive transformations that almost rebuild the method code. (This one also allows you to pull instructions to a list with new ArrayList<>(model.elementList()), modify that list, and do list.forEach(builder) to send the results, which is used in some older patterns) Regards, Chen Liang ________________________________ From: Mark Roberts Sent: Sunday, November 17, 2024 6:30 PM To: Chen Liang Subject: [External] : RE: class file transformation questions Thank you for your help, it is much appreciated. I have what I guess you would call a coding style question. If you wish to perform a lot of modifications to a class file I could envision a method with hundreds of lines of code. For a smaller example, see the chaining multiple transformations example in java/lang/classfile/ClassFileTransform.html. I find this very (extremely?) difficult to read. Is there any way of writing this code in a more ?traditional? way with several, smaller method bodies? Or is there no way to use java.lang.classfile without chaining together large numbers of builders written as lambda functions? Thank you, Mark From: Chen Liang > Sent: Friday, November 15, 2024 4:48 PM To: Mark Roberts >; classfile-api-dev at openjdk.org Subject: Re: class file transformation questions Hi Mark, 1. For injecting code, you can override the atStart of the CodeTransform interface. Unfortunately we don't have a utility method for you to pass a lambda ? but we can always add it later. If you wish to find an injection point, you can get the full list of code elements, and find your insertion point and add the desired instruction objects. 1. For adding new methods, you can also override the atStart/atEnd method of your ClassTransform, or do andThen(ClassTransform.endHandler(clb -> clb.withMethod(...))) to add methods. 1. To duplicate an existing method object, you can first call withMethod to create a new method, and once you have the method builder, it has a transform(otherMethod, transform) that can pipe otherMethod's contents through the transform, and finally everything goes to the method builder. Feel free to ask more or suggest. Unfortunately I am still trying to improve documentations for ClassFile API, so at this stage the documents may still be insufficient, and many of the useful methods are not covered by the JEP. Regards, Chen Liang ________________________________ From: classfile-api-dev > on behalf of Mark Roberts > Sent: Friday, November 15, 2024 1:42 PM To: classfile-api-dev at openjdk.org > Subject: class file transformation questions Several of our tools use the java.lang.instrument package to instrument class files when they are loaded. We currently use BCEL to do the instrumentation, but would like to move to the new java.lang.classfile package. I have gotten some basic transforms working in this environment and I see how to modify existing instructions. However, we need to perform some larger modifications to the classes we instrument. There are three basic transforms we need to perform (all on the same class file): 1. Injecting code into an existing method 2. Adding new methods 3. Duplicating an existing method with some changes Any suggestions as to how to accomplish these tasks would be much appreciated. Thank you, Mark Roberts -------------- next part -------------- An HTML attachment was scrubbed... URL: From chen.l.liang at oracle.com Tue Nov 19 23:24:50 2024 From: chen.l.liang at oracle.com (Chen Liang) Date: Tue, 19 Nov 2024 23:24:50 +0000 Subject: [External] : more class file transformation questions In-Reply-To: References: Message-ID: Hi Mark, ClassFileBuilder::with takes an element object, such as a FieldModel for a ClassBuilder. It does not modify the object when applied to the new builder, except implementation details like constant pool references.. ClassFileBuilder::transform takes the same type of the object as the object to build, such as a ClassModel for a ClassBuilder. In the same time, it allows you to process all element objects of that same type of object. For example, a ClassBuilder can with a method or a field. A ClassBuilder can transform a class, and use this to merge classes, like clb.transform(sourceClassA, ClassTransform.ACCEPT_ALL).transform(sourceClassB, ClassTransform.ACCEPT_ALL) to merge source classes A and B into the ongoing class builder. For your specific use case, I think what you need is a method transform that does something like: (mb, me) -> { if (mb instanceof CodeModel code) { List codeElements = ...// prepare your list of instructions mb.withCode(codeElements::forEach); // the list of instructions get sent to the code builder } else { mb.with(me); } } It is also possible to do this in CodeTransform by collecting everything in a list in accept(CodeBuilder, CodeElement) and only working on the builder in atEnd(CodeBuilder), when you can see all the elements. But that code will be a bit not straightforward compared to this pattern I am showing off, and this CodeTransform can only see all elements that is from one original model instead of those from all original models (applying the merge class A and B example). P.S. When you reply, remember to choose "reply all" so that your reply gets sent to the classfile-api-dev list too. Regards, Chen Liang ________________________________ From: Mark Roberts Sent: Tuesday, November 19, 2024 4:43 PM To: Chen Liang Subject: [External] : more class file transformation questions Thank you, those suggestions really helped! Next question, given a builder, I?m curious about the differences between the ?with? versus ?transform? methods. I see how the transforms can simplify coding and they support composition. The implication seems to be that with and build create new objects from old while transform modifies existing objects. But based on the examples I?ve seen they look very similar. Are there significant performance differences? Or other advantages? My use case, adding instrumentation code to class files, requires inspecting every byte code and possibly modifying a lot of them. I worry that I would have to composite many transforms where one linear walk through the byte codes would be sufficient. Am I missing something? (it wouldn?t surprise me if I was) Thank you, Mark From: Chen Liang > Sent: Sunday, November 17, 2024 4:59 PM To: Mark Roberts >; classfile-api-dev at openjdk.org Subject: Re: [External] : RE: class file transformation questions Hi Mark, For performing a lot of modifications, you first can create a private static void check(CodeBuilder builder, CodeElement element) { // check the elements, and if it matches a pattern, call another method // as long as you pass the CodeBuilder and make changes to it, your changes will be saved } And then refer to this as a CodeTransform in a method reference. Or, if you want to do a huge overhaul to a CodeModel, you can get a MethodTransform, and have a specific method like: if (me instanceof CodeModel code) { methodBuilder.withCode(cob -> rebuildCode(cob, code)); } private static void rebuildCode(CodeBuilder builder, CodeModel model) {} This gives you a more holistic view of the code, and should be more friendly for massive transformations that almost rebuild the method code. (This one also allows you to pull instructions to a list with new ArrayList<>(model.elementList()), modify that list, and do list.forEach(builder) to send the results, which is used in some older patterns) Regards, Chen Liang ________________________________ From: Mark Roberts > Sent: Sunday, November 17, 2024 6:30 PM To: Chen Liang > Subject: [External] : RE: class file transformation questions Thank you for your help, it is much appreciated. I have what I guess you would call a coding style question. If you wish to perform a lot of modifications to a class file I could envision a method with hundreds of lines of code. For a smaller example, see the chaining multiple transformations example in java/lang/classfile/ClassFileTransform.html. I find this very (extremely?) difficult to read. Is there any way of writing this code in a more ?traditional? way with several, smaller method bodies? Or is there no way to use java.lang.classfile without chaining together large numbers of builders written as lambda functions? Thank you, Mark From: Chen Liang > Sent: Friday, November 15, 2024 4:48 PM To: Mark Roberts >; classfile-api-dev at openjdk.org Subject: Re: class file transformation questions Hi Mark, 1. For injecting code, you can override the atStart of the CodeTransform interface. Unfortunately we don't have a utility method for you to pass a lambda ? but we can always add it later. If you wish to find an injection point, you can get the full list of code elements, and find your insertion point and add the desired instruction objects. 1. For adding new methods, you can also override the atStart/atEnd method of your ClassTransform, or do andThen(ClassTransform.endHandler(clb -> clb.withMethod(...))) to add methods. 1. To duplicate an existing method object, you can first call withMethod to create a new method, and once you have the method builder, it has a transform(otherMethod, transform) that can pipe otherMethod's contents through the transform, and finally everything goes to the method builder. Feel free to ask more or suggest. Unfortunately I am still trying to improve documentations for ClassFile API, so at this stage the documents may still be insufficient, and many of the useful methods are not covered by the JEP. Regards, Chen Liang ________________________________ From: classfile-api-dev > on behalf of Mark Roberts > Sent: Friday, November 15, 2024 1:42 PM To: classfile-api-dev at openjdk.org > Subject: class file transformation questions Several of our tools use the java.lang.instrument package to instrument class files when they are loaded. We currently use BCEL to do the instrumentation, but would like to move to the new java.lang.classfile package. I have gotten some basic transforms working in this environment and I see how to modify existing instructions. However, we need to perform some larger modifications to the classes we instrument. There are three basic transforms we need to perform (all on the same class file): 1. Injecting code into an existing method 2. Adding new methods 3. Duplicating an existing method with some changes Any suggestions as to how to accomplish these tasks would be much appreciated. Thank you, Mark Roberts -------------- next part -------------- An HTML attachment was scrubbed... URL: From markro at cs.washington.edu Fri Nov 22 00:12:17 2024 From: markro at cs.washington.edu (Mark Roberts) Date: Thu, 21 Nov 2024 16:12:17 -0800 Subject: java.lang.classfile model enumeration order Message-ID: <37fcc186afb12accc7750575fb4a9fa4@mail.gmail.com> Are there any rules/guarantees that the order of items in ClassModel::methods() will match the order in ?for (ClassElement ce : cm)? for the ce elements that are MethodModel? And also for similar items down the class hierarchy? Thank you, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Fri Nov 22 10:42:52 2024 From: adam.sotona at oracle.com (Adam Sotona) Date: Fri, 22 Nov 2024 10:42:52 +0000 Subject: java.lang.classfile model enumeration order In-Reply-To: <37fcc186afb12accc7750575fb4a9fa4@mail.gmail.com> References: <37fcc186afb12accc7750575fb4a9fa4@mail.gmail.com> Message-ID: Hi Mark, Fields and methods are included in the ClassElement stream or iterator in the same order, and this order corresponds to their order in the class file. This principle is common for the whole API, except for some PseudoInstructions identifying their start and stop positions in the stream by Labels. These PseudoInstructions are always streamed before respective LabelTargets. Adam From: classfile-api-dev on behalf of Mark Roberts Date: Friday, 22 November 2024 at 1:12 To: Chen Liang , classfile-api-dev at openjdk.org Subject: java.lang.classfile model enumeration order Are there any rules/guarantees that the order of items in ClassModel::methods() will match the order in ?for (ClassElement ce : cm)? for the ce elements that are MethodModel? And also for similar items down the class hierarchy? Thank you, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Fri Nov 22 15:32:07 2024 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 22 Nov 2024 10:32:07 -0500 Subject: java.lang.classfile model enumeration order In-Reply-To: <37fcc186afb12accc7750575fb4a9fa4@mail.gmail.com> References: <37fcc186afb12accc7750575fb4a9fa4@mail.gmail.com> Message-ID: <356dc25d-88aa-4fd3-a412-3d869b049bc4@oracle.com> Only for items whose order is significant to the classfile semantics.? So instructions come in the order they appear, but methods, fields, attributes, etc, could come in any order.? Pattern matching is your friend here. On 11/21/2024 7:12 PM, Mark Roberts wrote: > > Are there any rules/guarantees that the order of items in > ClassModel::methods() will match the order in ?for (ClassElement ce : > cm)? ?for the ce elements that are MethodModel?? And also for similar > items down the class hierarchy? > > Thank you, Mark > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Fri Nov 22 16:02:21 2024 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 22 Nov 2024 11:02:21 -0500 Subject: java.lang.classfile model enumeration order In-Reply-To: References: <37fcc186afb12accc7750575fb4a9fa4@mail.gmail.com> Message-ID: <3183a847-edfc-4bd1-b9ed-0ea691618279@oracle.com> This is a property of the implementation, but not something that the spec guarantees, though.? Also, in the middle of a transformation pipeline, there is no longer a "classfile order" to fall back on.? So it is best to not assume anything about ordering, unless the ordering is part of the semantics (like the bytecodes in a method body.) On 11/22/2024 5:42 AM, Adam Sotona wrote: > > Hi Mark, > > Fields and methods are included in the ClassElement stream or iterator > in the same order, and this order corresponds to their order in the > class file. > > This principle is common for the whole API, except for some > PseudoInstructions identifying their start and stop positions ?in the > stream by Labels. These PseudoInstructions are always streamed before > respective LabelTargets. > > Adam > > *From: *classfile-api-dev on > behalf of Mark Roberts > *Date: *Friday, 22 November 2024 at 1:12 > *To: *Chen Liang , > classfile-api-dev at openjdk.org > *Subject: *java.lang.classfile model enumeration order > > Are there any rules/guarantees that the order of items in > ClassModel::methods() will match the order in ?for (ClassElement ce : > cm)? ?for the ce elements that are MethodModel?? And also for similar > items down the class hierarchy? > > Thank you, Mark > -------------- next part -------------- An HTML attachment was scrubbed... URL: From markro at cs.washington.edu Fri Nov 22 18:07:24 2024 From: markro at cs.washington.edu (Mark Roberts) Date: Fri, 22 Nov 2024 10:07:24 -0800 Subject: java.lang.classfile model enumeration order In-Reply-To: <3183a847-edfc-4bd1-b9ed-0ea691618279@oracle.com> References: <37fcc186afb12accc7750575fb4a9fa4@mail.gmail.com> <3183a847-edfc-4bd1-b9ed-0ea691618279@oracle.com> Message-ID: <0a90ced46f14dd3dcb9fb58fcc2dc880@mail.gmail.com> So to summarize: it?s true today, but don?t rely on it. Mark *From:* Brian Goetz *Sent:* Friday, November 22, 2024 8:02 AM *To:* Adam Sotona ; Mark Roberts < markro at cs.washington.edu>; Chen Liang ; classfile-api-dev at openjdk.org *Subject:* Re: java.lang.classfile model enumeration order This is a property of the implementation, but not something that the spec guarantees, though. Also, in the middle of a transformation pipeline, there is no longer a "classfile order" to fall back on. So it is best to not assume anything about ordering, unless the ordering is part of the semantics (like the bytecodes in a method body.) On 11/22/2024 5:42 AM, Adam Sotona wrote: Hi Mark, Fields and methods are included in the ClassElement stream or iterator in the same order, and this order corresponds to their order in the class file. This principle is common for the whole API, except for some PseudoInstructions identifying their start and stop positions in the stream by Labels. These PseudoInstructions are always streamed before respective LabelTargets. Adam *From: *classfile-api-dev on behalf of Mark Roberts *Date: *Friday, 22 November 2024 at 1:12 *To: *Chen Liang , classfile-api-dev at openjdk.org *Subject: *java.lang.classfile model enumeration order Are there any rules/guarantees that the order of items in ClassModel::methods() will match the order in ?for (ClassElement ce : cm)? for the ce elements that are MethodModel? And also for similar items down the class hierarchy? Thank you, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Fri Nov 22 18:09:56 2024 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 22 Nov 2024 13:09:56 -0500 Subject: java.lang.classfile model enumeration order In-Reply-To: <0a90ced46f14dd3dcb9fb58fcc2dc880@mail.gmail.com> References: <37fcc186afb12accc7750575fb4a9fa4@mail.gmail.com> <3183a847-edfc-4bd1-b9ed-0ea691618279@oracle.com> <0a90ced46f14dd3dcb9fb58fcc2dc880@mail.gmail.com> Message-ID: Correct.? ASM does guarantee an order of events, but our experience is that such constraints do not help the user very much, are error-prone to rely on, and can limit future flexibility for the implementation. On 11/22/2024 1:07 PM, Mark Roberts wrote: > > So to summarize: it?s true today, but don?t rely on it. > > Mark > > *From:*Brian Goetz > *Sent:* Friday, November 22, 2024 8:02 AM > *To:* Adam Sotona ; Mark Roberts > ; Chen Liang ; > classfile-api-dev at openjdk.org > *Subject:* Re: java.lang.classfile model enumeration order > > This is a property of the implementation, but not something that the > spec guarantees, though.? Also, in the middle of a transformation > pipeline, there is no longer a "classfile order" to fall back on.? So > it is best to not assume anything about ordering, unless the ordering > is part of the semantics (like the bytecodes in a method body.) > > > > On 11/22/2024 5:42 AM, Adam Sotona wrote: > > Hi Mark, > > Fields and methods are included in the ClassElement stream or > iterator in the same order, and this order corresponds to their > order in the class file. > > This principle is common for the whole API, except for some > PseudoInstructions identifying their start and stop positions ?in > the stream by Labels. These PseudoInstructions are always streamed > before respective LabelTargets. > > Adam > > *From: *classfile-api-dev > on behalf of Mark > Roberts > *Date: *Friday, 22 November 2024 at 1:12 > *To: *Chen Liang > , classfile-api-dev at openjdk.org > > *Subject: *java.lang.classfile model enumeration order > > Are there any rules/guarantees that the order of items in > ClassModel::methods() will match the order in ?for (ClassElement > ce : cm)? ?for the ce elements that are MethodModel?? And also for > similar items down the class hierarchy? > > Thank you, Mark > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chen.l.liang at oracle.com Fri Nov 22 18:50:17 2024 From: chen.l.liang at oracle.com (Chen Liang) Date: Fri, 22 Nov 2024 18:50:17 +0000 Subject: java.lang.classfile model enumeration order In-Reply-To: References: <37fcc186afb12accc7750575fb4a9fa4@mail.gmail.com> <3183a847-edfc-4bd1-b9ed-0ea691618279@oracle.com> <0a90ced46f14dd3dcb9fb58fcc2dc880@mail.gmail.com> Message-ID: I think the interest in the order is more among the same kind of elements instead of among different kinds of elements. ASM currently allows you to mix-and-match fields and methods in class visiting; it just has weird requirements such as visitSource -> visitModule -> visitNestHost -> visit other attributes. Such requirements are extremely error-prone and make little sense, and do not exist in the ClassFile API. However, there might be value for the order for the same kind of element in class file streaming. For example, if you stream f1, m1, m2, f2, you might want f1 before f2 and m1 before m2. This order is currently a complementary gift from the implementation, but nothing guarantees that stay the way in the long run. The order of fields and methods are not important in class file interpretation, after all. Regards, Chen Liang ________________________________ From: Brian Goetz Sent: Friday, November 22, 2024 12:09 PM To: Mark Roberts ; Adam Sotona ; Chen Liang ; classfile-api-dev at openjdk.org Subject: Re: java.lang.classfile model enumeration order Correct. ASM does guarantee an order of events, but our experience is that such constraints do not help the user very much, are error-prone to rely on, and can limit future flexibility for the implementation. On 11/22/2024 1:07 PM, Mark Roberts wrote: So to summarize: it?s true today, but don?t rely on it. Mark From: Brian Goetz > Sent: Friday, November 22, 2024 8:02 AM To: Adam Sotona >; Mark Roberts >; Chen Liang >; classfile-api-dev at openjdk.org Subject: Re: java.lang.classfile model enumeration order This is a property of the implementation, but not something that the spec guarantees, though. Also, in the middle of a transformation pipeline, there is no longer a "classfile order" to fall back on. So it is best to not assume anything about ordering, unless the ordering is part of the semantics (like the bytecodes in a method body.) On 11/22/2024 5:42 AM, Adam Sotona wrote: Hi Mark, Fields and methods are included in the ClassElement stream or iterator in the same order, and this order corresponds to their order in the class file. This principle is common for the whole API, except for some PseudoInstructions identifying their start and stop positions in the stream by Labels. These PseudoInstructions are always streamed before respective LabelTargets. Adam From: classfile-api-dev on behalf of Mark Roberts Date: Friday, 22 November 2024 at 1:12 To: Chen Liang , classfile-api-dev at openjdk.org Subject: java.lang.classfile model enumeration order Are there any rules/guarantees that the order of items in ClassModel::methods() will match the order in ?for (ClassElement ce : cm)? for the ce elements that are MethodModel? And also for similar items down the class hierarchy? Thank you, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Fri Nov 22 18:57:54 2024 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 22 Nov 2024 13:57:54 -0500 Subject: java.lang.classfile model enumeration order In-Reply-To: References: <37fcc186afb12accc7750575fb4a9fa4@mail.gmail.com> <3183a847-edfc-4bd1-b9ed-0ea691618279@oracle.com> <0a90ced46f14dd3dcb9fb58fcc2dc880@mail.gmail.com> Message-ID: So, it sounds easy to do that when you are reading from the classfile, since all the field_info structures are colocated.? But as soon as you start transforming, you may inject methods between the fields, or vice versa.? So in practice the utility of such guarantees is low, because it will come with caveats like "as long as you are the first guy in the transform pipeline."? So better to just tell people "you'll get all the elements, but with no ordering guarantee" -- they'll write better code as a result. On 11/22/2024 1:50 PM, Chen Liang wrote: > I think the interest in the order is more among the same kind of > elements instead of among different kinds of elements. > > ASM currently allows you to mix-and-match fields and methods in class > visiting; it just has weird requirements such as visitSource -> > visitModule -> visitNestHost -> visit other attributes.? Such > requirements are extremely error-prone and make little sense, and do > not exist in the ClassFile API. > > However, there might be value for the order for the same kind of > element in class file streaming.? For example, if you stream f1, m1, > m2, f2, you might want f1 before f2 and m1 before m2.? This order is > currently a complementary gift from the implementation, but nothing > guarantees that stay the way in the long run. The order of fields and > methods are not important in class file interpretation, after all. > > Regards, > Chen Liang > ------------------------------------------------------------------------ > *From:* Brian Goetz > *Sent:* Friday, November 22, 2024 12:09 PM > *To:* Mark Roberts ; Adam Sotona > ; Chen Liang ; > classfile-api-dev at openjdk.org > *Subject:* Re: java.lang.classfile model enumeration order > Correct.? ASM does guarantee an order of events, but our experience is > that such constraints do not help the user very much, are error-prone > to rely on, and can limit future flexibility for the implementation. > > > On 11/22/2024 1:07 PM, Mark Roberts wrote: >> >> So to summarize: it?s true today, but don?t rely on it. >> >> Mark >> >> *From:*Brian Goetz > > >> *Sent:* Friday, November 22, 2024 8:02 AM >> *To:* Adam Sotona > >; Mark Roberts >> >; Chen >> Liang >; >> classfile-api-dev at openjdk.org >> *Subject:* Re: java.lang.classfile model enumeration order >> >> This is a property of the implementation, but not something that the >> spec guarantees, though.? Also, in the middle of a transformation >> pipeline, there is no longer a "classfile order" to fall back on.? So >> it is best to not assume anything about ordering, unless the ordering >> is part of the semantics (like the bytecodes in a method body.) >> >> >> >> On 11/22/2024 5:42 AM, Adam Sotona wrote: >> >> Hi Mark, >> >> Fields and methods are included in the ClassElement stream or >> iterator in the same order, and this order corresponds to their >> order in the class file. >> >> This principle is common for the whole API, except for some >> PseudoInstructions identifying their start and stop positions ?in >> the stream by Labels. These PseudoInstructions are always >> streamed before respective LabelTargets. >> >> Adam >> >> *From: *classfile-api-dev >> on behalf of Mark >> Roberts >> *Date: *Friday, 22 November 2024 at 1:12 >> *To: *Chen Liang >> , classfile-api-dev at openjdk.org >> >> >> >> *Subject: *java.lang.classfile model enumeration order >> >> Are there any rules/guarantees that the order of items in >> ClassModel::methods() will match the order in ?for (ClassElement >> ce : cm)? ?for the ce elements that are MethodModel?? And also >> for similar items down the class hierarchy? >> >> Thank you, Mark >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Fri Nov 22 19:20:58 2024 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 22 Nov 2024 20:20:58 +0100 (CET) Subject: java.lang.classfile model enumeration order In-Reply-To: References: <37fcc186afb12accc7750575fb4a9fa4@mail.gmail.com> <3183a847-edfc-4bd1-b9ed-0ea691618279@oracle.com> <0a90ced46f14dd3dcb9fb58fcc2dc880@mail.gmail.com> Message-ID: <840299693.1454655.1732303258598.JavaMail.zimbra@univ-eiffel.fr> Hello, ASM defines an order when reading/streaming, there is no order when writing to a ClassWriter. So the constraint is if you create an intermediary ClassVisitor, you need to keep the order. see section 2.2.6 of the ASM guide (https://asm.ow2.io/asm4-guide.pdf) regards, R?mi > From: "Chen Liang" > To: "Mark Roberts" , "Adam Sotona" > , "classfile-api-dev" > Sent: Friday, November 22, 2024 7:50:17 PM > Subject: Re: java.lang.classfile model enumeration order > I think the interest in the order is more among the same kind of elements > instead of among different kinds of elements. > ASM currently allows you to mix-and-match fields and methods in class visiting; > it just has weird requirements such as visitSource -> visitModule -> > visitNestHost -> visit other attributes. Such requirements are extremely > error-prone and make little sense, and do not exist in the ClassFile API. > However, there might be value for the order for the same kind of element in > class file streaming. For example, if you stream f1, m1, m2, f2, you might want > f1 before f2 and m1 before m2. This order is currently a complementary gift > from the implementation, but nothing guarantees that stay the way in the long > run. The order of fields and methods are not important in class file > interpretation, after all. > Regards, > Chen Liang > From: Brian Goetz > Sent: Friday, November 22, 2024 12:09 PM > To: Mark Roberts ; Adam Sotona > ; Chen Liang ; > classfile-api-dev at openjdk.org > Subject: Re: java.lang.classfile model enumeration order > Correct. ASM does guarantee an order of events, but our experience is that such > constraints do not help the user very much, are error-prone to rely on, and can > limit future flexibility for the implementation. > On 11/22/2024 1:07 PM, Mark Roberts wrote: >> So to summarize: it?s true today, but don?t rely on it. >> Mark >> From: Brian Goetz < [ mailto:brian.goetz at oracle.com | brian.goetz at oracle.com ] > >> Sent: Friday, November 22, 2024 8:02 AM >> To: Adam Sotona < [ mailto:adam.sotona at oracle.com | adam.sotona at oracle.com ] >; >> Mark Roberts < [ mailto:markro at cs.washington.edu | markro at cs.washington.edu ] >> >; Chen Liang < [ mailto:chen.l.liang at oracle.com | chen.l.liang at oracle.com ] >; >> [ mailto:classfile-api-dev at openjdk.org | classfile-api-dev at openjdk.org ] >> Subject: Re: java.lang.classfile model enumeration order >> This is a property of the implementation, but not something that the spec >> guarantees, though. Also, in the middle of a transformation pipeline, there is >> no longer a "classfile order" to fall back on. So it is best to not assume >> anything about ordering, unless the ordering is part of the semantics (like the >> bytecodes in a method body.) >> On 11/22/2024 5:42 AM, Adam Sotona wrote: >>> Hi Mark, >>> Fields and methods are included in the ClassElement stream or iterator in the >>> same order, and this order corresponds to their order in the class file. >>> This principle is common for the whole API, except for some PseudoInstructions >>> identifying their start and stop positions in the stream by Labels. These >>> PseudoInstructions are always streamed before respective LabelTargets. >>> Adam >>> From: classfile-api-dev [ mailto:classfile-api-dev-retn at openjdk.org | >>> ] on behalf of Mark Roberts [ >>> mailto:markro at cs.washington.edu | ] >>> Date: Friday, 22 November 2024 at 1:12 >>> To: Chen Liang [ mailto:chen.l.liang at oracle.com | ] , >>> [ mailto:classfile-api-dev at openjdk.org | classfile-api-dev at openjdk.org ] [ >>> mailto:classfile-api-dev at openjdk.org | ] >>> Subject: java.lang.classfile model enumeration order >>> Are there any rules/guarantees that the order of items in ClassModel::methods() >>> will match the order in ?for (ClassElement ce : cm)? for the ce elements that >>> are MethodModel? And also for similar items down the class hierarchy? >>> Thank you, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From david32768 at btinternet.com Sat Nov 23 15:11:02 2024 From: david32768 at btinternet.com (david32768@btinternet.com david32768@btinternet.com) Date: Sat, 23 Nov 2024 15:11:02 +0000 (GMT) Subject: TableSwitch checks Message-ID: The following are bugs or suggested improvements to the checking of TableSwitch instructions: ? cases can be empty (if all targets are the default label) which results in low > high. ``` ***** SWITCH24-25\CodeBuilder.java 2428: ? ? ? ? ?} 2429: ? ? ? ? ?return tableswitch(low, high, defaultTarget, cases); ***** SWITCH_MODS\CODEBUILDER.JAVA 2428: ? ? ? ? ?} 2429: ? ? ? ? ?if (cases.isEmpty()) { 2430: ? ? ? ? ? ? ?low = high = 0; 2431: ? ? ? ? ?} 2432: ? ? ? ? ?return tableswitch(low, high, defaultTarget, cases); ***** ``` low = Integer.MIN_VALUE, high = Integer.MAX_VALUE is too large a range. Integer.MAX_VALUE - Integer.MIN_VALUE + 1 == 0 ``` ***** SWITCH24-25\StackCounter.java ?278: ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?keys = high - low + 1; ?279: ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?if (keys < 0) { ?280: ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?throw error("too many keys in tableswitch"); ***** SWITCH_MODS\STACKCOUNTER.JAVA ?278: ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?keys = high - low + 1; ?279: ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?if (keys <= 0) { ?280: ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?throw error("too many keys in tableswitch"); ***** ``` do same checks on UnboundInstruction as BoundInstruction? ``` ***** SWITCH24-25\AbstractInstruction.java ?935: ? ? ? ? ? ? ?this.highValue = highValue; ?936: ? ? ? ? ? ? ?this.defaultTarget = requireNonNull(defaultTarget); ***** SWITCH_MODS\ABSTRACTINSTRUCTION.JAVA ?935: ? ? ? ? ? ? ?this.highValue = highValue; ?936: ? ? ? ? ? ? ?if (highValue < lowValue || (long)highValue - lowValue > (65535 - 4) >> 2 { ?937: ? ? ? ? ? ? ? ? ?throw new IllegalArgumentException("Invalid tableswitch values low: " + low + " high: " + high); ?938: ? ? ? ? ? ? ?} ?939: ? ? ? ? ? ? ?this.defaultTarget = requireNonNull(defaultTarget); ***** ``` check cases in range [low,high] and no duplicates (which can occur via UnboundInstruction) ``` ***** SWITCH24-25\DirectCodeBuilder.java ?641: ? ? ? ? ?for (var c : cases) { ?642: ? ? ? ? ? ? ?caseMap.put(c.caseValue(), c.target()); ?643: ? ? ? ? ?} ***** SWITCH_MODS\DIRECTCODEBUILDER.JAVA ?641: ? ? ? ? ?for (var c : cases) { ?642: ? ? ? ? ? ? ?int cv = c.caseValue(); ?643: ? ? ? ? ? ? ?if (cv < low || cv > high) { ?644: ? ? ? ? ? ? ? ? ?throw new IllegalArgumentException(String.format("Case value %d is not in [low=%d,high=%d]", cv, low,high)); ?645: ? ? ? ? ? ? ?} ?646: ? ? ? ? ? ? ?var previous = caseMap.put(cv, c.target()); ?647: ? ? ? ? ? ? ?if (previous != null) { ?648: ? ? ? ? ? ? ? ? ?throw new IllegalArgumentException(String.format("Duplicate case value %d", cv); ?649: ? ? ? ? ? ? ?} ?650: ? ? ? ? ?} ***** ``` ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From markro at cs.washington.edu Mon Nov 25 22:17:07 2024 From: markro at cs.washington.edu (Mark Roberts) Date: Mon, 25 Nov 2024 14:17:07 -0800 Subject: how does one get value of ConstantValueAttribute? Message-ID: <62769e65088c455d1986c6f1d8d6d006@mail.gmail.com> ConstantValueAttribute:const() is not public. The documentation implies it is. How do you get the value? Thanks, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From chen.l.liang at oracle.com Mon Nov 25 22:34:45 2024 From: chen.l.liang at oracle.com (Chen Liang) Date: Mon, 25 Nov 2024 22:34:45 +0000 Subject: [External] : how does one get value of ConstantValueAttribute? In-Reply-To: <62769e65088c455d1986c6f1d8d6d006@mail.gmail.com> References: <62769e65088c455d1986c6f1d8d6d006@mail.gmail.com> Message-ID: Hi Mark, I don't understand your question. Nowhere in any document mentions there's a "const" in ConstantValueAttribute. Its Javadoc is available here: https://download.java.net/java/early_access/jdk24/docs/api/java.base/java/lang/classfile/attribute/ConstantValueAttribute.html So once you get a ConstantValueAttribute from a field, you can check its constant() accessor, and pattern match to check if it is an IntegerEntry/StringEntry etc. Regards, Chen Liang ________________________________ From: Mark Roberts Sent: Monday, November 25, 2024 4:17 PM To: Chen Liang ; classfile-api-dev at openjdk.org Subject: [External] : how does one get value of ConstantValueAttribute? ConstantValueAttribute:const() is not public. The documentation implies it is. How do you get the value? Thanks, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From markro at cs.washington.edu Mon Nov 25 22:44:43 2024 From: markro at cs.washington.edu (Mark Roberts) Date: Mon, 25 Nov 2024 14:44:43 -0800 Subject: [External] : how does one get value of ConstantValueAttribute? In-Reply-To: References: <62769e65088c455d1986c6f1d8d6d006@mail.gmail.com> Message-ID: Sorry ? typo: ?constant()? *From:* Chen Liang *Sent:* Monday, November 25, 2024 2:35 PM *To:* Mark Roberts ; classfile-api-dev at openjdk.org *Subject:* Re: [External] : how does one get value of ConstantValueAttribute? Hi Mark, I don't understand your question. Nowhere in any document mentions there's a "const" in ConstantValueAttribute. Its Javadoc is available here: https://download.java.net/java/early_access/jdk24/docs/api/java.base/java/lang/classfile/attribute/ConstantValueAttribute.html So once you get a ConstantValueAttribute from a field, you can check its constant() accessor, and pattern match to check if it is an IntegerEntry/StringEntry etc. Regards, Chen Liang ------------------------------ *From:* Mark Roberts *Sent:* Monday, November 25, 2024 4:17 PM *To:* Chen Liang ; classfile-api-dev at openjdk.org < classfile-api-dev at openjdk.org> *Subject:* [External] : how does one get value of ConstantValueAttribute? ConstantValueAttribute:const() is not public. The documentation implies it is. How do you get the value? Thanks, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From chen.l.liang at oracle.com Mon Nov 25 22:49:01 2024 From: chen.l.liang at oracle.com (Chen Liang) Date: Mon, 25 Nov 2024 22:49:01 +0000 Subject: [External] : how does one get value of ConstantValueAttribute? In-Reply-To: References: <62769e65088c455d1986c6f1d8d6d006@mail.gmail.com> Message-ID: This is an interface method, which is implicitly public. You should be able to call it without any problem. If there is a compile error, can you share the snippet of code that you managed to reproduce an error? ________________________________ From: Mark Roberts Sent: Monday, November 25, 2024 4:44 PM To: Chen Liang ; classfile-api-dev at openjdk.org Subject: RE: [External] : how does one get value of ConstantValueAttribute? Sorry ? typo: ?constant()? From: Chen Liang > Sent: Monday, November 25, 2024 2:35 PM To: Mark Roberts >; classfile-api-dev at openjdk.org Subject: Re: [External] : how does one get value of ConstantValueAttribute? Hi Mark, I don't understand your question. Nowhere in any document mentions there's a "const" in ConstantValueAttribute. Its Javadoc is available here: https://download.java.net/java/early_access/jdk24/docs/api/java.base/java/lang/classfile/attribute/ConstantValueAttribute.html So once you get a ConstantValueAttribute from a field, you can check its constant() accessor, and pattern match to check if it is an IntegerEntry/StringEntry etc. Regards, Chen Liang ________________________________ From: Mark Roberts > Sent: Monday, November 25, 2024 4:17 PM To: Chen Liang >; classfile-api-dev at openjdk.org > Subject: [External] : how does one get value of ConstantValueAttribute? ConstantValueAttribute:const() is not public. The documentation implies it is. How do you get the value? Thanks, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From markro at cs.washington.edu Mon Nov 25 23:06:53 2024 From: markro at cs.washington.edu (Mark Roberts) Date: Mon, 25 Nov 2024 15:06:53 -0800 Subject: [External] : how does one get value of ConstantValueAttribute? In-Reply-To: References: <62769e65088c455d1986c6f1d8d6d006@mail.gmail.com> Message-ID: <22843cfa37fc77a6d4f129b5dc105501@mail.gmail.com> Here is snippet from my source: import java.lang.classfile.*; import java.lang.classfile.attribute.*; import java.lang.classfile.attribute.ConstantValueAttribute; import java.lang.classfile.Attributes; import java.lang.classfile.components.ClassPrinter; import java.lang.classfile.constantpool.*; import java.lang.classfile.instruction.*; import java.lang.constant.*; import java.lang.invoke.MethodHandles; import static java.lang.classfile.constantpool.PoolEntry.*; import static java.lang.constant.ConstantDescs.*; . . . // get constant static fields List fields = classModel.fields(); for (FieldModel fm : fields) { Optional cva = fm.findAttribute(Attributes.constantValue()); if (cva.isPresent()) { Utf8Entry utf8 = fm.fieldName(); String value = formatConstantDesc(cva.constant().constantValue()); System.out.printf(" Constant field: %s, value: %s%n", utf8.stringValue(), cva); } } and here is snippet from compilation: ./daikon/chicory/Instrument.java:316: error: cannot find symbol String value = formatConstantDesc(cva.constant().constantValue()); ^ symbol: method constant() location: variable cva of type Optional Let me know if you need more info. Thanks for the assistance. Mark *From:* Chen Liang *Sent:* Monday, November 25, 2024 2:49 PM *To:* Mark Roberts ; classfile-api-dev at openjdk.org *Subject:* Re: [External] : how does one get value of ConstantValueAttribute? This is an interface method, which is implicitly public. You should be able to call it without any problem. If there is a compile error, can you share the snippet of code that you managed to reproduce an error? ------------------------------ *From:* Mark Roberts *Sent:* Monday, November 25, 2024 4:44 PM *To:* Chen Liang ; classfile-api-dev at openjdk.org < classfile-api-dev at openjdk.org> *Subject:* RE: [External] : how does one get value of ConstantValueAttribute? Sorry ? typo: ?constant()? *From:* Chen Liang *Sent:* Monday, November 25, 2024 2:35 PM *To:* Mark Roberts ; classfile-api-dev at openjdk.org *Subject:* Re: [External] : how does one get value of ConstantValueAttribute? Hi Mark, I don't understand your question. Nowhere in any document mentions there's a "const" in ConstantValueAttribute. Its Javadoc is available here: https://download.java.net/java/early_access/jdk24/docs/api/java.base/java/lang/classfile/attribute/ConstantValueAttribute.html So once you get a ConstantValueAttribute from a field, you can check its constant() accessor, and pattern match to check if it is an IntegerEntry/StringEntry etc. Regards, Chen Liang ------------------------------ *From:* Mark Roberts *Sent:* Monday, November 25, 2024 4:17 PM *To:* Chen Liang ; classfile-api-dev at openjdk.org < classfile-api-dev at openjdk.org> *Subject:* [External] : how does one get value of ConstantValueAttribute? ConstantValueAttribute:const() is not public. The documentation implies it is. How do you get the value? Thanks, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From chen.l.liang at oracle.com Mon Nov 25 23:14:00 2024 From: chen.l.liang at oracle.com (Chen Liang) Date: Mon, 25 Nov 2024 23:14:00 +0000 Subject: [External] : how does one get value of ConstantValueAttribute? In-Reply-To: <22843cfa37fc77a6d4f129b5dc105501@mail.gmail.com> References: <62769e65088c455d1986c6f1d8d6d006@mail.gmail.com> <22843cfa37fc77a6d4f129b5dc105501@mail.gmail.com> Message-ID: Hi Mark, You are calling constant() on an Optional. You should call cva.get().constant().constantValue() instead. Chen ________________________________ From: Mark Roberts Sent: Monday, November 25, 2024 5:06 PM To: Chen Liang ; classfile-api-dev at openjdk.org Subject: RE: [External] : how does one get value of ConstantValueAttribute? Here is snippet from my source: import java.lang.classfile.*; import java.lang.classfile.attribute.*; import java.lang.classfile.attribute.ConstantValueAttribute; import java.lang.classfile.Attributes; import java.lang.classfile.components.ClassPrinter; import java.lang.classfile.constantpool.*; import java.lang.classfile.instruction.*; import java.lang.constant.*; import java.lang.invoke.MethodHandles; import static java.lang.classfile.constantpool.PoolEntry.*; import static java.lang.constant.ConstantDescs.*; . . . // get constant static fields List fields = classModel.fields(); for (FieldModel fm : fields) { Optional cva = fm.findAttribute(Attributes.constantValue()); if (cva.isPresent()) { Utf8Entry utf8 = fm.fieldName(); String value = formatConstantDesc(cva.constant().constantValue()); System.out.printf(" Constant field: %s, value: %s%n", utf8.stringValue(), cva); } } and here is snippet from compilation: ./daikon/chicory/Instrument.java:316: error: cannot find symbol String value = formatConstantDesc(cva.constant().constantValue()); ^ symbol: method constant() location: variable cva of type Optional Let me know if you need more info. Thanks for the assistance. Mark From: Chen Liang > Sent: Monday, November 25, 2024 2:49 PM To: Mark Roberts >; classfile-api-dev at openjdk.org Subject: Re: [External] : how does one get value of ConstantValueAttribute? This is an interface method, which is implicitly public. You should be able to call it without any problem. If there is a compile error, can you share the snippet of code that you managed to reproduce an error? ________________________________ From: Mark Roberts > Sent: Monday, November 25, 2024 4:44 PM To: Chen Liang >; classfile-api-dev at openjdk.org > Subject: RE: [External] : how does one get value of ConstantValueAttribute? Sorry ? typo: ?constant()? From: Chen Liang > Sent: Monday, November 25, 2024 2:35 PM To: Mark Roberts >; classfile-api-dev at openjdk.org Subject: Re: [External] : how does one get value of ConstantValueAttribute? Hi Mark, I don't understand your question. Nowhere in any document mentions there's a "const" in ConstantValueAttribute. Its Javadoc is available here: https://download.java.net/java/early_access/jdk24/docs/api/java.base/java/lang/classfile/attribute/ConstantValueAttribute.html So once you get a ConstantValueAttribute from a field, you can check its constant() accessor, and pattern match to check if it is an IntegerEntry/StringEntry etc. Regards, Chen Liang ________________________________ From: Mark Roberts > Sent: Monday, November 25, 2024 4:17 PM To: Chen Liang >; classfile-api-dev at openjdk.org > Subject: [External] : how does one get value of ConstantValueAttribute? ConstantValueAttribute:const() is not public. The documentation implies it is. How do you get the value? Thanks, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From markro at cs.washington.edu Mon Nov 25 23:17:43 2024 From: markro at cs.washington.edu (Mark Roberts) Date: Mon, 25 Nov 2024 15:17:43 -0800 Subject: [External] : how does one get value of ConstantValueAttribute? In-Reply-To: References: <62769e65088c455d1986c6f1d8d6d006@mail.gmail.com> <22843cfa37fc77a6d4f129b5dc105501@mail.gmail.com> Message-ID: <41e80d52ffc21d7a2956ea9bb92540b3@mail.gmail.com> Right! ? I?ve used Optional before and knew that. Sorry for the noise. *From:* Chen Liang *Sent:* Monday, November 25, 2024 3:14 PM *To:* Mark Roberts ; classfile-api-dev at openjdk.org *Subject:* Re: [External] : how does one get value of ConstantValueAttribute? Hi Mark, You are calling constant() on an Optional. You should call cva.get().constant().constantValue() instead. Chen ------------------------------ *From:* Mark Roberts *Sent:* Monday, November 25, 2024 5:06 PM *To:* Chen Liang ; classfile-api-dev at openjdk.org < classfile-api-dev at openjdk.org> *Subject:* RE: [External] : how does one get value of ConstantValueAttribute? Here is snippet from my source: import java.lang.classfile.*; import java.lang.classfile.attribute.*; import java.lang.classfile.attribute.ConstantValueAttribute; import java.lang.classfile.Attributes; import java.lang.classfile.components.ClassPrinter; import java.lang.classfile.constantpool.*; import java.lang.classfile.instruction.*; import java.lang.constant.*; import java.lang.invoke.MethodHandles; import static java.lang.classfile.constantpool.PoolEntry.*; import static java.lang.constant.ConstantDescs.*; . . . // get constant static fields List fields = classModel.fields(); for (FieldModel fm : fields) { Optional cva = fm.findAttribute(Attributes.constantValue()); if (cva.isPresent()) { Utf8Entry utf8 = fm.fieldName(); String value = formatConstantDesc(cva.constant().constantValue()); System.out.printf(" Constant field: %s, value: %s%n", utf8.stringValue(), cva); } } and here is snippet from compilation: ./daikon/chicory/Instrument.java:316: error: cannot find symbol String value = formatConstantDesc(cva.constant().constantValue()); ^ symbol: method constant() location: variable cva of type Optional Let me know if you need more info. Thanks for the assistance. Mark *From:* Chen Liang *Sent:* Monday, November 25, 2024 2:49 PM *To:* Mark Roberts ; classfile-api-dev at openjdk.org *Subject:* Re: [External] : how does one get value of ConstantValueAttribute? This is an interface method, which is implicitly public. You should be able to call it without any problem. If there is a compile error, can you share the snippet of code that you managed to reproduce an error? ------------------------------ *From:* Mark Roberts *Sent:* Monday, November 25, 2024 4:44 PM *To:* Chen Liang ; classfile-api-dev at openjdk.org < classfile-api-dev at openjdk.org> *Subject:* RE: [External] : how does one get value of ConstantValueAttribute? Sorry ? typo: ?constant()? *From:* Chen Liang *Sent:* Monday, November 25, 2024 2:35 PM *To:* Mark Roberts ; classfile-api-dev at openjdk.org *Subject:* Re: [External] : how does one get value of ConstantValueAttribute? Hi Mark, I don't understand your question. Nowhere in any document mentions there's a "const" in ConstantValueAttribute. Its Javadoc is available here: https://download.java.net/java/early_access/jdk24/docs/api/java.base/java/lang/classfile/attribute/ConstantValueAttribute.html So once you get a ConstantValueAttribute from a field, you can check its constant() accessor, and pattern match to check if it is an IntegerEntry/StringEntry etc. Regards, Chen Liang ------------------------------ *From:* Mark Roberts *Sent:* Monday, November 25, 2024 4:17 PM *To:* Chen Liang ; classfile-api-dev at openjdk.org < classfile-api-dev at openjdk.org> *Subject:* [External] : how does one get value of ConstantValueAttribute? ConstantValueAttribute:const() is not public. The documentation implies it is. How do you get the value? Thanks, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From markro at cs.washington.edu Wed Nov 27 23:54:38 2024 From: markro at cs.washington.edu (Mark Roberts) Date: Wed, 27 Nov 2024 15:54:38 -0800 Subject: [External] : more class file transformation questions In-Reply-To: References: Message-ID: <8e1bfb61f3bf19ed2a155fadaa5da0cd@mail.gmail.com> Okay, a List sounds interesting. 1. Would I just use CodeModel::elementList() as my initial list or do I have to make a copy somehow? 2. How would I insert a new instruction into this list? Builders don?t seem to return an item, perhaps I need to use the static instruction of() methods? 3. What is difference between Bound and Unbound instructions? Thank you, Mark *From:* Chen Liang *Sent:* Tuesday, November 19, 2024 3:25 PM *To:* Mark Roberts ; classfile-api-dev at openjdk.org *Subject:* Re: [External] : more class file transformation questions Hi Mark, ClassFileBuilder::with takes an element object, such as a FieldModel for a ClassBuilder. It does not modify the object when applied to the new builder, except implementation details like constant pool references.. ClassFileBuilder::transform takes the same type of the object as the object to build, such as a ClassModel for a ClassBuilder. In the same time, it allows you to process all element objects of that same type of object. For example, a ClassBuilder can with a method or a field. A ClassBuilder can transform a class, and use this to merge classes, like clb.transform(sourceClassA, ClassTransform.ACCEPT_ALL).transform(sourceClassB, ClassTransform.ACCEPT_ALL) to merge source classes A and B into the ongoing class builder. For your specific use case, I think what you need is a method transform that does something like: (mb, me) -> { if (mb instanceof CodeModel code) { List codeElements = ...// prepare your list of instructions mb.withCode(codeElements::forEach); // the list of instructions get sent to the code builderOK } else { mb.with(me); } } It is also possible to do this in CodeTransform by collecting everything in a list in accept(CodeBuilder, CodeElement) and only working on the builder in atEnd(CodeBuilder), when you can see all the elements. But that code will be a bit not straightforward compared to this pattern I am showing off, and this CodeTransform can only see all elements that is from one original model instead of those from all original models (applying the merge class A and B example). P.S. When you reply, remember to choose "reply all" so that your reply gets sent to the classfile-api-dev list too. Regards, Chen Liang ------------------------------ *From:* Mark Roberts *Sent:* Tuesday, November 19, 2024 4:43 PM *To:* Chen Liang *Subject:* [External] : more class file transformation questions Thank you, those suggestions really helped! Next question, given a builder, I?m curious about the differences between the ?with? versus ?transform? methods. I see how the transforms can simplify coding and they support composition. The implication seems to be that with and build create new objects from old while transform modifies existing objects. But based on the examples I?ve seen they look very similar. Are there significant performance differences? Or other advantages? My use case, adding instrumentation code to class files, requires inspecting every byte code and possibly modifying a lot of them. I worry that I would have to composite many transforms where one linear walk through the byte codes would be sufficient. Am I missing something? (it wouldn?t surprise me if I was) Thank you, Mark *From:* Chen Liang *Sent:* Sunday, November 17, 2024 4:59 PM *To:* Mark Roberts ; classfile-api-dev at openjdk.org *Subject:* Re: [External] : RE: class file transformation questions Hi Mark, For performing a lot of modifications, you first can create a private static void check(CodeBuilder builder, CodeElement element) { // check the elements, and if it matches a pattern, call another method // as long as you pass the CodeBuilder and make changes to it, your changes will be saved } And then refer to this as a CodeTransform in a method reference. Or, if you want to do a huge overhaul to a CodeModel, you can get a MethodTransform, and have a specific method like: if (me instanceof CodeModel code) { methodBuilder.withCode(cob -> rebuildCode(cob, code)); } private static void rebuildCode(CodeBuilder builder, CodeModel model) {} This gives you a more holistic view of the code, and should be more friendly for massive transformations that almost rebuild the method code. (This one also allows you to pull instructions to a list with new ArrayList<>(model.elementList()), modify that list, and do list.forEach(builder) to send the results, which is used in some older patterns) Regards, Chen Liang ------------------------------ *From:* Mark Roberts *Sent:* Sunday, November 17, 2024 6:30 PM *To:* Chen Liang *Subject:* [External] : RE: class file transformation questions Thank you for your help, it is much appreciated. I have what I guess you would call a coding style question. If you wish to perform a lot of modifications to a class file I could envision a method with hundreds of lines of code. For a smaller example, see the chaining multiple transformations example in java/lang/classfile/ClassFileTransform.html. I find this very (extremely?) difficult to read. Is there any way of writing this code in a more ?traditional? way with several, smaller method bodies? Or is there no way to use java.lang.classfile without chaining together large numbers of builders written as lambda functions? Thank you, Mark *From:* Chen Liang *Sent:* Friday, November 15, 2024 4:48 PM *To:* Mark Roberts ; classfile-api-dev at openjdk.org *Subject:* Re: class file transformation questions Hi Mark, 1. For injecting code, you can override the atStart of the CodeTransform interface. Unfortunately we don't have a utility method for you to pass a lambda ? but we can always add it later. If you wish to find an injection point, you can get the full list of code elements, and find your insertion point and add the desired instruction objects. 2. For adding new methods, you can also override the atStart/atEnd method of your ClassTransform, or do andThen(ClassTransform.endHandler(clb -> clb.withMethod(...))) to add methods. 3. To duplicate an existing method object, you can first call withMethod to create a new method, and once you have the method builder, it has a transform(otherMethod, transform) that can pipe otherMethod's contents through the transform, and finally everything goes to the method builder. Feel free to ask more or suggest. Unfortunately I am still trying to improve documentations for ClassFile API, so at this stage the documents may still be insufficient, and many of the useful methods are not covered by the JEP. Regards, Chen Liang ------------------------------ *From:* classfile-api-dev on behalf of Mark Roberts *Sent:* Friday, November 15, 2024 1:42 PM *To:* classfile-api-dev at openjdk.org *Subject:* class file transformation questions Several of our tools use the java.lang.instrument package to instrument class files when they are loaded. We currently use BCEL to do the instrumentation, but would like to move to the new java.lang.classfile package. I have gotten some basic transforms working in this environment and I see how to modify existing instructions. However, we need to perform some larger modifications to the classes we instrument. There are three basic transforms we need to perform (all on the same class file): 1. Injecting code into an existing method 2. Adding new methods 3. Duplicating an existing method with some changes Any suggestions as to how to accomplish these tasks would be much appreciated. Thank you, Mark Roberts -------------- next part -------------- An HTML attachment was scrubbed... URL: From chen.l.liang at oracle.com Thu Nov 28 00:01:02 2024 From: chen.l.liang at oracle.com (Chen Liang) Date: Thu, 28 Nov 2024 00:01:02 +0000 Subject: [External] : more class file transformation questions In-Reply-To: <8e1bfb61f3bf19ed2a155fadaa5da0cd@mail.gmail.com> References: <8e1bfb61f3bf19ed2a155fadaa5da0cd@mail.gmail.com> Message-ID: Hi Mark, 1. No, you have to do new ArrayList<>(code.elementList()) as the list returned is immutable. 2. To insert an instruction, just use List::add. To get an instruction, create one from the factory methods (usually named "of") of Instruction subclasses. Once you are done with the list, do for (var e : list) codeBuilder.with(e); 3. Bound and unbound instructions have no difference in usage when you write them. The bound instruction can be seen as a specialized version of unbound instruction, that they are cheaper for the API to obtain and supports fast copying to another CodeBuilder when condition allows (that is, the new CodeBuilder has a constantpool that extends from the constant pool of the bound instruction). Bound instructions can only be read from Class Files; unbound instructions can always be created by factory methods on the instruction interfaces. Regards, Chen ________________________________ From: Mark Roberts Sent: Wednesday, November 27, 2024 5:54 PM To: Chen Liang ; classfile-api-dev at openjdk.org Subject: RE: [External] : more class file transformation questions Okay, a List sounds interesting. 1. Would I just use CodeModel::elementList() as my initial list or do I have to make a copy somehow? 2. How would I insert a new instruction into this list? Builders don?t seem to return an item, perhaps I need to use the static instruction of() methods? 3. What is difference between Bound and Unbound instructions? Thank you, Mark From: Chen Liang > Sent: Tuesday, November 19, 2024 3:25 PM To: Mark Roberts >; classfile-api-dev at openjdk.org Subject: Re: [External] : more class file transformation questions Hi Mark, ClassFileBuilder::with takes an element object, such as a FieldModel for a ClassBuilder. It does not modify the object when applied to the new builder, except implementation details like constant pool references.. ClassFileBuilder::transform takes the same type of the object as the object to build, such as a ClassModel for a ClassBuilder. In the same time, it allows you to process all element objects of that same type of object. For example, a ClassBuilder can with a method or a field. A ClassBuilder can transform a class, and use this to merge classes, like clb.transform(sourceClassA, ClassTransform.ACCEPT_ALL).transform(sourceClassB, ClassTransform.ACCEPT_ALL) to merge source classes A and B into the ongoing class builder. For your specific use case, I think what you need is a method transform that does something like: (mb, me) -> { if (mb instanceof CodeModel code) { List codeElements = ...// prepare your list of instructions mb.withCode(codeElements::forEach); // the list of instructions get sent to the code builderOK } else { mb.with(me); } } It is also possible to do this in CodeTransform by collecting everything in a list in accept(CodeBuilder, CodeElement) and only working on the builder in atEnd(CodeBuilder), when you can see all the elements. But that code will be a bit not straightforward compared to this pattern I am showing off, and this CodeTransform can only see all elements that is from one original model instead of those from all original models (applying the merge class A and B example). P.S. When you reply, remember to choose "reply all" so that your reply gets sent to the classfile-api-dev list too. Regards, Chen Liang ________________________________ From: Mark Roberts > Sent: Tuesday, November 19, 2024 4:43 PM To: Chen Liang > Subject: [External] : more class file transformation questions Thank you, those suggestions really helped! Next question, given a builder, I?m curious about the differences between the ?with? versus ?transform? methods. I see how the transforms can simplify coding and they support composition. The implication seems to be that with and build create new objects from old while transform modifies existing objects. But based on the examples I?ve seen they look very similar. Are there significant performance differences? Or other advantages? My use case, adding instrumentation code to class files, requires inspecting every byte code and possibly modifying a lot of them. I worry that I would have to composite many transforms where one linear walk through the byte codes would be sufficient. Am I missing something? (it wouldn?t surprise me if I was) Thank you, Mark From: Chen Liang > Sent: Sunday, November 17, 2024 4:59 PM To: Mark Roberts >; classfile-api-dev at openjdk.org Subject: Re: [External] : RE: class file transformation questions Hi Mark, For performing a lot of modifications, you first can create a private static void check(CodeBuilder builder, CodeElement element) { // check the elements, and if it matches a pattern, call another method // as long as you pass the CodeBuilder and make changes to it, your changes will be saved } And then refer to this as a CodeTransform in a method reference. Or, if you want to do a huge overhaul to a CodeModel, you can get a MethodTransform, and have a specific method like: if (me instanceof CodeModel code) { methodBuilder.withCode(cob -> rebuildCode(cob, code)); } private static void rebuildCode(CodeBuilder builder, CodeModel model) {} This gives you a more holistic view of the code, and should be more friendly for massive transformations that almost rebuild the method code. (This one also allows you to pull instructions to a list with new ArrayList<>(model.elementList()), modify that list, and do list.forEach(builder) to send the results, which is used in some older patterns) Regards, Chen Liang ________________________________ From: Mark Roberts > Sent: Sunday, November 17, 2024 6:30 PM To: Chen Liang > Subject: [External] : RE: class file transformation questions Thank you for your help, it is much appreciated. I have what I guess you would call a coding style question. If you wish to perform a lot of modifications to a class file I could envision a method with hundreds of lines of code. For a smaller example, see the chaining multiple transformations example in java/lang/classfile/ClassFileTransform.html. I find this very (extremely?) difficult to read. Is there any way of writing this code in a more ?traditional? way with several, smaller method bodies? Or is there no way to use java.lang.classfile without chaining together large numbers of builders written as lambda functions? Thank you, Mark From: Chen Liang > Sent: Friday, November 15, 2024 4:48 PM To: Mark Roberts >; classfile-api-dev at openjdk.org Subject: Re: class file transformation questions Hi Mark, 1. For injecting code, you can override the atStart of the CodeTransform interface. Unfortunately we don't have a utility method for you to pass a lambda ? but we can always add it later. If you wish to find an injection point, you can get the full list of code elements, and find your insertion point and add the desired instruction objects. 1. For adding new methods, you can also override the atStart/atEnd method of your ClassTransform, or do andThen(ClassTransform.endHandler(clb -> clb.withMethod(...))) to add methods. 1. To duplicate an existing method object, you can first call withMethod to create a new method, and once you have the method builder, it has a transform(otherMethod, transform) that can pipe otherMethod's contents through the transform, and finally everything goes to the method builder. Feel free to ask more or suggest. Unfortunately I am still trying to improve documentations for ClassFile API, so at this stage the documents may still be insufficient, and many of the useful methods are not covered by the JEP. Regards, Chen Liang ________________________________ From: classfile-api-dev > on behalf of Mark Roberts > Sent: Friday, November 15, 2024 1:42 PM To: classfile-api-dev at openjdk.org > Subject: class file transformation questions Several of our tools use the java.lang.instrument package to instrument class files when they are loaded. We currently use BCEL to do the instrumentation, but would like to move to the new java.lang.classfile package. I have gotten some basic transforms working in this environment and I see how to modify existing instructions. However, we need to perform some larger modifications to the classes we instrument. There are three basic transforms we need to perform (all on the same class file): 1. Injecting code into an existing method 2. Adding new methods 3. Duplicating an existing method with some changes Any suggestions as to how to accomplish these tasks would be much appreciated. Thank you, Mark Roberts -------------- next part -------------- An HTML attachment was scrubbed... URL: From markro at cs.washington.edu Fri Nov 29 23:53:56 2024 From: markro at cs.washington.edu (Mark Roberts) Date: Fri, 29 Nov 2024 15:53:56 -0800 Subject: CodeLocalsShifter Message-ID: <072caeb1878a1f716ca8c48fad15d30c@mail.gmail.com> CodeLocalsShifter seems to shift the locals down a variable number based on the number of current locals. How do I shift the locals down a fixed number that I designate? Thanks, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Sat Nov 30 01:15:47 2024 From: liangchenblue at gmail.com (Chen Liang) Date: Fri, 29 Nov 2024 19:15:47 -0600 Subject: CodeLocalsShifter In-Reply-To: <072caeb1878a1f716ca8c48fad15d30c@mail.gmail.com> References: <072caeb1878a1f716ca8c48fad15d30c@mail.gmail.com> Message-ID: Hi Mark, Unfortunately, I think you have to write your own CodeTransfrom to shift by a fixed number; CodeLocalsShifter is intended to avoid conflicts between new variables and the required variables from method parameters. We can consider exposing a new method to create a fixed-slots CodeLocalsShifter, if you make an argument here and send an enhancement to https://bugs.java.com Regards, Chen On Fri, Nov 29, 2024 at 5:54?PM Mark Roberts wrote: > CodeLocalsShifter seems to shift the locals down a variable number based > on the number of current locals. How do I shift the locals down a fixed > number that I designate? > > > > Thanks, > > Mark > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: