From adam.sotona at oracle.com Tue Feb 7 14:33:51 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Tue, 7 Feb 2023 14:33:51 +0000 Subject: Classfile API AttributedElement.Kind removal proposal Message-ID: Hi, Based on discussion in the Classfile API PR: https://github.com/openjdk/jdk/pull/10982#discussion_r1098601988 I would like to propose removal of AttributedElement.Kind across all Classfile API. The AttributedElement.Kind models Attributes ?where applicable? and it is a duplication of each Attribute extending ClassElement, MethodElement, CodeElement, etc? Classfile API is not actively using the AttributedElement.Kind except for parsing, where inappropriate AttributedElement.Kind is resolved as UnknownAttribute. Following proposal removes all usages of AttributedElement.Kind from Classfile API: https://github.com/openjdk/jdk-sandbox/pull/48/files Please let me know is there are any objections. Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Tue Feb 7 15:02:48 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 7 Feb 2023 10:02:48 -0500 Subject: Classfile API AttributedElement.Kind removal proposal In-Reply-To: References: Message-ID: <7fe26b30-c2ec-e48e-6b57-50883fffb256@oracle.com> Just a little bit of history here.? We started writing this API before we had pattern support in the language, so we used the classic trick of having an enum constant per node type that could be switched over.? When patterns came along, we thought about removing them, but we were still a little diffident as we knew that the cost of switching over types was higher than switching over enums, and worried about performance-sensitive transformation code. I wouldn't mind seeing a quick microbench comparing switching over bytecodes the two different ways, just to see if it still shows up on the profile. On 2/7/2023 9:33 AM, Adam Sotona wrote: > > Hi, > > Based on discussion in the Classfile API PR: > https://github.com/openjdk/jdk/pull/10982#discussion_r1098601988 > > I would like to propose removal of AttributedElement.Kind across all > Classfile API. > > The AttributedElement.Kind models Attributes ?where applicable? and it > is a duplication of each Attribute extending ClassElement, > MethodElement, CodeElement, etc? > > Classfile API is not actively using the AttributedElement.Kind except > for parsing, where inappropriate AttributedElement.Kind is resolved as > UnknownAttribute. > > Following proposal removes all usages of AttributedElement.Kind from > Classfile API: > > https://github.com/openjdk/jdk-sandbox/pull/48/files > > Please let me know is there are any objections. > > Thanks, > > Adam > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Tue Feb 7 16:22:49 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Tue, 7 Feb 2023 16:22:49 +0000 Subject: Classfile API AttributedElement.Kind removal proposal In-Reply-To: <7fe26b30-c2ec-e48e-6b57-50883fffb256@oracle.com> References: <7fe26b30-c2ec-e48e-6b57-50883fffb256@oracle.com> Message-ID: Actually we have no use case to benchmark. Newly proposed code simply does not check for appropriate ?where applicable? during bound attributes parsing and passes the Attribute instance on anyway. There was no other use of that information across the Classfile API. Alternatively I can implement the check by single switch asking if the provided attribute is instanceof XyzElement according to the enclosing AttributedEleement type, and exchange it for UnknownAttribute if not appropriate. In comparison to Set::contains there should be no regression. From: Brian Goetz Date: Tuesday, 7 February 2023 16:02 To: Adam Sotona , classfile-api-dev at openjdk.org Subject: Re: Classfile API AttributedElement.Kind removal proposal Just a little bit of history here. We started writing this API before we had pattern support in the language, so we used the classic trick of having an enum constant per node type that could be switched over. When patterns came along, we thought about removing them, but we were still a little diffident as we knew that the cost of switching over types was higher than switching over enums, and worried about performance-sensitive transformation code. I wouldn't mind seeing a quick microbench comparing switching over bytecodes the two different ways, just to see if it still shows up on the profile. On 2/7/2023 9:33 AM, Adam Sotona wrote: Hi, Based on discussion in the Classfile API PR: https://github.com/openjdk/jdk/pull/10982#discussion_r1098601988 I would like to propose removal of AttributedElement.Kind across all Classfile API. The AttributedElement.Kind models Attributes ?where applicable? and it is a duplication of each Attribute extending ClassElement, MethodElement, CodeElement, etc? Classfile API is not actively using the AttributedElement.Kind except for parsing, where inappropriate AttributedElement.Kind is resolved as UnknownAttribute. Following proposal removes all usages of AttributedElement.Kind from Classfile API: https://github.com/openjdk/jdk-sandbox/pull/48/files Please let me know is there are any objections. Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Thu Feb 9 12:49:34 2023 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 9 Feb 2023 12:49:34 +0000 Subject: ClassReader and BufWriter Message-ID: <58056373-8808-8d62-bd90-61a45b557154@oracle.com> Hi, as I'm looking at the classfile PR (great work!), I have few questions on ClassReader and BufWriter: * I believe they are only used for attribute reading/writing - so perhaps they should belong to the `attribute` package? * BufWriter should be renamed ClassWriter (as it's the dual?) While I understand this is in bikeshed territory, I think it would be nice if the number of classes in the main package was kept as low as possible, to let the true nature of the API (models, elements, builders) stand out more. Cheers Maurizio From brian.goetz at oracle.com Thu Feb 9 14:15:06 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 9 Feb 2023 09:15:06 -0500 Subject: ClassReader and BufWriter In-Reply-To: <58056373-8808-8d62-bd90-61a45b557154@oracle.com> References: <58056373-8808-8d62-bd90-61a45b557154@oracle.com> Message-ID: <560aa92b-befe-1b87-4f1f-3d88d158df13@oracle.com> On 2/9/2023 7:49 AM, Maurizio Cimadamore wrote: > Hi, > as I'm looking at the classfile PR (great work!), I have few questions > on ClassReader and BufWriter: > > * I believe they are only used for attribute reading/writing - so > perhaps they should belong to the `attribute` package? They are used for more than attribute reading and writing. BufWriter is used, for example, by DirectClassBuilder::build to build the entire classfile -- the ClassFile, constant pool, method_info structures, field_info structures, etc.? (Because we don't always know the sizes, to minimize buffer copies, we write pieces of the classfile in chunks and join them.? DBC::build has two BWs for this, one for the "head" (magic number, fixed ClassFile fields, constant pool) and one for the tail (fields, methods, attributes), because we don't know how big the CP will be until we finish rendering the entire "tail". A BufWriter would be a pure buffer abstraction -- it is 99% about just writing various kinds of data to a growable buffer -- but it does carry with it a reference to a constant pool builder (which may be shared across multiple BufWriters if they are writing different chunks of the same classfile), since it needs to be able to write not only ints, but constant pool indexes, and this might require growing the constant pool to add a new entry. > * BufWriter should be renamed ClassWriter (as it's the dual?) Yes, but no :)? This code got refactored a number of times (including one like what you suggest), but the naming asymmetry derives from a functional asymmetry.? We write classfiles in lots of small chunks -- we accumulate the bytes of constant pools, methods, attributes, etc in their own little buffers -- but we read the classfile monolithically.? (We have to, because you can't even find the methods until you've parsed the constant pool.)? So ClassReader is tied to a single byte[] which represents the whole classfile, and manages the constant pool, navigation, byte access, everything -- whereas BufWriter only knows about writing to a buffer, plus uses the service of a ConstantPoolBuilder to render constant pool offsets. So the two do exist as different levels of abstraction. > While I understand this is in bikeshed territory, I think it would be > nice if the number of classes in the main package was kept as low as > possible, to let the true nature of the API (models, elements, > builders) stand out more. I did consider this organizational question.? The `attributes` package is easy, because it contains almost exclusively model classes for the various attributes, which are derived almost mechanically from JVMS.? You could arguably move a few things like BufWriter (which mostly serve the implementation) into a `util` package, but this wouldn't really do very much to declutter the main package; there are just a lot of different kinds of entities (methods, fields, classes, attributes, constant pool entries, instructions, plus elements of the classfile header like Superclass) in a classfile, and each gets models, builders, elements, transforms, etc.? These feel like they go together.? So it felt that this process reached diminishing returns the last time I tried this. From maurizio.cimadamore at oracle.com Thu Feb 9 14:28:54 2023 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 9 Feb 2023 14:28:54 +0000 Subject: ClassReader and BufWriter In-Reply-To: <560aa92b-befe-1b87-4f1f-3d88d158df13@oracle.com> References: <58056373-8808-8d62-bd90-61a45b557154@oracle.com> <560aa92b-befe-1b87-4f1f-3d88d158df13@oracle.com> Message-ID: On 09/02/2023 14:15, Brian Goetz wrote: > > > On 2/9/2023 7:49 AM, Maurizio Cimadamore wrote: >> Hi, >> as I'm looking at the classfile PR (great work!), I have few >> questions on ClassReader and BufWriter: >> >> * I believe they are only used for attribute reading/writing - so >> perhaps they should belong to the `attribute` package? > > They are used for more than attribute reading and writing. BufWriter > is used, for example, by DirectClassBuilder::build to build the entire > classfile -- the ClassFile, constant pool, method_info structures, > field_info structures, etc.? (Because we don't always know the sizes, > to minimize buffer copies, we write pieces of the classfile in chunks > and join them.? DBC::build has two BWs for this, one for the "head" > (magic number, fixed ClassFile fields, constant pool) and one for the > tail (fields, methods, attributes), because we don't know how big the > CP will be until we finish rendering the entire "tail". Sure, but DirectClassBuilder is an implementation class. I'm talking about API usages here. It seems to me that BufWriter is an implementation abstraction that "happens" to implement the interface for attribute writing (which is the only place where the user needs to write things by hand). > > A BufWriter would be a pure buffer abstraction -- it is 99% about just > writing various kinds of data to a growable buffer -- but it does > carry with it a reference to a constant pool builder (which may be > shared across multiple BufWriters if they are writing different chunks > of the same classfile), since it needs to be able to write not only > ints, but constant pool indexes, and this might require growing the > constant pool to add a new entry. > >> * BufWriter should be renamed ClassWriter (as it's the dual?) > > Yes, but no :)? This code got refactored a number of times (including > one like what you suggest), but the naming asymmetry derives from a > functional asymmetry.? We write classfiles in lots of small chunks -- > we accumulate the bytes of constant pools, methods, attributes, etc in > their own little buffers -- but we read the classfile monolithically.? > (We have to, because you can't even find the methods until you've > parsed the constant pool.)? So ClassReader is tied to a single byte[] > which represents the whole classfile, and manages the constant pool, > navigation, byte access, everything -- whereas BufWriter only knows > about writing to a buffer, plus uses the service of a > ConstantPoolBuilder to render constant pool offsets. > > So the two do exist as different levels of abstraction. I get the subtle, implementation-driven distinction - but I find BufWriter a rather weak name API-wise. That said, this stuff is subjective. > >> While I understand this is in bikeshed territory, I think it would be >> nice if the number of classes in the main package was kept as low as >> possible, to let the true nature of the API (models, elements, >> builders) stand out more. > > I did consider this organizational question.? The `attributes` package > is easy, because it contains almost exclusively model classes for the > various attributes, which are derived almost mechanically from JVMS.? > You could arguably move a few things like BufWriter (which mostly > serve the implementation) into a `util` package, but this wouldn't > really do very much to declutter the main package; there are just a > lot of different kinds of entities (methods, fields, classes, > attributes, constant pool entries, instructions, plus elements of the > classfile header like Superclass) in a classfile, and each gets > models, builders, elements, transforms, etc.? These feel like they go > together.? So it felt that this process reached diminishing returns > the last time I tried this. Maurizio > > From maurizio.cimadamore at oracle.com Thu Feb 9 14:37:03 2023 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 9 Feb 2023 14:37:03 +0000 Subject: ClassReader and BufWriter In-Reply-To: References: <58056373-8808-8d62-bd90-61a45b557154@oracle.com> <560aa92b-befe-1b87-4f1f-3d88d158df13@oracle.com> Message-ID: On 09/02/2023 14:28, Maurizio Cimadamore wrote: > Sure, but DirectClassBuilder is an implementation class. I'm talking > about API usages here. It seems to me that BufWriter is an > implementation abstraction that "happens" to implement the interface > for attribute writing (which is the only place where the user needs to > write things by hand). I stand corrected - there's more usages: https://htmlpreview.github.io/?https://raw.githubusercontent.com/openjdk/jdk-sandbox/classfile-api-javadoc-branch/doc/classfile-api/javadoc/java.base/jdk/internal/classfile/class-use/BufWriter.html That said, I'm unsure as to what these usages refer to (my IDE shows no usages). Maurizio From brian.goetz at oracle.com Thu Feb 9 14:54:49 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 9 Feb 2023 09:54:49 -0500 Subject: ClassReader and BufWriter In-Reply-To: References: <58056373-8808-8d62-bd90-61a45b557154@oracle.com> <560aa92b-befe-1b87-4f1f-3d88d158df13@oracle.com> Message-ID: On 2/9/2023 9:28 AM, Maurizio Cimadamore wrote: > > On 09/02/2023 14:15, Brian Goetz wrote: >> >> >> On 2/9/2023 7:49 AM, Maurizio Cimadamore wrote: >>> Hi, >>> as I'm looking at the classfile PR (great work!), I have few >>> questions on ClassReader and BufWriter: >>> >>> * I believe they are only used for attribute reading/writing - so >>> perhaps they should belong to the `attribute` package? >> >> They are used for more than attribute reading and writing. BufWriter >> is used, for example, by DirectClassBuilder::build to build the >> entire classfile -- the ClassFile, constant pool, method_info >> structures, field_info structures, etc.? (Because we don't always >> know the sizes, to minimize buffer copies, we write pieces of the >> classfile in chunks and join them.? DBC::build has two BWs for this, >> one for the "head" (magic number, fixed ClassFile fields, constant >> pool) and one for the tail (fields, methods, attributes), because we >> don't know how big the CP will be until we finish rendering the >> entire "tail". > Sure, but DirectClassBuilder is an implementation class. I'm talking > about API usages here. It seems to me that BufWriter is an > implementation abstraction that "happens" to implement the interface > for attribute writing (which is the only place where the user needs to > write things by hand). It may be the case at this point that the only place that BufWriter pokes its head out in the APIs is in the attributes (that wasn't always true), but it would definitely be wrong to put BufWriter in the attribute package -- it has nothing to do with attributes, and it is entirely imaginable that it could show up elsewhere again. You could argue it is a "utility" abstraction and therefore goes in a 'util' package, and that would be a fair argument.? I'm not sure whether it buys us much in terms of API approachability, but I wouldn't object.? (Though the current `util` package is more user-facing utilities, like ClassPrinter, and has only one thing in it.) Other things that could be moved elsewhere: ?- The ClassFile-specific elements -- Superclass, Interfaces, ClassfileVersion, maybe AccessFlags (which is shared with method and field models).? But, where?? And there's just not many of them. ?- Instruction-related classes -- Instruction, PseudoInstruction, Opcode could go into `instruction` (though its a little weird for the Element types (Instruction) to be elsewhere, which leaves just Opcode as a candidate) ?- AttributeMapper and Attributes could go into `attribute` ?- BootstrapMethodEntry could go into `constantpool` ?- *Transform could go into a `transform` package, but that severs the Xxx{Model,Element,Builder,Transform} symmetry, and there's just a handful of them A bigger reorg would put all the Model and Element types into a `model` package.? This might achieve the outcome I think you are looking for -- that when someone looks at the top level javadoc, they see the key entry points. From gary.frost at oracle.com Thu Feb 9 15:00:17 2023 From: gary.frost at oracle.com (Gary Frost) Date: Thu, 9 Feb 2023 15:00:17 +0000 Subject: Access to BCI for Unbound and Bound instructions Message-ID: First thanks for this API. I am switching to it from a proprietary library I wrote a long time ago, I really like it. But have a question about Unbound vs Bound instructions. I am interested in the rationale behind Bound and Unbound Instructions. It seems to be related to whether they are somehow ?tethered? to a BCI/Pos? or indeed even to a specific CodeModel or var slot. Is that correct? Or is there some other reason for separating the instructions this way? In my use case I am interested in the relative order of instructions and therefore need the BCI for all instructions. For Bound instructions I can extract the pos (which can be easily mapped to BCI). For Unbound I cannot easily determine the BCI. Undeterred ? my first hack was to just create a side mappings Map< Instruction, Integer> as I walk through the elements. So.... codeModel.elements().forEach((codeElement -> { /* Build side mappings here */ }); Sadly the lack of guaranteed 'identity' from CodeImpl.bcToInstruction(int bc, int pos) caught me off-guard :) It turns out that some instructions are indeed allocated return switch (bc) { case BIPUSH -> new BoundArgumentConstantInstruction(... pos); whereas others (those pesky Unbound ones) are pulled from a cache of pre-allocated instructions default -> { InstructionData.singletonInstructions[bc]; rendering my Map somewhat useless ;) I doubt if my 'need' for accessing the BCI of each instruction is unique, so I am wondering whether the value of this Unbound/Bound separation might be worth it in the long run. Or at least whether a CodeModel.getBCI(Instruction) helper method could be introduced, which has tricks to step around this 'identity' issue. Gary -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Thu Feb 9 15:16:40 2023 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 9 Feb 2023 15:16:40 +0000 Subject: ClassReader and BufWriter In-Reply-To: References: <58056373-8808-8d62-bd90-61a45b557154@oracle.com> <560aa92b-befe-1b87-4f1f-3d88d158df13@oracle.com> Message-ID: <524e7713-4fb0-1924-0435-8350568d7ee9@oracle.com> On 09/02/2023 14:54, Brian Goetz wrote: > ?- The ClassFile-specific elements -- Superclass, Interfaces, > ClassfileVersion, maybe AccessFlags (which is shared with method and > field models).? But, where?? And there's just not many of them. And annotations, and signatures. Unfortunately, as these have nested types, they impact quite a lot on the javadoc :-( https://htmlpreview.github.io/?https://raw.githubusercontent.com/openjdk/jdk-sandbox/classfile-api-javadoc-branch/doc/classfile-api/javadoc/java.base/jdk/internal/classfile/package-summary.html I counted 93 classes in the main package javadoc. Of these, 10 are for signatures, 17 are for annotations/annotation values and 16 are for type annotations. That's 43 types - which is quite a lot, considering that these types are not meant to be the star of the show. Maurizio From gary.frost at oracle.com Thu Feb 9 16:03:10 2023 From: gary.frost at oracle.com (Gary Frost) Date: Thu, 9 Feb 2023 16:03:10 +0000 Subject: Reflective access to bytes[] of loaded class Message-ID: I would like to make a case for adding a reflection API for getting the bytes for class loaded by the VM as part of this Classfile API. Something akin to byte[] Class.getClassfileBytes(); At present developers usually resort to something like clazz.getClassLoader().getResourceAsStream( clazz.getName().replace(".", "/") + ".class") Ignoring the fact that we may have just forced an expensive network fetch, to bytes that the JVM clearly already has 'squirrelled away' somewhere... For me this is hugely problematic as there is no guarentee that the bytes fetched from such a stream will match the bytes that the JVM is using for the class in question.... Java offers all sorts of opportunities (JVMTI agents, Java agents, even custom ClassLoaders) for mutating the incoming classfile's bytes. All of which the stream fetched via getResourceAsStream() completely sidesteps. This may well be considered 'out of bounds' for the Classfile API, but I think we should consider it, as it seems to be something that users of the Classfile API will need going forward. Gary -------------- next part -------------- An HTML attachment was scrubbed... URL: From heidinga at redhat.com Thu Feb 9 16:35:36 2023 From: heidinga at redhat.com (Dan Heidinga) Date: Thu, 9 Feb 2023 11:35:36 -0500 Subject: Reflective access to bytes[] of loaded class In-Reply-To: References: Message-ID: On Thu, Feb 9, 2023 at 11:03 AM Gary Frost wrote: > I would like to make a case for adding a reflection API for getting the > bytes for class loaded by the VM as part of this Classfile API. > > Something akin to > > byte[] Class.getClassfileBytes(); > I've often wanted the same kind of API when doing one off tests and minor modifications so I'm very sympathetic to the request. Typically when doing more "serious" class modification, I've used the Instrumentation::retransform API [0] or written a native JVMTI agent [1] and then such an api to get the current classfile bytes isn't required. Does this match your typical use cases as well? The JVM does some interesting handling of classfile bytes in these cases - it allows non-retransform-capable agents to do a 1 time modification of the original classfile bytes and then saves those bytes away before passing them on to the retransform-capable agents. One subsequent retransform events, those saved bytes are reused and only the retransform-capable agents get a chance to modify the bytes. The reason this matters is that existing retransform-capable agents aren't expecting to see the modifications they've made present in the classfile bytes they get passed. Providing an easy api to get the currently executing bytecodes will mean that these agents will now "see" their own modifications which may result in incompatibilities. I'd be interested in feedback from existing major agent providers to see how much of a problem such a change would cause them before pursuing it unless there's a more common pattern of use I'm unaware of. [0] https://cr.openjdk.java.net/~iris/se/17/latestSpec/api/java.instrument/java/lang/instrument/Instrumentation.html#retransformClasses(java.lang.Class.. .) [1] https://cr.openjdk.java.net/~iris/se/17/latestSpec/specs/jvmti.html#RetransformClasses > > > > At present developers usually resort to something like > > clazz.getClassLoader().getResourceAsStream( > clazz.getName().replace(".", "/") + ".class") > > Ignoring the fact that we may have just forced an expensive network fetch, > to bytes that the JVM clearly already has 'squirrelled away' somewhere... > > For me this is hugely problematic as there is no guarentee that the bytes > fetched from such a stream will match the bytes that the JVM is using for > the class in question.... > > Java offers all sorts of opportunities (JVMTI agents, Java agents, even > custom ClassLoaders) for mutating the incoming classfile's bytes. All of > which the stream fetched via getResourceAsStream() completely sidesteps. > See above (or the JVMTI spec) for which bytes get passed through to agents. The story is more complex than it first appears. --Dan > > This may well be considered 'out of bounds' for the Classfile API, but I > think we should consider it, as it seems to be something that users of the > Classfile API will need going forward. > > Gary > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gary.frost at oracle.com Thu Feb 9 16:58:53 2023 From: gary.frost at oracle.com (Gary Frost) Date: Thu, 9 Feb 2023 16:58:53 +0000 Subject: [External] : Re: Reflective access to bytes[] of loaded class In-Reply-To: References: Message-ID: Dan I have indeed used JVMTI (and Java) agents for this. In the case of JVMTI by keeping a std::map, and providing a JNI call to get the bytes. It works well for cases where we know the set of classes we want bytes for at JVM 'launch time' , or there is something in the class (name match? constant pool entry?) that we can trigger to prune the size of the map. Otherwise we are forced to retain a map for all classes, just in case. And of course who knew whether our JVMTI agent was the last 'actor' in the chain of possible mutators to play with the bytes. Hence the desire for runtime help ________________________________ From: Dan Heidinga Sent: Thursday, February 9, 2023 4:35 PM To: Gary Frost Cc: classfile-api-dev at openjdk.org Subject: [External] : Re: Reflective access to bytes[] of loaded class On Thu, Feb 9, 2023 at 11:03 AM Gary Frost > wrote: I would like to make a case for adding a reflection API for getting the bytes for class loaded by the VM as part of this Classfile API. Something akin to byte[] Class.getClassfileBytes(); I've often wanted the same kind of API when doing one off tests and minor modifications so I'm very sympathetic to the request. Typically when doing more "serious" class modification, I've used the Instrumentation::retransform API [0] or written a native JVMTI agent [1] and then such an api to get the current classfile bytes isn't required. Does this match your typical use cases as well? The JVM does some interesting handling of classfile bytes in these cases - it allows non-retransform-capable agents to do a 1 time modification of the original classfile bytes and then saves those bytes away before passing them on to the retransform-capable agents. One subsequent retransform events, those saved bytes are reused and only the retransform-capable agents get a chance to modify the bytes. The reason this matters is that existing retransform-capable agents aren't expecting to see the modifications they've made present in the classfile bytes they get passed. Providing an easy api to get the currently executing bytecodes will mean that these agents will now "see" their own modifications which may result in incompatibilities. I'd be interested in feedback from existing major agent providers to see how much of a problem such a change would cause them before pursuing it unless there's a more common pattern of use I'm unaware of. [0] https://cr.openjdk.java.net/~iris/se/17/latestSpec/api/java.instrument/java/lang/instrument/Instrumentation.html#retransformClasses(java.lang.Class...) [1] https://cr.openjdk.java.net/~iris/se/17/latestSpec/specs/jvmti.html#RetransformClasses At present developers usually resort to something like clazz.getClassLoader().getResourceAsStream( clazz.getName().replace(".", "/") + ".class") Ignoring the fact that we may have just forced an expensive network fetch, to bytes that the JVM clearly already has 'squirrelled away' somewhere... For me this is hugely problematic as there is no guarentee that the bytes fetched from such a stream will match the bytes that the JVM is using for the class in question.... Java offers all sorts of opportunities (JVMTI agents, Java agents, even custom ClassLoaders) for mutating the incoming classfile's bytes. All of which the stream fetched via getResourceAsStream() completely sidesteps. See above (or the JVMTI spec) for which bytes get passed through to agents. The story is more complex than it first appears. --Dan This may well be considered 'out of bounds' for the Classfile API, but I think we should consider it, as it seems to be something that users of the Classfile API will need going forward. Gary -------------- next part -------------- An HTML attachment was scrubbed... URL: From heidinga at redhat.com Thu Feb 9 18:24:44 2023 From: heidinga at redhat.com (Dan Heidinga) Date: Thu, 9 Feb 2023 13:24:44 -0500 Subject: [External] : Re: Reflective access to bytes[] of loaded class In-Reply-To: References: Message-ID: Thanks for confirming, Gary. On Thu, Feb 9, 2023 at 11:59 AM Gary Frost wrote: > Dan > > I have indeed used JVMTI (and Java) agents for this. In the case of JVMTI > by keeping a std::map, and providing a JNI call to get > the bytes. > > It works well for cases where we know the set of classes we want bytes for > at JVM 'launch time' , or there is something in the class (name match? > constant pool entry?) that we can trigger to prune the size of the map. > Otherwise we are forced to retain a map for all classes, just in case. > > And of course who knew whether our JVMTI agent was the last 'actor' in the > chain of possible mutators to play with the bytes. > Can you expand on the use case for getting the current bytes from the runtime? As I said in the previous email, they aren't a good candidate for feeding back into the runtime due to the existing JVMTI agent process. Is this mostly for testing purposes? --Dan > > > Hence the desire for runtime help > > > ------------------------------ > *From:* Dan Heidinga > *Sent:* Thursday, February 9, 2023 4:35 PM > *To:* Gary Frost > *Cc:* classfile-api-dev at openjdk.org > *Subject:* [External] : Re: Reflective access to bytes[] of loaded class > > > > On Thu, Feb 9, 2023 at 11:03 AM Gary Frost wrote: > > I would like to make a case for adding a reflection API for getting the > bytes for class loaded by the VM as part of this Classfile API. > > Something akin to > > byte[] Class.getClassfileBytes(); > > > I've often wanted the same kind of API when doing one off tests and minor > modifications so I'm very sympathetic to the request. Typically when doing > more "serious" class modification, I've used the > Instrumentation::retransform API [0] or written a native JVMTI agent [1] > and then such an api to get the current classfile bytes isn't required. > Does this match your typical use cases as well? > > The JVM does some interesting handling of classfile bytes in these cases - > it allows non-retransform-capable agents to do a 1 time modification of the > original classfile bytes and then saves those bytes away before passing > them on to the retransform-capable agents. One subsequent retransform > events, those saved bytes are reused and only the retransform-capable > agents get a chance to modify the bytes. > > The reason this matters is that existing retransform-capable agents aren't > expecting to see the modifications they've made present in the classfile > bytes they get passed. Providing an easy api to get the currently > executing bytecodes will mean that these agents will now "see" their own > modifications which may result in incompatibilities. > > I'd be interested in feedback from existing major agent providers to see > how much of a problem such a change would cause them before pursuing it > unless there's a more common pattern of use I'm unaware of. > > [0] > https://cr.openjdk.java.net/~iris/se/17/latestSpec/api/java.instrument/java/lang/instrument/Instrumentation.html#retransformClasses(java.lang.Class.. > .) > [1] > https://cr.openjdk.java.net/~iris/se/17/latestSpec/specs/jvmti.html#RetransformClasses > > > > > At present developers usually resort to something like > > clazz.getClassLoader().getResourceAsStream( > clazz.getName().replace(".", "/") + ".class") > > Ignoring the fact that we may have just forced an expensive network fetch, > to bytes that the JVM clearly already has 'squirrelled away' somewhere... > > For me this is hugely problematic as there is no guarentee that the bytes > fetched from such a stream will match the bytes that the JVM is using for > the class in question.... > > Java offers all sorts of opportunities (JVMTI agents, Java agents, even > custom ClassLoaders) for mutating the incoming classfile's bytes. All of > which the stream fetched via getResourceAsStream() completely sidesteps. > > > See above (or the JVMTI spec) for which bytes get passed through to > agents. The story is more complex than it first appears. > > --Dan > > > > This may well be considered 'out of bounds' for the Classfile API, but I > think we should consider it, as it seems to be something that users of the > Classfile API will need going forward. > > Gary > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Thu Feb 9 18:52:37 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 9 Feb 2023 13:52:37 -0500 Subject: [External] : Re: Reflective access to bytes[] of loaded class In-Reply-To: References: Message-ID: <578390b7-b8f7-633c-fef7-dbe0722d4625@oracle.com> One concern that we have is that unrestricted access to bytecode of loaded classes may constitute an attack vector for malicious code. While we don't recommend people program like this: ??? boolean checkPassword(String pw) { ??????? return "s00perSeekrit".equals(pw); ??? } arbitrary access to bytecode will compromise the "security" of this code in ways that the author might not have reasonably forseen. On 2/9/2023 1:24 PM, Dan Heidinga wrote: > Thanks for confirming, Gary. > > On Thu, Feb 9, 2023 at 11:59 AM Gary Frost wrote: > > Dan > > I have indeed used JVMTI (and Java) agents for this. In the case > of JVMTI by keeping a std::map, and providing > a JNI call to get the bytes. > > It works well for cases where we know the set of classes we want > bytes for at JVM 'launch time' , or there is something in the > class (name match? constant pool entry?) that we can trigger to > prune the size of the map.??? Otherwise we are forced to retain a > map for all classes, just in case. > > And of course who knew whether our JVMTI agent was the last > 'actor' in the chain of possible mutators to play with the bytes. > > > Can you expand on the use case for getting the current bytes from the > runtime?? As I said in the previous email, they aren't a good > candidate for feeding back into the runtime due to the existing JVMTI > agent process.? Is this mostly for testing purposes? > > --Dan > > > > Hence the desire for runtime help > > > ------------------------------------------------------------------------ > *From:* Dan Heidinga > *Sent:* Thursday, February 9, 2023 4:35 PM > *To:* Gary Frost > *Cc:* classfile-api-dev at openjdk.org > *Subject:* [External] : Re: Reflective access to bytes[] of loaded > class > > > On Thu, Feb 9, 2023 at 11:03 AM Gary Frost > wrote: > > I would like to make a case for adding a reflection API for > getting the bytes for class loaded by the VM as part of this > Classfile API. > > Something akin to > > byte[] Class.getClassfileBytes(); > > > I've often wanted the same kind of API when doing one off tests > and minor modifications so I'm very sympathetic to the request.? > Typically when doing more "serious" class modification, I've used > the Instrumentation::retransform API [0] or written a native JVMTI > agent [1] and then such an api to get the current classfile bytes > isn't required.? Does this match your typical use cases as well? > > The JVM does some interesting handling of classfile bytes in these > cases - it allows non-retransform-capable agents to do a 1 time > modification of the original classfile bytes and then saves those > bytes away before passing them on to the retransform-capable > agents.? One subsequent retransform events, those saved bytes are > reused and only the retransform-capable agents get a chance to > modify the bytes. > > The reason this matters is that existing retransform-capable > agents aren't expecting to see the modifications they've made > present in the classfile bytes they get passed.? Providing an easy > api to get the currently executing bytecodes will mean that these > agents will now "see" their own modifications which may result in > incompatibilities. > > I'd be interested in feedback from existing major agent providers > to see how much of a problem such a change would cause them before > pursuing it unless there's a more common pattern of use I'm > unaware of. > > [0] > https://cr.openjdk.java.net/~iris/se/17/latestSpec/api/java.instrument/java/lang/instrument/Instrumentation.html#retransformClasses(java.lang.Class...) > > [1] > https://cr.openjdk.java.net/~iris/se/17/latestSpec/specs/jvmti.html#RetransformClasses > > > > > At present developers usually resort to something like > > clazz.getClassLoader().getResourceAsStream( > ??? clazz.getName().replace(".", "/") + ".class") > > Ignoring the fact that we may have just forced an expensive > network fetch, to bytes that the JVM clearly already has > 'squirrelled away' somewhere... > > For me this is hugely problematic as there is no guarentee > that the bytes fetched from such a stream will match the bytes > that the JVM is using for the class in question.... > > Java offers all sorts of opportunities (JVMTI agents, Java > agents, even custom ClassLoaders) for mutating the incoming > classfile's bytes. All of which the stream fetched via > getResourceAsStream() completely sidesteps. > > > See above (or the JVMTI spec) for which bytes get passed through > to agents.? The story is more complex than it first appears. > > --Dan > > > This may well be considered 'out of bounds' for the Classfile > API, but I think we should consider it, as it seems to be > something that users of the Classfile API will need going > forward. > > Gary > -------------- next part -------------- An HTML attachment was scrubbed... URL: From izzeldeen03 at gmail.com Thu Feb 9 19:03:08 2023 From: izzeldeen03 at gmail.com (izz) Date: Thu, 9 Feb 2023 19:03:08 +0000 Subject: Reflective access to bytes[] of loaded class In-Reply-To: <578390b7-b8f7-633c-fef7-dbe0722d4625@oracle.com> References: , <578390b7-b8f7-633c-fef7-dbe0722d4625@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: E22F4410D3A149378C5C02109992A60E.png Type: image/png Size: 143 bytes Desc: not available URL: From gary.frost at oracle.com Fri Feb 10 13:54:37 2023 From: gary.frost at oracle.com (Gary Frost) Date: Fri, 10 Feb 2023 13:54:37 +0000 Subject: [External] : Re: Reflective access to bytes[] of loaded class In-Reply-To: <578390b7-b8f7-633c-fef7-dbe0722d4625@oracle.com> References: <578390b7-b8f7-633c-fef7-dbe0722d4625@oracle.com> Message-ID: Brian I am not sure how having a reflective method to access bytecode opens up any more security issues than having access to javap -p -c ? Unless the password itself (in your example) was injected by a transformation of the code by ClassLoader/JVMTI/Java agent. Maybe I am missing something. ________________________________ From: Brian Goetz Sent: Thursday, February 9, 2023 6:52 PM To: Dan Heidinga ; Gary Frost Cc: classfile-api-dev at openjdk.org Subject: Re: [External] : Re: Reflective access to bytes[] of loaded class One concern that we have is that unrestricted access to bytecode of loaded classes may constitute an attack vector for malicious code. While we don't recommend people program like this: boolean checkPassword(String pw) { return "s00perSeekrit".equals(pw); } arbitrary access to bytecode will compromise the "security" of this code in ways that the author might not have reasonably forseen. On 2/9/2023 1:24 PM, Dan Heidinga wrote: Thanks for confirming, Gary. On Thu, Feb 9, 2023 at 11:59 AM Gary Frost > wrote: Dan I have indeed used JVMTI (and Java) agents for this. In the case of JVMTI by keeping a std::map, and providing a JNI call to get the bytes. It works well for cases where we know the set of classes we want bytes for at JVM 'launch time' , or there is something in the class (name match? constant pool entry?) that we can trigger to prune the size of the map. Otherwise we are forced to retain a map for all classes, just in case. And of course who knew whether our JVMTI agent was the last 'actor' in the chain of possible mutators to play with the bytes. Can you expand on the use case for getting the current bytes from the runtime? As I said in the previous email, they aren't a good candidate for feeding back into the runtime due to the existing JVMTI agent process. Is this mostly for testing purposes? --Dan Hence the desire for runtime help ________________________________ From: Dan Heidinga > Sent: Thursday, February 9, 2023 4:35 PM To: Gary Frost > Cc: classfile-api-dev at openjdk.org > Subject: [External] : Re: Reflective access to bytes[] of loaded class On Thu, Feb 9, 2023 at 11:03 AM Gary Frost > wrote: I would like to make a case for adding a reflection API for getting the bytes for class loaded by the VM as part of this Classfile API. Something akin to byte[] Class.getClassfileBytes(); I've often wanted the same kind of API when doing one off tests and minor modifications so I'm very sympathetic to the request. Typically when doing more "serious" class modification, I've used the Instrumentation::retransform API [0] or written a native JVMTI agent [1] and then such an api to get the current classfile bytes isn't required. Does this match your typical use cases as well? The JVM does some interesting handling of classfile bytes in these cases - it allows non-retransform-capable agents to do a 1 time modification of the original classfile bytes and then saves those bytes away before passing them on to the retransform-capable agents. One subsequent retransform events, those saved bytes are reused and only the retransform-capable agents get a chance to modify the bytes. The reason this matters is that existing retransform-capable agents aren't expecting to see the modifications they've made present in the classfile bytes they get passed. Providing an easy api to get the currently executing bytecodes will mean that these agents will now "see" their own modifications which may result in incompatibilities. I'd be interested in feedback from existing major agent providers to see how much of a problem such a change would cause them before pursuing it unless there's a more common pattern of use I'm unaware of. [0] https://cr.openjdk.java.net/~iris/se/17/latestSpec/api/java.instrument/java/lang/instrument/Instrumentation.html#retransformClasses(java.lang.Class...) [1] https://cr.openjdk.java.net/~iris/se/17/latestSpec/specs/jvmti.html#RetransformClasses At present developers usually resort to something like clazz.getClassLoader().getResourceAsStream( clazz.getName().replace(".", "/") + ".class") Ignoring the fact that we may have just forced an expensive network fetch, to bytes that the JVM clearly already has 'squirrelled away' somewhere... For me this is hugely problematic as there is no guarentee that the bytes fetched from such a stream will match the bytes that the JVM is using for the class in question.... Java offers all sorts of opportunities (JVMTI agents, Java agents, even custom ClassLoaders) for mutating the incoming classfile's bytes. All of which the stream fetched via getResourceAsStream() completely sidesteps. See above (or the JVMTI spec) for which bytes get passed through to agents. The story is more complex than it first appears. --Dan This may well be considered 'out of bounds' for the Classfile API, but I think we should consider it, as it seems to be something that users of the Classfile API will need going forward. Gary -------------- next part -------------- An HTML attachment was scrubbed... URL: From gary.frost at oracle.com Fri Feb 10 14:00:57 2023 From: gary.frost at oracle.com (Gary Frost) Date: Fri, 10 Feb 2023 14:00:57 +0000 Subject: [External] : Re: Reflective access to bytes[] of loaded class In-Reply-To: References: Message-ID: Sorry Dan. I missed your question ? "Can you expand on the use case for getting the current bytes from the runtime? As I said in the previous email, they aren't a good candidate for feeding back into the runtime due to the existing JVMTI agent process. Is this mostly for testing purposes?" My usecase is 'lifting' program structure from bytecode so that I can target a different runtime. Specifically, bytecode->OpenCL... or CUDA or SPIRV or PTX or Metal so that we can execute code on SIMD/GPU devices. Essentially performing a similar function as IntelliJ's Fernflower based plugin, which maps (quite successfully) bytecode->Java source for files without source available. Gary ________________________________ From: Dan Heidinga Sent: Thursday, February 9, 2023 6:24 PM To: Gary Frost Cc: classfile-api-dev at openjdk.org Subject: Re: [External] : Re: Reflective access to bytes[] of loaded class Thanks for confirming, Gary. On Thu, Feb 9, 2023 at 11:59 AM Gary Frost > wrote: Dan I have indeed used JVMTI (and Java) agents for this. In the case of JVMTI by keeping a std::map, and providing a JNI call to get the bytes. It works well for cases where we know the set of classes we want bytes for at JVM 'launch time' , or there is something in the class (name match? constant pool entry?) that we can trigger to prune the size of the map. Otherwise we are forced to retain a map for all classes, just in case. And of course who knew whether our JVMTI agent was the last 'actor' in the chain of possible mutators to play with the bytes. Can you expand on the use case for getting the current bytes from the runtime? As I said in the previous email, they aren't a good candidate for feeding back into the runtime due to the existing JVMTI agent process. Is this mostly for testing purposes? --Dan Hence the desire for runtime help ________________________________ From: Dan Heidinga > Sent: Thursday, February 9, 2023 4:35 PM To: Gary Frost > Cc: classfile-api-dev at openjdk.org > Subject: [External] : Re: Reflective access to bytes[] of loaded class On Thu, Feb 9, 2023 at 11:03 AM Gary Frost > wrote: I would like to make a case for adding a reflection API for getting the bytes for class loaded by the VM as part of this Classfile API. Something akin to byte[] Class.getClassfileBytes(); I've often wanted the same kind of API when doing one off tests and minor modifications so I'm very sympathetic to the request. Typically when doing more "serious" class modification, I've used the Instrumentation::retransform API [0] or written a native JVMTI agent [1] and then such an api to get the current classfile bytes isn't required. Does this match your typical use cases as well? The JVM does some interesting handling of classfile bytes in these cases - it allows non-retransform-capable agents to do a 1 time modification of the original classfile bytes and then saves those bytes away before passing them on to the retransform-capable agents. One subsequent retransform events, those saved bytes are reused and only the retransform-capable agents get a chance to modify the bytes. The reason this matters is that existing retransform-capable agents aren't expecting to see the modifications they've made present in the classfile bytes they get passed. Providing an easy api to get the currently executing bytecodes will mean that these agents will now "see" their own modifications which may result in incompatibilities. I'd be interested in feedback from existing major agent providers to see how much of a problem such a change would cause them before pursuing it unless there's a more common pattern of use I'm unaware of. [0] https://cr.openjdk.java.net/~iris/se/17/latestSpec/api/java.instrument/java/lang/instrument/Instrumentation.html#retransformClasses(java.lang.Class...) [1] https://cr.openjdk.java.net/~iris/se/17/latestSpec/specs/jvmti.html#RetransformClasses At present developers usually resort to something like clazz.getClassLoader().getResourceAsStream( clazz.getName().replace(".", "/") + ".class") Ignoring the fact that we may have just forced an expensive network fetch, to bytes that the JVM clearly already has 'squirrelled away' somewhere... For me this is hugely problematic as there is no guarentee that the bytes fetched from such a stream will match the bytes that the JVM is using for the class in question.... Java offers all sorts of opportunities (JVMTI agents, Java agents, even custom ClassLoaders) for mutating the incoming classfile's bytes. All of which the stream fetched via getResourceAsStream() completely sidesteps. See above (or the JVMTI spec) for which bytes get passed through to agents. The story is more complex than it first appears. --Dan This may well be considered 'out of bounds' for the Classfile API, but I think we should consider it, as it seems to be something that users of the Classfile API will need going forward. Gary -------------- next part -------------- An HTML attachment was scrubbed... URL: