From liangchenblue at gmail.com Tue Apr 4 13:38:45 2023 From: liangchenblue at gmail.com (-) Date: Tue, 4 Apr 2023 08:38:45 -0500 Subject: Classfile API patch review request Message-ID: Since the Classfile API was added, a few patches have been submitted, fixing various bugs in the API. Since Classfile API has no reviewer closely tied to it, I wish JDK reviewers on this list can take a peek at the following patches to facilitate the bug fixes to the Classfile API: https://bugs.openjdk.org/browse/JDK-8304937 BufferedFieldBuilder.Model missing writeTo(DirectClassBuilder) https://github.com/openjdk/jdk/pull/13187 https://bugs.openjdk.org/browse/JDK-8304837 Classfile API throws IOOBE for MethodParameters attribute without parameter names https://github.com/openjdk/jdk/pull/13167 https://bugs.openjdk.org/browse/JDK-8304148 Remapping a class with Invokedynamic constant loses static bootstrap arguments https://github.com/openjdk/jdk/pull/13021 https://bugs.openjdk.org/browse/JDK-8304031 Classfile API cannot encode Primitive Class as Condy https://github.com/openjdk/jdk/pull/12996 Thanks for taking your time to review. Chen Liang From brian.goetz at oracle.com Tue Apr 4 15:48:51 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 4 Apr 2023 15:48:51 +0000 Subject: Classfile API patch review request In-Reply-To: References: Message-ID: <75B8FC48-44B0-499B-A09B-56C64B640BCF@oracle.com> I have been reviewing API and structural changes, Adam can review code changes. I went through your list: > Since the Classfile API was added, a few patches have been submitted, > fixing various bugs in the API. Since Classfile API has no reviewer > closely tied to it, I wish JDK reviewers on this list can take a peek > at the following patches to facilitate the bug fixes to the Classfile > API: > > https://bugs.openjdk.org/browse/JDK-8304937 BufferedFieldBuilder.Model > missing writeTo(DirectClassBuilder) > https://github.com/openjdk/jdk/pull/13187 Yes, this does look like a straightforward omission to me. Adam can review; please also check for similar omissions (other buffered builders, other kinds of elements.) > https://bugs.openjdk.org/browse/JDK-8304837 Classfile API throws IOOBE > for MethodParameters attribute without parameter names > https://github.com/openjdk/jdk/pull/13167 Yes, this is just a bug. JVMS 4.7.24 allows nameIndex to be zero. > https://bugs.openjdk.org/browse/JDK-8304148 Remapping a class with > Invokedynamic constant loses static bootstrap arguments > https://github.com/openjdk/jdk/pull/13021 Yes, seems like a straightforward omission. > https://bugs.openjdk.org/browse/JDK-8304031 Classfile API cannot > encode Primitive Class as Condy > https://github.com/openjdk/jdk/pull/12996 Looks reasonable. So all of these look like straightforward patches; no objections from a specification or API perspective. Please work with Adam for review, test coverage, etc. From liangchenblue at gmail.com Tue Apr 4 17:20:46 2023 From: liangchenblue at gmail.com (-) Date: Tue, 4 Apr 2023 12:20:46 -0500 Subject: Classfile API patch review request In-Reply-To: <75B8FC48-44B0-499B-A09B-56C64B640BCF@oracle.com> References: <75B8FC48-44B0-499B-A09B-56C64B640BCF@oracle.com> Message-ID: Thanks for the peek. Omissions do happen a lot, but unfortunately many of them are hardly detected until the code is actually used in production. For example, I discovered the writeTo omission when I am using the API to patch class files to remove some attributes to test core reflection behavior, and same for SirYwell for the MethodParameters nullable name issue. Adam has already reviewed all these patches. Unfortunately, since Adam is not a JDK reviewer, these patches still need a JDK reviewer's review to be eligible for integration. Brian, as a reviewer, would you do us a favor? Especially for the first 3, since it currently impedes the regular usage of Classfile API and has no workaround. Chen On Tue, Apr 4, 2023, 10:49 AM Brian Goetz wrote: > I have been reviewing API and structural changes, Adam can review code > changes. I went through your list: > > > Since the Classfile API was added, a few patches have been submitted, > > fixing various bugs in the API. Since Classfile API has no reviewer > > closely tied to it, I wish JDK reviewers on this list can take a peek > > at the following patches to facilitate the bug fixes to the Classfile > > API: > > > > https://bugs.openjdk.org/browse/JDK-8304937 BufferedFieldBuilder.Model > > missing writeTo(DirectClassBuilder) > > https://github.com/openjdk/jdk/pull/13187 > > Yes, this does look like a straightforward omission to me. Adam can > review; please also check for similar omissions (other buffered builders, > other kinds of elements.) > > > > https://bugs.openjdk.org/browse/JDK-8304837 Classfile API throws IOOBE > > for MethodParameters attribute without parameter names > > https://github.com/openjdk/jdk/pull/13167 > > Yes, this is just a bug. JVMS 4.7.24 allows nameIndex to be zero. > > > https://bugs.openjdk.org/browse/JDK-8304148 Remapping a class with > > Invokedynamic constant loses static bootstrap arguments > > https://github.com/openjdk/jdk/pull/13021 > > Yes, seems like a straightforward omission. > > > https://bugs.openjdk.org/browse/JDK-8304031 Classfile API cannot > > encode Primitive Class as Condy > > https://github.com/openjdk/jdk/pull/12996 > > Looks reasonable. > > > So all of these look like straightforward patches; no objections from a > specification or API perspective. Please work with Adam for review, test > coverage, etc. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Mon Apr 10 23:34:21 2023 From: liangchenblue at gmail.com (-) Date: Mon, 10 Apr 2023 18:34:21 -0500 Subject: Exceptions from invalid constant pool indices in the API Message-ID: When I was taking a peek at the conversion of javap to Classfile API, I found that the old API has a feature that the API might be interested in: a dedicated exception ConstantPoolException for invalid constant pool indices, caused by incorrect types or out-of-bounds. We currently throw IllegalArgumentException and IndexOutOfBoundsException in the Classfile API, so it is a bit harder to catch such invalid indices. Javap is tolerant to invalid class file formats while the reading part of Classfile API is not so much; in general, Classfile API fails fast if it encounters invalid constant pool entries; this I agree, that invalid entries render a classfile invalid, but maybe we can unify the reading exceptions from ConstantPoolReader into a common type like in the old API, so we can more easily report constant pool errors or skip invalid but optional entries. Chen Liang From brian.goetz at oracle.com Tue Apr 11 00:15:39 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 11 Apr 2023 00:15:39 +0000 Subject: Exceptions from invalid constant pool indices in the API In-Reply-To: References: Message-ID: Good suggestion! > On Apr 10, 2023, at 4:34 PM, liangchenblue at gmail.com wrote: > > When I was taking a peek at the conversion of javap to Classfile API, > I found that the old API has a feature that the API might be > interested in: a dedicated exception ConstantPoolException for invalid > constant pool indices, caused by incorrect types or out-of-bounds. We > currently throw IllegalArgumentException and IndexOutOfBoundsException > in the Classfile API, so it is a bit harder to catch such invalid > indices. > > Javap is tolerant to invalid class file formats while the reading part > of Classfile API is not so much; in general, Classfile API fails fast > if it encounters invalid constant pool entries; this I agree, that > invalid entries render a classfile invalid, but maybe we can unify the > reading exceptions from ConstantPoolReader into a common type like in > the old API, so we can more easily report constant pool errors or skip > invalid but optional entries. > > Chen Liang From adam.sotona at oracle.com Thu Apr 13 13:00:46 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Thu, 13 Apr 2023 13:00:46 +0000 Subject: Exceptions from invalid constant pool indices in the API In-Reply-To: References: Message-ID: I?ve tried to convert all CP related exceptions to a new ConstantPoolException, however IllegalArgumentException is still in the game (in javap) as a secondary exception thrown during validation of symbols. For example printing of Signature attribute or any ClassDesc can now fail with: * IndexOutOfBoundsException when the index is invalid * IllegalArgumentException when the CP entry is invalid * IllegalArgumentException when the symbol is invalid After merge of CP-related IndexOutOfBoundsException and IllegalArgumentException into ConstantPoolException we can get: * ConstantPoolException when the index is invalid * ConstantPoolException when the CP entry is invalid * IllegalArgumentException when the symbol fails validation Is it still worth to introduce a new exception when javap still must individually safeguard also IllegalArgumentExceptions ? What about to replace CP-relate IndexOutOfBoundsException with IllegalArgumentException so we will have one single type of exception to catch when class reading? Or what if we make ConstantPoolException a sub-class of IllegalArgumentException? Thanks, Adam From: classfile-api-dev on behalf of Brian Goetz Date: Tuesday, 11 April 2023 2:15 To: liangchenblue at gmail.com Cc: classfile-api-dev at openjdk.org Subject: Re: Exceptions from invalid constant pool indices in the API Good suggestion! > On Apr 10, 2023, at 4:34 PM, liangchenblue at gmail.com wrote: > > When I was taking a peek at the conversion of javap to Classfile API, > I found that the old API has a feature that the API might be > interested in: a dedicated exception ConstantPoolException for invalid > constant pool indices, caused by incorrect types or out-of-bounds. We > currently throw IllegalArgumentException and IndexOutOfBoundsException > in the Classfile API, so it is a bit harder to catch such invalid > indices. > > Javap is tolerant to invalid class file formats while the reading part > of Classfile API is not so much; in general, Classfile API fails fast > if it encounters invalid constant pool entries; this I agree, that > invalid entries render a classfile invalid, but maybe we can unify the > reading exceptions from ConstantPoolReader into a common type like in > the old API, so we can more easily report constant pool errors or skip > invalid but optional entries. > > Chen Liang -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Thu Apr 13 15:46:13 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 13 Apr 2023 15:46:13 +0000 Subject: Exceptions from invalid constant pool indices in the API In-Reply-To: References: Message-ID: <00C1A1CE-0A47-4F7D-9882-5DC5F27E9A3F@oracle.com> So, I think there are two reasons we might consider a more precise exception here. One is Chen?s concern of ?what do I catch?, and the other is simply getting a more human-readable explanation of what went wrong. Exceptions like AIOOBE and IAE are often not helpful in explaining what actually went wrong (what index? Into what? Where did the bad value come from?). Part of this is the exception type (which helps both of these audiences), and part of this is the exception message (which helps only the latter.). We should probably do a pass and make sure we have informative error text in all throw points. What do you mean ?when the symbol fails validation?? On Apr 13, 2023, at 9:00 AM, Adam Sotona > wrote: I?ve tried to convert all CP related exceptions to a new ConstantPoolException, however IllegalArgumentException is still in the game (in javap) as a secondary exception thrown during validation of symbols. For example printing of Signature attribute or any ClassDesc can now fail with: * IndexOutOfBoundsException when the index is invalid * IllegalArgumentException when the CP entry is invalid * IllegalArgumentException when the symbol is invalid After merge of CP-related IndexOutOfBoundsException and IllegalArgumentException into ConstantPoolException we can get: * ConstantPoolException when the index is invalid * ConstantPoolException when the CP entry is invalid * IllegalArgumentException when the symbol fails validation Is it still worth to introduce a new exception when javap still must individually safeguard also IllegalArgumentExceptions ? What about to replace CP-relate IndexOutOfBoundsException with IllegalArgumentException so we will have one single type of exception to catch when class reading? Or what if we make ConstantPoolException a sub-class of IllegalArgumentException? Thanks, Adam From: classfile-api-dev > on behalf of Brian Goetz > Date: Tuesday, 11 April 2023 2:15 To: liangchenblue at gmail.com > Cc: classfile-api-dev at openjdk.org > Subject: Re: Exceptions from invalid constant pool indices in the API Good suggestion! > On Apr 10, 2023, at 4:34 PM, liangchenblue at gmail.com wrote: > > When I was taking a peek at the conversion of javap to Classfile API, > I found that the old API has a feature that the API might be > interested in: a dedicated exception ConstantPoolException for invalid > constant pool indices, caused by incorrect types or out-of-bounds. We > currently throw IllegalArgumentException and IndexOutOfBoundsException > in the Classfile API, so it is a bit harder to catch such invalid > indices. > > Javap is tolerant to invalid class file formats while the reading part > of Classfile API is not so much; in general, Classfile API fails fast > if it encounters invalid constant pool entries; this I agree, that > invalid entries render a classfile invalid, but maybe we can unify the > reading exceptions from ConstantPoolReader into a common type like in > the old API, so we can more easily report constant pool errors or skip > invalid but optional entries. > > Chen Liang -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Thu Apr 13 16:18:18 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Thu, 13 Apr 2023 16:18:18 +0000 Subject: Exceptions from invalid constant pool indices in the API In-Reply-To: <00C1A1CE-0A47-4F7D-9882-5DC5F27E9A3F@oracle.com> References: <00C1A1CE-0A47-4F7D-9882-5DC5F27E9A3F@oracle.com> Message-ID: By failing symbols validation I mean when for example index points to correct ClassEntry pointing to correct Utf8Entry name, however the name is not valid class name and ClassDesc construction throws IAE. Each such case must be individually safeguarded to do not prematurely terminate the whole javap. From: Brian Goetz Date: Thursday, 13 April 2023 17:46 To: Adam Sotona Cc: liangchenblue at gmail.com , classfile-api-dev at openjdk.org Subject: Re: Exceptions from invalid constant pool indices in the API So, I think there are two reasons we might consider a more precise exception here. One is Chen?s concern of ?what do I catch?, and the other is simply getting a more human-readable explanation of what went wrong. Exceptions like AIOOBE and IAE are often not helpful in explaining what actually went wrong (what index? Into what? Where did the bad value come from?). Part of this is the exception type (which helps both of these audiences), and part of this is the exception message (which helps only the latter.). We should probably do a pass and make sure we have informative error text in all throw points. What do you mean ?when the symbol fails validation?? On Apr 13, 2023, at 9:00 AM, Adam Sotona > wrote: I?ve tried to convert all CP related exceptions to a new ConstantPoolException, however IllegalArgumentException is still in the game (in javap) as a secondary exception thrown during validation of symbols. For example printing of Signature attribute or any ClassDesc can now fail with: * IndexOutOfBoundsException when the index is invalid * IllegalArgumentException when the CP entry is invalid * IllegalArgumentException when the symbol is invalid After merge of CP-related IndexOutOfBoundsException and IllegalArgumentException into ConstantPoolException we can get: * ConstantPoolException when the index is invalid * ConstantPoolException when the CP entry is invalid * IllegalArgumentException when the symbol fails validation Is it still worth to introduce a new exception when javap still must individually safeguard also IllegalArgumentExceptions ? What about to replace CP-relate IndexOutOfBoundsException with IllegalArgumentException so we will have one single type of exception to catch when class reading? Or what if we make ConstantPoolException a sub-class of IllegalArgumentException? Thanks, Adam From: classfile-api-dev > on behalf of Brian Goetz > Date: Tuesday, 11 April 2023 2:15 To: liangchenblue at gmail.com > Cc: classfile-api-dev at openjdk.org > Subject: Re: Exceptions from invalid constant pool indices in the API Good suggestion! > On Apr 10, 2023, at 4:34 PM, liangchenblue at gmail.com wrote: > > When I was taking a peek at the conversion of javap to Classfile API, > I found that the old API has a feature that the API might be > interested in: a dedicated exception ConstantPoolException for invalid > constant pool indices, caused by incorrect types or out-of-bounds. We > currently throw IllegalArgumentException and IndexOutOfBoundsException > in the Classfile API, so it is a bit harder to catch such invalid > indices. > > Javap is tolerant to invalid class file formats while the reading part > of Classfile API is not so much; in general, Classfile API fails fast > if it encounters invalid constant pool entries; this I agree, that > invalid entries render a classfile invalid, but maybe we can unify the > reading exceptions from ConstantPoolReader into a common type like in > the old API, so we can more easily report constant pool errors or skip > invalid but optional entries. > > Chen Liang -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Thu Apr 13 17:53:08 2023 From: liangchenblue at gmail.com (-) Date: Thu, 13 Apr 2023 12:53:08 -0500 Subject: Exceptions from invalid constant pool indices in the API In-Reply-To: References: <00C1A1CE-0A47-4F7D-9882-5DC5F27E9A3F@oracle.com> Message-ID: In your example, I don't think that would constitute a valid class constant pool entry: per JVMS 4.4.1 https://docs.oracle.com/javase/specs/jvms/se20/html/jvms-4.html#jvms-4.4.1, the CONSTANT_Class_info is not valid without a valid Utf8 entry content. I think the 3rd case is essentially the same as the 2nd case you've listed, and they can both be converted from IllegalArgumentException to ConstantPoolException, as they fall in the same error category (invalid name_index) under the VM spec. They can just be distinguished by different messages. On Thu, Apr 13, 2023 at 11:18?AM Adam Sotona wrote: > > By failing symbols validation I mean when for example index points to correct ClassEntry pointing to correct Utf8Entry name, however the name is not valid class name and ClassDesc construction throws IAE. > > Each such case must be individually safeguarded to do not prematurely terminate the whole javap. > > > > From: Brian Goetz > Date: Thursday, 13 April 2023 17:46 > To: Adam Sotona > Cc: liangchenblue at gmail.com , classfile-api-dev at openjdk.org > Subject: Re: Exceptions from invalid constant pool indices in the API > > So, I think there are two reasons we might consider a more precise exception here. One is Chen?s concern of ?what do I catch?, and the other is simply getting a more human-readable explanation of what went wrong. Exceptions like AIOOBE and IAE are often not helpful in explaining what actually went wrong (what index? Into what? Where did the bad value come from?). > > > > Part of this is the exception type (which helps both of these audiences), and part of this is the exception message (which helps only the latter.). We should probably do a pass and make sure we have informative error text in all throw points. > > > > What do you mean ?when the symbol fails validation?? > > > > On Apr 13, 2023, at 9:00 AM, Adam Sotona wrote: > > > > I?ve tried to convert all CP related exceptions to a new ConstantPoolException, however IllegalArgumentException is still in the game (in javap) as a secondary exception thrown during validation of symbols. > > For example printing of Signature attribute or any ClassDesc can now fail with: > > IndexOutOfBoundsException when the index is invalid > IllegalArgumentException when the CP entry is invalid > IllegalArgumentException when the symbol is invalid > > > > After merge of CP-related IndexOutOfBoundsException and IllegalArgumentException into ConstantPoolException we can get: > > ConstantPoolException when the index is invalid > ConstantPoolException when the CP entry is invalid > IllegalArgumentException when the symbol fails validation > > > > Is it still worth to introduce a new exception when javap still must individually safeguard also IllegalArgumentExceptions ? > > > > What about to replace CP-relate IndexOutOfBoundsException with IllegalArgumentException so we will have one single type of exception to catch when class reading? > > Or what if we make ConstantPoolException a sub-class of IllegalArgumentException? > > > > Thanks, > > Adam > > > > From: classfile-api-dev on behalf of Brian Goetz > Date: Tuesday, 11 April 2023 2:15 > To: liangchenblue at gmail.com > Cc: classfile-api-dev at openjdk.org > Subject: Re: Exceptions from invalid constant pool indices in the API > > Good suggestion! > > > On Apr 10, 2023, at 4:34 PM, liangchenblue at gmail.com wrote: > > > > When I was taking a peek at the conversion of javap to Classfile API, > > I found that the old API has a feature that the API might be > > interested in: a dedicated exception ConstantPoolException for invalid > > constant pool indices, caused by incorrect types or out-of-bounds. We > > currently throw IllegalArgumentException and IndexOutOfBoundsException > > in the Classfile API, so it is a bit harder to catch such invalid > > indices. > > > > Javap is tolerant to invalid class file formats while the reading part > > of Classfile API is not so much; in general, Classfile API fails fast > > if it encounters invalid constant pool entries; this I agree, that > > invalid entries render a classfile invalid, but maybe we can unify the > > reading exceptions from ConstantPoolReader into a common type like in > > the old API, so we can more easily report constant pool errors or skip > > invalid but optional entries. > > > > Chen Liang > > From adam.sotona at oracle.com Fri Apr 14 15:42:00 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Fri, 14 Apr 2023 15:42:00 +0000 Subject: Exceptions from invalid constant pool indices in the API In-Reply-To: References: <00C1A1CE-0A47-4F7D-9882-5DC5F27E9A3F@oracle.com> Message-ID: Throwing ConstantPoolException also for symbol conversions is another option. Classfile API have to catch all the IAEs whenever the conversion happen and wrap them to ConstantPoolException. There are actually approx. 30 places across Classfile API, where the conversions to symbols happen. Some of them are in constantpool package, however many symbol conversions are spread in methods across annotations, models, attribute infos and instructions. Should all of them wrap IAE and throw ConstantPoolException instead? For example for annotations the ClassDesc is constructed from class descriptor stored in Utf8Entry, so it is not technically a ConstantPoolException. On the other side if ClassEntry::asSymbol will throw ConstantPoolException and Annotation::classSymbol will throw IAE ? that would be even more confusing. I think it is a bit overkill for the very specific use case in javap and I?m not sure it is worth the price. I propose to unify on IAE (for simplicity in major use cases) and possible refine on ConstantPoolException (or more) as sub-class(es) of IAE (for the specific cases where we might want to differentiate). Thanks, Adam From: liangchenblue at gmail.com Date: Thursday, 13 April 2023 19:53 To: Adam Sotona Cc: Brian Goetz , classfile-api-dev at openjdk.org Subject: Re: Exceptions from invalid constant pool indices in the API In your example, I don't think that would constitute a valid class constant pool entry: per JVMS 4.4.1 https://docs.oracle.com/javase/specs/jvms/se20/html/jvms-4.html#jvms-4.4.1, the CONSTANT_Class_info is not valid without a valid Utf8 entry content. I think the 3rd case is essentially the same as the 2nd case you've listed, and they can both be converted from IllegalArgumentException to ConstantPoolException, as they fall in the same error category (invalid name_index) under the VM spec. They can just be distinguished by different messages. On Thu, Apr 13, 2023 at 11:18?AM Adam Sotona wrote: > > By failing symbols validation I mean when for example index points to correct ClassEntry pointing to correct Utf8Entry name, however the name is not valid class name and ClassDesc construction throws IAE. > > Each such case must be individually safeguarded to do not prematurely terminate the whole javap. > > > > From: Brian Goetz > Date: Thursday, 13 April 2023 17:46 > To: Adam Sotona > Cc: liangchenblue at gmail.com , classfile-api-dev at openjdk.org > Subject: Re: Exceptions from invalid constant pool indices in the API > > So, I think there are two reasons we might consider a more precise exception here. One is Chen?s concern of ?what do I catch?, and the other is simply getting a more human-readable explanation of what went wrong. Exceptions like AIOOBE and IAE are often not helpful in explaining what actually went wrong (what index? Into what? Where did the bad value come from?). > > > > Part of this is the exception type (which helps both of these audiences), and part of this is the exception message (which helps only the latter.). We should probably do a pass and make sure we have informative error text in all throw points. > > > > What do you mean ?when the symbol fails validation?? > > > > On Apr 13, 2023, at 9:00 AM, Adam Sotona wrote: > > > > I?ve tried to convert all CP related exceptions to a new ConstantPoolException, however IllegalArgumentException is still in the game (in javap) as a secondary exception thrown during validation of symbols. > > For example printing of Signature attribute or any ClassDesc can now fail with: > > IndexOutOfBoundsException when the index is invalid > IllegalArgumentException when the CP entry is invalid > IllegalArgumentException when the symbol is invalid > > > > After merge of CP-related IndexOutOfBoundsException and IllegalArgumentException into ConstantPoolException we can get: > > ConstantPoolException when the index is invalid > ConstantPoolException when the CP entry is invalid > IllegalArgumentException when the symbol fails validation > > > > Is it still worth to introduce a new exception when javap still must individually safeguard also IllegalArgumentExceptions ? > > > > What about to replace CP-relate IndexOutOfBoundsException with IllegalArgumentException so we will have one single type of exception to catch when class reading? > > Or what if we make ConstantPoolException a sub-class of IllegalArgumentException? > > > > Thanks, > > Adam > > > > From: classfile-api-dev on behalf of Brian Goetz > Date: Tuesday, 11 April 2023 2:15 > To: liangchenblue at gmail.com > Cc: classfile-api-dev at openjdk.org > Subject: Re: Exceptions from invalid constant pool indices in the API > > Good suggestion! > > > On Apr 10, 2023, at 4:34 PM, liangchenblue at gmail.com wrote: > > > > When I was taking a peek at the conversion of javap to Classfile API, > > I found that the old API has a feature that the API might be > > interested in: a dedicated exception ConstantPoolException for invalid > > constant pool indices, caused by incorrect types or out-of-bounds. We > > currently throw IllegalArgumentException and IndexOutOfBoundsException > > in the Classfile API, so it is a bit harder to catch such invalid > > indices. > > > > Javap is tolerant to invalid class file formats while the reading part > > of Classfile API is not so much; in general, Classfile API fails fast > > if it encounters invalid constant pool entries; this I agree, that > > invalid entries render a classfile invalid, but maybe we can unify the > > reading exceptions from ConstantPoolReader into a common type like in > > the old API, so we can more easily report constant pool errors or skip > > invalid but optional entries. > > > > Chen Liang > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Fri Apr 14 17:35:50 2023 From: liangchenblue at gmail.com (-) Date: Fri, 14 Apr 2023 12:35:50 -0500 Subject: Exceptions from invalid constant pool indices in the API In-Reply-To: References: <00C1A1CE-0A47-4F7D-9882-5DC5F27E9A3F@oracle.com> Message-ID: You are right. Making a ConstantPoolException also argues for making an AttributeException, etc. which will explode the hierarchy with little gain. The real purpose of us is to not fail when we encounter an invalid optional attribute in the class file; to detect how a failure happens programmatically is a bit unrealistic, as we usually can't fix it programmatically. With this purpose in mind, your proposal of merging constant pool index out of bounds exceptions into IAE or its subclasses is a right step forward. I think whether we have a custom subclass of IAE depends on if it makes catching errors in optional attribute definitions (say, broken MethodParameters or RuntimeVisibleAnnotations that doesn't fail execution in JVM unless inspected by reflection) easier. Thanks, Chen Liang On Fri, Apr 14, 2023 at 10:42?AM Adam Sotona wrote: > > Throwing ConstantPoolException also for symbol conversions is another option. Classfile API have to catch all the IAEs whenever the conversion happen and wrap them to ConstantPoolException. > > There are actually approx. 30 places across Classfile API, where the conversions to symbols happen. > > Some of them are in constantpool package, however many symbol conversions are spread in methods across annotations, models, attribute infos and instructions. > > Should all of them wrap IAE and throw ConstantPoolException instead? For example for annotations the ClassDesc is constructed from class descriptor stored in Utf8Entry, so it is not technically a ConstantPoolException. > > On the other side if ClassEntry::asSymbol will throw ConstantPoolException and Annotation::classSymbol will throw IAE ? that would be even more confusing. > > > > I think it is a bit overkill for the very specific use case in javap and I?m not sure it is worth the price. > > I propose to unify on IAE (for simplicity in major use cases) and possible refine on ConstantPoolException (or more) as sub-class(es) of IAE (for the specific cases where we might want to differentiate). > > > > Thanks, > > Adam > > > > > > > > From: liangchenblue at gmail.com > Date: Thursday, 13 April 2023 19:53 > To: Adam Sotona > Cc: Brian Goetz , classfile-api-dev at openjdk.org > Subject: Re: Exceptions from invalid constant pool indices in the API > > In your example, I don't think that would constitute a valid class > constant pool entry: per JVMS 4.4.1 > https://docs.oracle.com/javase/specs/jvms/se20/html/jvms-4.html#jvms-4.4.1, > the CONSTANT_Class_info is not valid without a valid Utf8 entry > content. I think the 3rd case is essentially the same as the 2nd case > you've listed, and they can both be converted from > IllegalArgumentException to ConstantPoolException, as they fall in the > same error category (invalid name_index) under the VM spec. They can > just be distinguished by different messages. > > On Thu, Apr 13, 2023 at 11:18?AM Adam Sotona wrote: > > > > By failing symbols validation I mean when for example index points to correct ClassEntry pointing to correct Utf8Entry name, however the name is not valid class name and ClassDesc construction throws IAE. > > > > Each such case must be individually safeguarded to do not prematurely terminate the whole javap. > > > > > > > > From: Brian Goetz > > Date: Thursday, 13 April 2023 17:46 > > To: Adam Sotona > > Cc: liangchenblue at gmail.com , classfile-api-dev at openjdk.org > > Subject: Re: Exceptions from invalid constant pool indices in the API > > > > So, I think there are two reasons we might consider a more precise exception here. One is Chen?s concern of ?what do I catch?, and the other is simply getting a more human-readable explanation of what went wrong. Exceptions like AIOOBE and IAE are often not helpful in explaining what actually went wrong (what index? Into what? Where did the bad value come from?). > > > > > > > > Part of this is the exception type (which helps both of these audiences), and part of this is the exception message (which helps only the latter.). We should probably do a pass and make sure we have informative error text in all throw points. > > > > > > > > What do you mean ?when the symbol fails validation?? > > > > > > > > On Apr 13, 2023, at 9:00 AM, Adam Sotona wrote: > > > > > > > > I?ve tried to convert all CP related exceptions to a new ConstantPoolException, however IllegalArgumentException is still in the game (in javap) as a secondary exception thrown during validation of symbols. > > > > For example printing of Signature attribute or any ClassDesc can now fail with: > > > > IndexOutOfBoundsException when the index is invalid > > IllegalArgumentException when the CP entry is invalid > > IllegalArgumentException when the symbol is invalid > > > > > > > > After merge of CP-related IndexOutOfBoundsException and IllegalArgumentException into ConstantPoolException we can get: > > > > ConstantPoolException when the index is invalid > > ConstantPoolException when the CP entry is invalid > > IllegalArgumentException when the symbol fails validation > > > > > > > > Is it still worth to introduce a new exception when javap still must individually safeguard also IllegalArgumentExceptions ? > > > > > > > > What about to replace CP-relate IndexOutOfBoundsException with IllegalArgumentException so we will have one single type of exception to catch when class reading? > > > > Or what if we make ConstantPoolException a sub-class of IllegalArgumentException? > > > > > > > > Thanks, > > > > Adam > > > > > > > > From: classfile-api-dev on behalf of Brian Goetz > > Date: Tuesday, 11 April 2023 2:15 > > To: liangchenblue at gmail.com > > Cc: classfile-api-dev at openjdk.org > > Subject: Re: Exceptions from invalid constant pool indices in the API > > > > Good suggestion! > > > > > On Apr 10, 2023, at 4:34 PM, liangchenblue at gmail.com wrote: > > > > > > When I was taking a peek at the conversion of javap to Classfile API, > > > I found that the old API has a feature that the API might be > > > interested in: a dedicated exception ConstantPoolException for invalid > > > constant pool indices, caused by incorrect types or out-of-bounds. We > > > currently throw IllegalArgumentException and IndexOutOfBoundsException > > > in the Classfile API, so it is a bit harder to catch such invalid > > > indices. > > > > > > Javap is tolerant to invalid class file formats while the reading part > > > of Classfile API is not so much; in general, Classfile API fails fast > > > if it encounters invalid constant pool entries; this I agree, that > > > invalid entries render a classfile invalid, but maybe we can unify the > > > reading exceptions from ConstantPoolReader into a common type like in > > > the old API, so we can more easily report constant pool errors or skip > > > invalid but optional entries. > > > > > > Chen Liang > > > > From brian.goetz at oracle.com Fri Apr 14 18:13:32 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 14 Apr 2023 18:13:32 +0000 Subject: Exceptions from invalid constant pool indices in the API In-Reply-To: References: <00C1A1CE-0A47-4F7D-9882-5DC5F27E9A3F@oracle.com> Message-ID: This seems a pragmatic direction to me. On Apr 14, 2023, at 11:42 AM, Adam Sotona > wrote: Throwing ConstantPoolException also for symbol conversions is another option. Classfile API have to catch all the IAEs whenever the conversion happen and wrap them to ConstantPoolException. There are actually approx. 30 places across Classfile API, where the conversions to symbols happen. Some of them are in constantpool package, however many symbol conversions are spread in methods across annotations, models, attribute infos and instructions. Should all of them wrap IAE and throw ConstantPoolException instead? For example for annotations the ClassDesc is constructed from class descriptor stored in Utf8Entry, so it is not technically a ConstantPoolException. On the other side if ClassEntry::asSymbol will throw ConstantPoolException and Annotation::classSymbol will throw IAE ? that would be even more confusing. I think it is a bit overkill for the very specific use case in javap and I?m not sure it is worth the price. I propose to unify on IAE (for simplicity in major use cases) and possible refine on ConstantPoolException (or more) as sub-class(es) of IAE (for the specific cases where we might want to differentiate). Thanks, Adam From: liangchenblue at gmail.com > Date: Thursday, 13 April 2023 19:53 To: Adam Sotona > Cc: Brian Goetz >, classfile-api-dev at openjdk.org > Subject: Re: Exceptions from invalid constant pool indices in the API In your example, I don't think that would constitute a valid class constant pool entry: per JVMS 4.4.1 https://docs.oracle.com/javase/specs/jvms/se20/html/jvms-4.html#jvms-4.4.1, the CONSTANT_Class_info is not valid without a valid Utf8 entry content. I think the 3rd case is essentially the same as the 2nd case you've listed, and they can both be converted from IllegalArgumentException to ConstantPoolException, as they fall in the same error category (invalid name_index) under the VM spec. They can just be distinguished by different messages. On Thu, Apr 13, 2023 at 11:18?AM Adam Sotona > wrote: > > By failing symbols validation I mean when for example index points to correct ClassEntry pointing to correct Utf8Entry name, however the name is not valid class name and ClassDesc construction throws IAE. > > Each such case must be individually safeguarded to do not prematurely terminate the whole javap. > > > > From: Brian Goetz > > Date: Thursday, 13 April 2023 17:46 > To: Adam Sotona > > Cc: liangchenblue at gmail.com >, classfile-api-dev at openjdk.org > > Subject: Re: Exceptions from invalid constant pool indices in the API > > So, I think there are two reasons we might consider a more precise exception here. One is Chen?s concern of ?what do I catch?, and the other is simply getting a more human-readable explanation of what went wrong. Exceptions like AIOOBE and IAE are often not helpful in explaining what actually went wrong (what index? Into what? Where did the bad value come from?). > > > > Part of this is the exception type (which helps both of these audiences), and part of this is the exception message (which helps only the latter.). We should probably do a pass and make sure we have informative error text in all throw points. > > > > What do you mean ?when the symbol fails validation?? > > > > On Apr 13, 2023, at 9:00 AM, Adam Sotona > wrote: > > > > I?ve tried to convert all CP related exceptions to a new ConstantPoolException, however IllegalArgumentException is still in the game (in javap) as a secondary exception thrown during validation of symbols. > > For example printing of Signature attribute or any ClassDesc can now fail with: > > IndexOutOfBoundsException when the index is invalid > IllegalArgumentException when the CP entry is invalid > IllegalArgumentException when the symbol is invalid > > > > After merge of CP-related IndexOutOfBoundsException and IllegalArgumentException into ConstantPoolException we can get: > > ConstantPoolException when the index is invalid > ConstantPoolException when the CP entry is invalid > IllegalArgumentException when the symbol fails validation > > > > Is it still worth to introduce a new exception when javap still must individually safeguard also IllegalArgumentExceptions ? > > > > What about to replace CP-relate IndexOutOfBoundsException with IllegalArgumentException so we will have one single type of exception to catch when class reading? > > Or what if we make ConstantPoolException a sub-class of IllegalArgumentException? > > > > Thanks, > > Adam > > > > From: classfile-api-dev > on behalf of Brian Goetz > > Date: Tuesday, 11 April 2023 2:15 > To: liangchenblue at gmail.com > > Cc: classfile-api-dev at openjdk.org > > Subject: Re: Exceptions from invalid constant pool indices in the API > > Good suggestion! > > > On Apr 10, 2023, at 4:34 PM, liangchenblue at gmail.com wrote: > > > > When I was taking a peek at the conversion of javap to Classfile API, > > I found that the old API has a feature that the API might be > > interested in: a dedicated exception ConstantPoolException for invalid > > constant pool indices, caused by incorrect types or out-of-bounds. We > > currently throw IllegalArgumentException and IndexOutOfBoundsException > > in the Classfile API, so it is a bit harder to catch such invalid > > indices. > > > > Javap is tolerant to invalid class file formats while the reading part > > of Classfile API is not so much; in general, Classfile API fails fast > > if it encounters invalid constant pool entries; this I agree, that > > invalid entries render a classfile invalid, but maybe we can unify the > > reading exceptions from ConstantPoolReader into a common type like in > > the old API, so we can more easily report constant pool errors or skip > > invalid but optional entries. > > > > Chen Liang > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.sotona at oracle.com Wed Apr 19 13:29:50 2023 From: adam.sotona at oracle.com (Adam Sotona) Date: Wed, 19 Apr 2023 13:29:50 +0000 Subject: Classfile API support for JSR and RET instructions Message-ID: Hi, Work on javap conversion from ASM to Classfile API and recent bug (JDK-8305990) discovered in StripJavaDebugAttributesPlugin brought attention to missing JSR and RET instructions support in Classfile API. I?m proposing following changes to Classfile API: * Drop Opcode.Kind.UNSUPPORTED and add Opcode.Kind.DISCONTINUED_JSR and DISCONTINUED_RET * Add DiscontinuedInstruction interface with inner JsrInstruction and RetInstruction with standard factory methods * CodeImpl will parse and stream these instructions when present in the Code attribute * Do not add any new conveniency methods to CodeBuilder, the only way to build a code with these instructions will be cob.with(DiscontinuedInstruction. JsrInstruction.of(?)) and similar for RetInstruction * Fix DirectCodeBuilder so it invokes StackMapGenerator only for class file version >= 51.0 (BufWriterImpl must hold class version for that purpose) * Fix StackMapGenerator error message when these instruction appear in during generation * Implement fallback to jump target inflation in CodeImpl when the StackMapTable attribute is missing and class file version is < 51.0 * Extend tests to verify JSR and RET instructions are correctly generated, transformed and reported as error (based on class file version). Please review and comment this proposal under following pull request: https://github.com/openjdk/jdk/pull/13478 Next step is to implement simple maxStack and maxLocals counter into DirectCodeBuilder for the cases where StackMapGenerator is not involved. Actual default to maxStack = maxLocals = 255 is not correct. Thanks, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From liangchenblue at gmail.com Wed Apr 19 14:13:30 2023 From: liangchenblue at gmail.com (-) Date: Wed, 19 Apr 2023 09:13:30 -0500 Subject: Classfile API support for JSR and RET instructions In-Reply-To: References: Message-ID: Thanks for your great work, Adam! I took a peek at the patch. Looks great to me. I believe this should suffice to handle transforming all older bytecode without upgrading them. In the future, we might have a code transform in the components that migrates code from jsr/ret to modern StackMapTable-friendly bytecode as well. Chen Liang On Wed, Apr 19, 2023 at 8:30?AM Adam Sotona wrote: > > Hi, > > Work on javap conversion from ASM to Classfile API and recent bug (JDK-8305990) discovered in StripJavaDebugAttributesPlugin brought attention to missing JSR and RET instructions support in Classfile API. > > > > I?m proposing following changes to Classfile API: > > Drop Opcode.Kind.UNSUPPORTED and add Opcode.Kind.DISCONTINUED_JSR and DISCONTINUED_RET > Add DiscontinuedInstruction interface with inner JsrInstruction and RetInstruction with standard factory methods > CodeImpl will parse and stream these instructions when present in the Code attribute > Do not add any new conveniency methods to CodeBuilder, the only way to build a code with these instructions will be cob.with(DiscontinuedInstruction. JsrInstruction.of(?)) and similar for RetInstruction > Fix DirectCodeBuilder so it invokes StackMapGenerator only for class file version >= 51.0 (BufWriterImpl must hold class version for that purpose) > Fix StackMapGenerator error message when these instruction appear in during generation > Implement fallback to jump target inflation in CodeImpl when the StackMapTable attribute is missing and class file version is < 51.0 > Extend tests to verify JSR and RET instructions are correctly generated, transformed and reported as error (based on class file version). > > > > Please review and comment this proposal under following pull request: > > https://github.com/openjdk/jdk/pull/13478 > > > > Next step is to implement simple maxStack and maxLocals counter into DirectCodeBuilder for the cases where StackMapGenerator is not involved. Actual default to maxStack = maxLocals = 255 is not correct. > > > > Thanks, > > Adam From brian.goetz at oracle.com Wed Apr 19 15:21:52 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 19 Apr 2023 11:21:52 -0400 Subject: Classfile API support for JSR and RET instructions In-Reply-To: References: Message-ID: <95dd4ef3-5ca2-7f39-7b81-2f715405ace7@oracle.com> Overall this is a good plan. There's a small gap regarding control of stack map generation. The stack map attribute was introduced in classfile version 50; JSR and RET were eliminated in classfile version 51.? So there's a small window where stackmaps and JSR/RET can co-exist. Our strategy with stackmaps is to generate based on an option, whose default is to generate.? You are right that we should refine this so that we never generate stack maps for classfiles < 50 because the stackmap attribute wasn't defined until then.? But that leaves us with a quandary about what to do for 51; in your proposed changes below, there is no way a user could get a stack map table even if they wanted one and didn't use JSR/RET.? Instead, I think we should: ?- Generate a stackmap if the option is set and classfile version >= 50 ?- If JSR/RET is generated in classfile > 51, throw from CodeBuilder ?- If stackmap generation encounters JSR/RET in classfile 50, throw from StackMapGenerator On 4/19/2023 9:29 AM, Adam Sotona wrote: > > Hi, > > Work on javap conversion from ASM to Classfile API and recent bug > (JDK-8305990 ) discovered > in StripJavaDebugAttributesPlugin brought attention to missing JSR and > RET instructions support in Classfile API. > > I?m proposing following changes to Classfile API: > > * Drop Opcode.Kind.UNSUPPORTED and add Opcode.Kind.DISCONTINUED_JSR > and DISCONTINUED_RET > * Add DiscontinuedInstruction interface with inner JsrInstruction > and RetInstruction with standard factory methods > * CodeImpl will parse and stream these instructions when present in > the Code attribute > * Do not add any new conveniency methods to CodeBuilder, the only > way to build a code with these instructions will be > cob.with(DiscontinuedInstruction. JsrInstruction.of(?)) and > similar for RetInstruction > * Fix DirectCodeBuilder so it invokes StackMapGenerator only for > class file version >= 51.0 (BufWriterImpl must hold class version > for that purpose) > * Fix ?StackMapGenerator error message when these instruction appear > in during generation > * Implement fallback to jump target inflation in CodeImpl when the > StackMapTable attribute is missing and class file version is < 51.0 > * Extend tests to verify JSR and RET instructions are correctly > generated, transformed and reported as error (based on class file > version). > > Please review and comment this proposal under following pull request: > > https://github.com/openjdk/jdk/pull/13478 > > Next step is to implement simple maxStack and maxLocals counter into > DirectCodeBuilder for the cases where StackMapGenerator is not > involved. Actual default to maxStack = maxLocals = 255 is not correct. > > Thanks, > > Adam > -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Wed Apr 19 15:43:57 2023 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 19 Apr 2023 17:43:57 +0200 (CEST) Subject: Classfile API support for JSR and RET instructions In-Reply-To: <95dd4ef3-5ca2-7f39-7b81-2f715405ace7@oracle.com> References: <95dd4ef3-5ca2-7f39-7b81-2f715405ace7@oracle.com> Message-ID: <1618176872.39220763.1681919037278.JavaMail.zimbra@univ-eiffel.fr> > From: "Brian Goetz" > To: "Adam Sotona" , "classfile-api-dev" > > Sent: Wednesday, April 19, 2023 5:21:52 PM > Subject: Re: Classfile API support for JSR and RET instructions > Overall this is a good plan. There's a small gap regarding control of stack map > generation. > The stack map attribute was introduced in classfile version 50; JSR and RET were > eliminated in classfile version 51. So there's a small window where stackmaps > and JSR/RET can co-exist. > Our strategy with stackmaps is to generate based on an option, whose default is > to generate. You are right that we should refine this so that we never generate > stack maps for classfiles < 50 because the stackmap attribute wasn't defined > until then. Why ? StackMapTable is an attribute, it can be present in class file version < 50 and it will be ignored by the VM. Generating stack map is quite costly so generating them beforehand even if the class file version is < 50 is a trick to make agents that upgrade the class file version at runtime faster (because for most transformations, patching the stack maps is linear). R?mi > On 4/19/2023 9:29 AM, Adam Sotona wrote: >> Hi, >> Work on javap conversion from ASM to Classfile API and recent bug ( [ >> https://bugs.openjdk.org/browse/JDK-8305990 | JDK-8305990 ] ) discovered in >> StripJavaDebugAttributesPlugin brought attention to missing JSR and RET >> instructions support in Classfile API. >> I?m proposing following changes to Classfile API: >> * Drop Opcode.Kind.UNSUPPORTED and add Opcode.Kind.DISCONTINUED_JSR and >> DISCONTINUED_RET >> * Add DiscontinuedInstruction interface with inner JsrInstruction and >> RetInstruction with standard factory methods >> * CodeImpl will parse and stream these instructions when present in the Code >> attribute >> * Do not add any new conveniency methods to CodeBuilder, the only way to build a >> code with these instructions will be cob.with(DiscontinuedInstruction. >> JsrInstruction.of(?)) and similar for RetInstruction >> * Fix DirectCodeBuilder so it invokes StackMapGenerator only for class file >> version >= 51.0 (BufWriterImpl must hold class version for that purpose) >> * Fix StackMapGenerator error message when these instruction appear in during >> generation >> * Implement fallback to jump target inflation in CodeImpl when the StackMapTable >> attribute is missing and class file version is < 51.0 >> * Extend tests to verify JSR and RET instructions are correctly generated, >> transformed and reported as error (based on class file version). >> Please review and comment this proposal under following pull request: >> [ https://github.com/openjdk/jdk/pull/13478 | >> https://github.com/openjdk/jdk/pull/13478 ] >> Next step is to implement simple maxStack and maxLocals counter into >> DirectCodeBuilder for the cases where StackMapGenerator is not involved. Actual >> default to maxStack = maxLocals = 255 is not correct. >> Thanks, >> Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Wed Apr 19 17:33:00 2023 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 19 Apr 2023 13:33:00 -0400 Subject: [External] : Re: ClassHierarchyResolver using Reflection information In-Reply-To: References: <0883d896-3723-f15b-2573-6a7316feebcf@oracle.com> <248a1a23-4bcd-076b-5124-5e418b3292f2@oracle.com> <17d29737-43f9-2713-da67-459ab59f1cc8@oracle.com> Message-ID: <67351be6-43e3-e8fd-6994-622250c6ecbd@oracle.com> After the below is integrated, I'd like to propose the following API changes. When I reviewed the code for cached hierarchy resolvers, I was concerned that the cache is hidden and shared across all users, meaning it will continue to fill up over time.? The change outlined below provide half the story, which is allowing more control over the locus of caching.? The other half is making that control map easily to abstractions that users deal with. What I'd like to do is move the static methods in Classfile onto some sort of ClassfileReaderWriter object (needs a better name.)? Then, the CRW becomes a sensible locus of caching, as well as a place to hang options.? (Yet again, something that seemingly made sense to be a static method at the beginning, has gotten complicated enough that it bites us.) interface ClassfileReaderWriter { ??? static CRW of() { ... } ??? static CRW of(Collection