RFR: JDK-8311076: RedefineClasses doesn't check for ConstantPool overflow

[Leonid Mesnik]

On Wed, 7 Feb 2024 20:53:53 GMT, Alex Menkov <amenkov at openjdk.org> wrote:

> The fix adds check that merged constant pool does not overflow u2 (two-byte unsigned).
> The check is added after merging `the_class` and `scratch_class` constant pools, but before rewriting constant pool references.
> 
> testing:
>  - sanity tier1;
>  - all RedefineClasses/RetransformClasses tests:
>    - test/jdk/java/lang/instrument
>    - test/hotspot/jtreg/serviceability/jvmti/RedefineClasses
>    - test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses
>    - test/hotspot/jtreg/vmTestbase/nsk/jvmti/RetransformClasses

src/hotspot/share/prims/jvmtiRedefineClasses.cpp line 1828:

> 1826:   // ensure merged constant pool size does not overflow u2
> 1827:   if (merge_cp_length > 0xFFFF) {
> 1828:     return JVMTI_ERROR_INTERNAL;

Doesn't it make a sense to add some logging there so user could easier understand the problem?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17759#discussion_r1483803900