RFR: 8308762: Metaspace leak with Instrumentation.retransform

[Jean-Philippe Bempel]

On Thu, 13 Jul 2023 14:34:38 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Fix a small leak in constant pool merging during retransformation of a class. If this class has a catch block with `Throwable`, the class `Throwable` is pre-resolved in the constant pool, while all the other classes are in a unresolved state. So the constant pool merging process was considering the entry with pre-resolved class as different compared to the destination and create a new entry. We now try to consider it as equal specially for Methodref/Fieldref.
>
> Also there is a nice test harness for class redefinition in the test/hotspot/jtreg/serviceability/jvmti/RedefineClasses tests that you might be able to use to add a test for this.

Thanks @coleenp for the hints about fixing this issue.
Please review again the new changes that include:
 - jtreg test that reproduces the issue
 - revert the unresolved class state if the current entry is resolved
 - remove `is_unresolved_class_mismatch` method as useless now.

Thanks
(sorry to the force push, old habit)

-------------

PR Comment: https://git.openjdk.org/jdk/pull/14780#issuecomment-1640580798