RFR: 8308762: Metaspace leak with Instrumentation.retransform [v4]

[Coleen Phillimore]

On Tue, 18 Jul 2023 16:48:55 GMT, Jean-Philippe Bempel <jpbempel at openjdk.org> wrote:

>> Fix a small leak in constant pool merging during retransformation of a class. If this class has a catch block with `Throwable`, the class `Throwable` is pre-resolved in the constant pool, while all the other classes are in a unresolved state. So the constant pool merging process was considering the entry with pre-resolved class as different compared to the destination and create a new entry. We now try to consider it as equal specially for Methodref/Fieldref.
>
> Jean-Philippe Bempel has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision:
> 
>   Revert resolved class to unresolved for comparison
>   
>   remove is_unresolved_class_mismatch

Sorry I didn't see this update.  The change looks good to me.  You could do the cleanup that Serguei suggests with t2 and removing JVM_CONSTANT_Class case.  I thought maybe they should be left in case we want to generalize this compare_entry_to function someday, that's why I didn't suggest removing it.

-------------

Marked as reviewed by coleenp (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/14780#pullrequestreview-1550650275