RFR: 8308762: Metaspace leak with Instrumentation.retransform [v7]

[Jean-Philippe Bempel]

On Wed, 20 Sep 2023 16:13:50 GMT, Jean-Philippe Bempel <jpbempel at openjdk.org> wrote:

>> Fix a small leak in constant pool merging during retransformation of a class. If this class has a catch block with `Throwable`, the class `Throwable` is pre-resolved in the constant pool, while all the other classes are in a unresolved state. So the constant pool merging process was considering the entry with pre-resolved class as different compared to the destination and create a new entry. We now try to consider it as equal specially for Methodref/Fieldref.
>
> Jean-Philippe Bempel has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision:
> 
>  - Merge branch 'openjdk:master' into 8308762
>  - remove trailing whitespace
>  - remove now useless comment
>  - Rewrite unit test
>    
>    unresolved t2 too, cleanup JVM_CONSTANT_Class useless case
>  - Revert resolved class to unresolved for comparison
>    
>    remove is_unresolved_class_mismatch
>  - add jtreg test for leak
>  - 8308762: Metaspace leak with Instrumentation.retransform
>    
>    Fix a small leak in constant pool merging during retransformation of
>    a class. If this class has a catch block with Throwable, the class
>    Throwable is pre-resolved in the constant pool, while all the other
>    classes are in a unresolved state. So the constant pool merging
>    process was considering the entry with pre-resolved class as different
>    compared to the destination and create a new entry. We now try to
>    consider it as equal specially for Methodref/Fieldref.

Thanks!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/14780#issuecomment-1728833538