RFR: 7124710: interleaved RedefineClasses() and RetransformClasses() calls may have a problem [v7]

[Daniel D. Daugherty]

On Thu, 6 Oct 2022 17:44:22 GMT, Alex Menkov <amenkov at openjdk.org> wrote:

>> The problem is RedefineClasses does not update cached_class_bytes, so subsequent RetransformClasses gets obsolete class bytes (this are testcases 3-6 from the new test)
>> 
>> cached_class_bytes are set when an agent instruments the class from ClassFileLoadHook.
>> After successful RedefineClasses it should be reset.
>> The fix updates ClassFileLoadHook caller to not use old cached_class_bytes with RedefineClasses (if some agent instruments the class, new cached_class_bytes are allocated for scratch_class) and updates cached_class_bytes after successful RedefineClasses or RetransformClasses.
>
> Alex Menkov has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fixed compilation error on linux

Thanks for making the minor changes.

test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineRetransform/libRedefineRetransform.cpp line 48:

> 46: /*
> 47:  * Helper class for data exchange between RedefineClasses/RetransformClasses and
> 48:  * ClassFileLoadHook callback (saves class bytes passed to CFLH,

typo: s/passed to/to be passed to/

test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineRetransform/libRedefineRetransform.cpp line 49:

> 47:  * Helper class for data exchange between RedefineClasses/RetransformClasses and
> 48:  * ClassFileLoadHook callback (saves class bytes passed to CFLH,
> 49:  * allows to set new class bytes to return from CFLH).

typo: s/allows to set new/allows setting new/

-------------

Marked as reviewed by dcubed (Reviewer).

PR: https://git.openjdk.org/jdk/pull/10032