RFR: 7124710: interleaved RedefineClasses() and RetransformClasses() calls may have a problem

[Serguei Spitsyn]

On Thu, 25 Aug 2022 21:16:22 GMT, Alex Menkov <amenkov at openjdk.org> wrote:

> The problem is RedefineClasses does not update cached_class_bytes, so subsequent RetransformClasses gets obsolete class bytes (this are testcases 3-6 from the new test)
> 
> cached_class_bytes are set when an agent instruments the class from ClassFileLoadHook.
> After successful RedefineClasses it should be reset.
> The fix updates ClassFileLoadHook caller to not use old cached_class_bytes with RedefineClasses (if some agent instruments the class, new cached_class_bytes are allocated for scratch_class) and updates cached_class_bytes after successful RedefineClasses or RetransformClasses.

src/hotspot/share/classfile/klassFactory.cpp line 135:

> 133:     if (state != NULL) {
> 134:       Klass* k = state->get_class_being_redefined();
> 135:       // use cached_class_file only for RetransformClasses

More detailed comment would be nice to have here to explain in what scenario this change is needed.

-------------

PR: https://git.openjdk.org/jdk/pull/10032