RFR: 8277444: Race condition on Instrumentation.retransformClasses() and class linking [v3]

[Evgeny Astigeevich]

On Wed, 27 Aug 2025 19:28:33 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> Evgeny Astigeevich has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Symplify comments; Get JavaThread::current in variable
>
> https://github.com/openjdk/jdk/pull/26971
> 
> We do link the class during redefinition, when we load the new classfile version.  For the retransformer, we read the classfile from a potentially unlinked class, then link the class after when we do the redefinition/retransformation.  To avoid this race, we can link the bytecodes first, I think then we would not have to redo the linking.  This change is relatively small as well and does not rely on the implementation of what we lock when we link and initialize the class.  I think most of the time, the class is already linked at this stage anyway so it wouldn't have any effect on performance.  I didn't try this on your test yet.
> 
> This isn't that clear.  Now we do:
> retransform class -> class reconstitutor to get current classfile version -> VM_RedefineClasses -> load class version with this set of bytecodes -> link bytecodes -> etc.
> 
> My change:
> retransform class -> link class -> class reconstitutor to get current classfile version -> etc.

Hi @coleenp,

Thank you for https://github.com/openjdk/jdk/pull/26971.

Don't you mind if we follow the following plan:
1. I pull changes from https://github.com/openjdk/jdk/pull/26971 which fix `RetransformClasses` into this PR.
2. I create a new JBS issues to cover `GetBytecodes`.
3. A separate PR fixing `GetBytecodes` is created.

I'd like to have changes for `RetransformClasses` and `GetBytecodes` be in different PRs to simplify backporting processes. IMO fixing `GetBytecodes` is not critical because it is not available for java agents. Only JVMTI agents can use it. We have not seen any bug reports regarding `GetBytecodes`. Regarding `RetransformClasses` we've got reports of crashing 11, 17 and 21 from different customers.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26863#issuecomment-3229563219