RFR: 8277444: Race condition on Instrumentation.retransformClasses() and class linking [v3]

[Coleen Phillimore]

On Tue, 26 Aug 2025 13:39:14 GMT, Evgeny Astigeevich <eastigeevich at openjdk.org> wrote:

>> There is a race between `JvmtiClassFileReconstituter::copy_bytecodes` and `InstanceKlass::link_class_impl`.  `InstanceKlass::link_class_impl` can be rewriting bytecodes. `JvmtiClassFileReconstituter::copy_bytecodes` will not restore them to the original ones because the flag `rewritten` is `false`. This will result in invalid bytecode.
>> 
>> This PR adds a lock (`init_lock`) to the `copy_bytecodes` method to prevent reading bytecodes while they are being rewritten during class linking.
>> 
>> Tested fastdebug and release builds: Linux x86_64 and arm64
>> - The reproducer from JDK-8277444 passed.
>> - Tier1 - tier3 passed.
>
> Evgeny Astigeevich has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Symplify comments; Get JavaThread::current in variable

https://github.com/openjdk/jdk/pull/26971

We do link the class during redefinition, when we load the new classfile version.  For the retransformer, we read the classfile from a potentially unlinked class, then link the class after when we do the redefinition/retransformation.  To avoid this race, we can link the bytecodes first, I think then we would not have to redo the linking.  This change is relatively small as well and does not rely on the implementation of what we lock when we link and initialize the class.  I think most of the time, the class is already linked at this stage anyway so it wouldn't have any effect on performance.  I didn't try this on your test yet.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26863#issuecomment-3229490287