RFR: 8241390: 'Deadlock' with VM_RedefineClasses::lock_classes()

[Serguei Spitsyn]

On Tue, 15 Sep 2020 20:02:33 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:

>> The change fixes a 'deadlock' situation in VM_RedefineClasses::lock_classes() when the current thread is in the middle
>> of redefining the same class. The change is based on the fix [1] suggested in the Jira issue [2] comment.
>> [1] http://cr.openjdk.java.net/~jiangli/8241390/webrev.00/
>> [2] https://bugs.openjdk.java.net/browse/JDK-8241390
>> 
>> Testing: :jdk_instrument, tier1-tier3, and tier5 tests pass.
>
> src/hotspot/share/prims/jvmtiRedefineClasses.cpp line 159:
> 
>> 157:     if (!cls->contains(def_ik)) {
>> 158:       def_ik->set_is_being_redefined(false);
>> 159:     }
> 
> Ok, so adding the Klass to the thread local list for each recursion works like ref counting.  Can't think of a simpler
> way to do this.  Looks good.

Yes, the same class can be pushed to the list multiple times (not more than once by each recursive redefinition). It'd
make sense to add a comment about it as it is not obvious.

> test/jdk/java/lang/instrument/MakeAgentJAR.sh line 1:
> 
>> 1: #!/bin/sh
> 
> There are tests in test/hotspot/jtreg/serviceability/jvmti/RedefineClasses that don't use shell scripts that are much
> better.  Can you add this test using that framework instead?

I'm second for this suggestion from Coleen to get rid of the shell script in the test.

-------------

PR: https://git.openjdk.java.net/jdk/pull/190