RFR: 8241390: 'Deadlock' with VM_RedefineClasses::lock_classes()

[Coleen Phillimore]

On Tue, 15 Sep 2020 16:43:01 GMT, Daniil Titov <dtitov at openjdk.org> wrote:

> The change fixes a 'deadlock' situation in VM_RedefineClasses::lock_classes() when the current thread is in the middle
> of redefining the same class. The change is based on the fix [1] suggested in the Jira issue [2] comment.
> [1] http://cr.openjdk.java.net/~jiangli/8241390/webrev.00/
> [2] https://bugs.openjdk.java.net/browse/JDK-8241390
> 
> Testing: :jdk_instrument, tier1-tier3, and tier5 tests pass.

Changes requested by coleenp (Reviewer).

test/jdk/java/lang/instrument/MakeAgentJAR.sh line 1:

> 1: #!/bin/sh

There are tests in test/hotspot/jtreg/serviceability/jvmti/RedefineClasses that don't use shell scripts that are much
better.  Can you add this test using that framework instead?

src/hotspot/share/prims/jvmtiRedefineClasses.cpp line 159:

> 157:     if (!cls->contains(def_ik)) {
> 158:       def_ik->set_is_being_redefined(false);
> 159:     }

Ok, so adding the Klass to the thread local list for each recursion works like ref counting.  Can't think of a simpler
way to do this.  Looks good.

-------------

PR: https://git.openjdk.java.net/jdk/pull/190