RFR: 8277213: CompileTask_lock is acquired out of order with MethodCompileQueue_lock

[Rickard Bäckman]

On Thu, 18 Nov 2021 09:09:30 GMT, Tobias Hartmann <thartmann at openjdk.org> wrote:

> In the rare case that the compiler threads fail during initialization or the code cache is full and flushing is disabled, we completely disable JIT compilation and shutdown the compiler runtime:
> https://github.com/openjdk/jdk/blob/2f4b5405f0b53782f3ed5274f68b31eb968efb6d/src/hotspot/share/compiler/compileBroker.cpp#L1813-L1817
> 
> In the process, we free all compiler queues and notify potentially waiting compiler threads via the `CompileTask::lock()`:
> https://github.com/openjdk/jdk/blob/2f4b5405f0b53782f3ed5274f68b31eb968efb6d/src/hotspot/share/compiler/compileBroker.cpp#L397-L408
> 
> The problem is that since [JDK-8273917](https://bugs.openjdk.java.net/browse/JDK-8273917) (see [commit](https://github.com/openjdk/jdk/commit/b8af6a9bfb28aaf0fea0cfdaba13236dc8cbaa3a)), the rank of `CompileTask_lock` is `Mutex::safepoint` which is equal to the rank of `MethodCompileQueue_lock` which we are already holding because we modify the compile queue.
> 
> I propose to fix this by modifying the rank of the `CompileTask_lock` similar to what is done for other locks:
> https://github.com/openjdk/jdk/blob/2f4b5405f0b53782f3ed5274f68b31eb968efb6d/src/hotspot/share/oops/methodData.cpp#L1211-L1212
> 
> The test that triggered this will be added with [PR 6364](https://git.openjdk.java.net/jdk/pull/6364). I verified that it now passes.
> 
> Thanks,
> Tobias

Looks good.

-------------

Marked as reviewed by rbackman (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/6449