[11] RFR(S): 8205472: Deadlock in Kitchensink when trying to print compile queues causing timeout

[Tobias Hartmann]

Hi,

please review the following patch:
https://bugs.openjdk.java.net/browse/JDK-8205472
http://cr.openjdk.java.net/~thartmann/8205472/webrev.00/

When registering a compiled method in ciEnv::register_method() the compiler thread acquires the
MethodCompileQueue_lock and then performs a safepoint check before acquiring the Compile_lock as
well. If we enter a safepoint, the compiler thread is still holding the MethodCompileQueue_lock
while the VM thread may execute the VM_PrintCompileQueue VM operation which requires the
MethodCompileQueue_lock as well. We deadlock.

This bug is very intermittent and I was only able to reproduce it by modifying the VM to always
enter a safepoint (see comments in JIRA). The solution is to not acquire the MethodCompileQueue_lock
in CompileBroker::print_compile_queues() because we only execute that code at a safepoint anyway
(VM_PrintCompileQueue has evaluation_mode() == _safepoint).

Thanks,
Tobias