RFR: 8311557: [JVMCI] deadlock with JVMTI thread suspension

[Tom Rodriguez]

On Fri, 7 Jul 2023 06:13:21 GMT, Tom Rodriguez <never at openjdk.org> wrote:

> Java based JVMCI compiler threads are more like normal Java threads so they aren't `hidden_from_external_view` like the native compilers.  This can leak to deadlocks if you use JVMTI to suspend all threads since this will block the compiler queue and can block execution if background compilation is disabled.  It's reasonable to treat libgraal threads like native threads in this regard.  Making jargraal threads hidden too would interfere with using profiling and debugging tool on them so I've left that alone but it might be worth changing the JVMTI suspend and resume functions to explicitly skip compiler threads as well.

Thanks for the review.  Part of the problem is that it's not clear what the relationship between things like `can_call_java` and `hidden_from_external_view` is and whether making some of these methods safer for JVMCI threads will confuse users of JVMTI.  There will likely be some follow up issues related to this as we navigate these distinctions.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/14799#issuecomment-1644458471