RFR: 8273608: Deadlock when jcmd of OnError attaches to itself

[Thomas Stuefe]

On Thu, 7 Oct 2021 01:57:39 GMT, David Holmes <david.holmes at oracle.com> wrote:

> 1. `HeapDumpOnOutOfMemoryError` can't cover OOME from Java world and thread exhaustion.

What do you mean with "java world"? About thread exhaustion, you are right. We tried to fix this but run into "works as designed" arguments. We do this downstream in SapMachine though, so worst thing you could just take over your patch for Coretto too (see https://github.com/SAP/SapMachine/blob/d574d26282fd2e67aa242de176bbe6cdc1e41205/src/hotspot/share/prims/jvm.cpp#L2920). 

> 2. I admit that unlocking mutexes is hacky.  To be fair, this problem predates this patch.   There is [Jfr::on_vm_shutdown(true)](https://github.com/openjdk/jdk/blob/master/src/hotspot/share/utilities/vmError.cpp#L1560) in VMError::report_and_die. if you use ""-XX:+FlightRecorder", "-XX:StartFlightRecording:dumponexit=true", this will tamper the coredump because it unlocks all holding mutexes.

Yes, and arguably they should not do that either.

-------------

PR: https://git.openjdk.java.net/jdk/pull/5590