RFR: 8274298: JFR Thread Sampler thread must not acquire malloc lock after suspending a thread because of possible deadlock

[Erik Gahlin]

On Sun, 17 Oct 2021 13:01:18 GMT, Markus Grönlund <mgronlun at openjdk.org> wrote:

> Greetings,
> 
> [JDK-8233705](https://bugs.openjdk.java.net/browse/JDK-8233705) introduced a means to only iterate the incrementally tagged Klasses, instead of the entire set of Klasses in the JVM, as part of serializing metadata information. The newly tagged Klasses are enqueued onto a thread-local buffer using a load barrier. If the buffer runs out of space, a new buffer is accommodated, either from a free list or via a new allocation.
> 
> This is problematic for the JfrThreadSampler because the thread it has suspended could hold the malloc lock. For example, this could be the case if the suspended thread was in the process of deoptimization. The Deoptimization::UnrollBlock is a CHeapObj, and the vframes arrays use the NEW_C_HEAP_ARRAY macro.
> 
> The solution to this problem is to explicitly monitor the size of the thread-local buffer of the JfrThreadSampler thread, and pre-emptively renew it before thread suspension.
> 
> Testing: jdk_jfr, stress testing
> 
> Thanks
> Markus

Marked as reviewed by egahlin (Reviewer).

-------------

PR: https://git.openjdk.java.net/jdk/pull/5977