RFR: 8298377: JfrVframeStream causes deadlocks in ZGC

[Erik Österlund]

On Thu, 8 Dec 2022 11:23:57 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

> The JfrVFrameStream is used while generating stack traces for events. One of the events are the ZPage allocation event. This event is sometimes sent when ZGC is relocating. The current implementation of JfrVFrameStream uses WalkContinuation::include, which causes JFR to walk the continuation and perform GC barriers. This is problematic, since ZGC has a requirement that we never perform load barriers while running the relocation code. If we do, we might end up performing other reloctions from the the relocation code, and in some cases that causes dead locks.
> 
> I propose that JFR doesn't walk the continuations when sending events. An alternative could be to limit this to ZGC, but I'd like to get some feedback around that from JFR / Loom devs.
> 
> We've been testing this patch in the Generational ZGC repository.

In Deoptimization::unpack_frames we set the frame anchor SP to null across a critical section in which the stack isn't very parseable for tracing tools like JFR, which protects against stack walks there. We could use a very similar solution when doing the processing of the stack watermark - essentially annotate that while we do that, we don't really want any stack tracing here, by clearing the anchor last Java SP. Would that constitute a more desirable solution?

-------------

PR: https://git.openjdk.org/jdk/pull/11586