RFR: 8298377: JfrVframeStream causes deadlocks in ZGC

[Stefan Karlsson]

On Thu, 8 Dec 2022 11:23:57 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

> The JfrVFrameStream is used while generating stack traces for events. One of the events are the ZPage allocation event. This event is sometimes sent when ZGC is relocating. The current implementation of JfrVFrameStream uses WalkContinuation::include, which causes JFR to walk the continuation and perform GC barriers. This is problematic, since ZGC has a requirement that we never perform load barriers while running the relocation code. If we do, we might end up performing other reloctions from the the relocation code, and in some cases that causes dead locks.
> 
> I propose that JFR doesn't walk the continuations when sending events. An alternative could be to limit this to ZGC, but I'd like to get some feedback around that from JFR / Loom devs.
> 
> We've been testing this patch in the Generational ZGC repository.

FYI: I reran testing to reproduce this deadlock and hit an assert instead. The assert happens because we trigger stack walking on a frame form the stack watermark processing:

#  assert(is_stack_watermark_processing_started(thread != nullptr ? thread : JavaThread::current())) failed: Not processed


V  [libjvm.dylib+0x563c48]  report_vm_error(char const*, int, char const*, char const*, ...)+0x80
V  [libjvm.dylib+0x4f055c]  ContinuationEntry::cont_oop(JavaThread const*) const+0x124
V  [libjvm.dylib+0x51e850]  Continuation::top_frame(frame const&, RegisterMap*)+0x128
V  [libjvm.dylib+0x90a9c4]  JfrVframeStream::next_vframe()+0xdc
V  [libjvm.dylib+0x90b168]  JfrStackTrace::record(JavaThread*, frame const&, int)+0x30c
V  [libjvm.dylib+0x90b348]  JfrStackTrace::record(JavaThread*, int)+0xbc
V  [libjvm.dylib+0x90c0d4]  JfrStackTraceRepository::record(JavaThread*, int, JfrStackFrame*, unsigned int)+0x38
V  [libjvm.dylib+0x110ab74]  JfrEvent<EventZPageAllocation>::write_event()+0x8c
V  [libjvm.dylib+0x1108a6c]  EventZPageAllocation::commit(unsigned long long, unsigned long long, unsigned long long, unsigned long long, unsigned int, bool)+0x174
V  [libjvm.dylib+0x1107134]  ZPageAllocator::alloc_page(unsigned char, unsigned long, ZAllocationFlags)+0x1c4
V  [libjvm.dylib+0x10e3eb8]  ZHeap::alloc_page(unsigned char, unsigned long, ZAllocationFlags)+0x1c
V  [libjvm.dylib+0x1105004]  ZObjectAllocator::alloc_page(unsigned char, unsigned long, ZAllocationFlags)+0x6c
V  [libjvm.dylib+0x1105300]  ZObjectAllocator::alloc_object_in_shared_page(ZPage**, unsigned char, unsigned long, unsigned long, ZAllocationFlags)+0x160
V  [libjvm.dylib+0x110591c]  ZObjectAllocator::alloc_object_for_relocation(ZPageTable const*, unsigned long)+0x28
V  [libjvm.dylib+0x111451c]  ZRelocate::relocate_object(ZForwarding*, unsigned long) const+0x1b4
V  [libjvm.dylib+0x5a1030]  oop ZBarrier::barrier<&(ZBarrier::is_good_or_null_fast_path(unsigned long)), &(ZBarrier::load_barrier_on_oop_slow_path(unsigned long))>(oop volatile*, oop)+0xa0
V  [libjvm.dylib+0xf48a40]  ZBarrier::load_barrier_on_oop_field(oop volatile*)+0x88
V  [libjvm.dylib+0x10d05a4]  ZLoadBarrierOopClosure::do_oop(oop*)+0x18
V  [libjvm.dylib+0x7bc81c]  HandleArea::oops_do(OopClosure*)+0xa0
V  [libjvm.dylib+0x89b834]  JavaThread::oops_do_no_frames(OopClosure*, CodeBlobClosure*)+0x64
V  [libjvm.dylib+0x111d4f4]  ZStackWatermark::start_processing_impl(void*)+0xb0
V  [libjvm.dylib+0xef5aa4]  StackWatermark::on_safepoint()+0x6c
V  [libjvm.dylib+0xe8c578]  SafepointMechanism::process(JavaThread*, bool, bool)+0xa0
V  [libjvm.dylib+0x86bfd0]  JavaCallWrapper::JavaCallWrapper(methodHandle const&, Handle, JavaValue*, JavaThread*)+0x174
V  [libjvm.dylib+0x86d65c]  JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*)+0x2dc
V  [libjvm.dylib+0x86cecc]  JavaCalls::call_static(JavaValue*, Klass*, Symbol*, Symbol*, JavaCallArguments*, JavaThread*)+0xd0
V  [libjvm.dylib+0xef8ae4]  LiveFrameStream::create_primitive_slot_instance(StackValueCollection*, int, BasicType, JavaThread*)+0x200
V  [libjvm.dylib+0xef8d70]  LiveFrameStream::values_to_object_array(StackValueCollection*, JavaThread*)+0x1b4
V  [libjvm.dylib+0xef80e8]  LiveFrameStream::fill_live_stackframe(Handle, methodHandle const&, JavaThread*)+0x144
V  [libjvm.dylib+0xef7f84]  LiveFrameStream::fill_frame(int, objArrayHandle, methodHandle const&, JavaThread*)+0x1b8
V  [libjvm.dylib+0xef79bc]  StackWalk::fill_in_frames(long, BaseFrameStream&, int, int, objArrayHandle, int&, JavaThread*)+0x64c
V  [libjvm.dylib+0xef9f20]  StackWalk::fetchNextBatch(Handle, long, long, int, int, objArrayHandle, JavaThread*)+0x2d0
V  [libjvm.dylib+0x9a41b0]  JVM_MoreStackWalk+0x420
J 1762  java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(JJII[Ljava/lang/Object;)I java.base at 21-internal (0 bytes) @ 0x0000000113f7aa78 [0x0000000113f7a9c0+0x00000000000000b8]

-------------

PR: https://git.openjdk.org/jdk/pull/11586