Any plans to emit Thread Stall events through MXBeans?

[Misha Dmitriev]

My understanding is that gc.log always contains all the information about GC events, including Allocation Stalls. It's just that the other mechanisms for communication of JVM events, especially MXBeans, may or may not cover all possible events.

Misha

________________________________
From: Peter Booth <peter_booth at me.com>
Sent: Wednesday, April 23, 2025 11:39 AM
To: Misha Dmitriev <mdmitriev at linkedin.com>
Cc: Stefan Karlsson <stefan.karlsson at oracle.com>; zgc-dev at openjdk.org <zgc-dev at openjdk.org>
Subject: Re: Any plans to emit Thread Stall events through MXBeans?

I’m a little puzzled by the thread. Are we saying that, at present, there is a type of GC related stall that isn’t visible from log parsing or are all stalls visible from existing logs. I’m trying to understand whether this is an issue of *how* the JVM emits events about its state or *what* information is being emitted (regardless of mechanism).

In the past, when managing JVMs that included Oracle, Azul, IBM I encountered difficulties interpreting MXBean memory data due to inconsistent terminology /concepts between collectors which, eight years ago, seemed insurmountable.

Peter


On Apr 22, 2025, at 9:34 PM, Misha Dmitriev <mdmitriev at linkedin.com> wrote:

Hi Stefan,

Thank you for the quick reply, and sorry about the terminology. Yes, I am talking about the Allocation Stall events. For the purposes of this conversation, we are interested specifically in the GC impact on app behavior.

In my specific case, I was interested in getting Allocation Stall notifications via JMX, because one of our existing systems that monitors GC performance from inside each running app, and makes some decisions based on that, already uses JMX to get STW pause notifications from other GCs. But if for ZGC we will need to use something else anyway, then we will figure it out. Good to know that JFR now provides the streaming API, and looks like it has been implemented in Java 14? We should probably consider using it at some point.

Misha

________________________________
From: Stefan Karlsson <stefan.karlsson at oracle.com>
Sent: Monday, April 21, 2025 11:59 PM
To: Misha Dmitriev <mdmitriev at linkedin.com>; zgc-dev at openjdk.org <zgc-dev at openjdk.org>
Subject: Re: Any plans to emit Thread Stall events through MXBeans?

Hi Misha,

On 2025-04-21 23:49, Misha Dmitriev wrote:
Hi ZGC team,

Are there any plans at this time to enable emission of Thread Stall events through MXBeans? We all know that ZGC stop-the-world GC pauses are normally very short. But we've noticed that for real apps at LinkedIn, thread stalls can sometimes be really long (up to a few sec), and apparently, they may affect latencies very noticeably at times. Thus, any system that monitors ZGC performance needs to track both types of GC events. We currently achieve that in some of our monitoring systems through gc.log parsing in real time, but MXBeans may have some advantages in some situations.

Just to be sure we are on the same page, when you are saying Thread Stall events are you referring to what we call Allocation Stalls, where Java threads are waiting for the GC to reclaim memory? Or are referring to all kinds of stall that could block the threads?

We don't have any plans to add more MXBeans to  ZGC, but maybe we should? Nowadays we tend to send events through JFR instead, and I don't see much new development of features with MXBeans in the JVM. I'll bring question up with the Serviceability team.

If JFR would work for you, we do have the ZAllocationStall event that gets sent every time we hit an allocation stall. Maybe this together with the newish JFR streaming API would be enough to monitor these stalls? See: https://openjdk.org/jeps/349

Cheers,
StefanK


The same question is valid for Shenandoah as well.

Thanks,

Misha

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/zgc-dev/attachments/20250423/69f46c60/attachment-0001.htm>