From stefan.karlsson at oracle.com Mon Apr 14 08:39:30 2025 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 14 Apr 2025 10:39:30 +0200 Subject: =?UTF-8?Q?CFV=3A_New_ZGC_Committer=3A_Joel_Sikstr=C3=B6m?= Message-ID: <0baaee2f-a5ea-4429-a2b8-d335b24a5403@oracle.com> I hereby nominate Joel Siktr?m to ZGC Committer. Joel has been working on ZGC and created a number of patches to optimize and fix ZGC issues. See JBS issues below: ??? 8350441: ZGC: Overhaul Page Allocation ??? 8353471: ZGC: Redundant generation id in ZGeneration ??? 8351167: ZGC: Lazily initialize livemap ??? 8351216: ZGC: Store NUMA node count ??? 8342102: ZGC: Optimize copy constructors in ZPhysicalMemory ??? 8339161: ZGC: Remove unused remembered sets ??? 8337674: ZGC: Consistent style for naming private static constants ??? 8339661: ZGC: Move some page resets and verification to callsites ??? 8339579: ZGC: Race results in only one of two remembered sets being cleared ??? 8339163: ZGC: Race in clearing of remembered sets ??? 8339399: ZGC: Remove unnecessary page reset when splitting pages ??? 8337939: ZGC: Make assertions and checks less convoluted and explicit Votes are due by 2025-04-28T10:36+02:00. Only current ZGC Committers [1] are eligible to vote on this nomination.? Votes must be cast in the open by replying to this mailing list. For Lazy Consensus voting instructions, see [2]. Stefan Karlsson [1] https://openjdk.org/census [2] https://openjdk.org/projects/#committer-vote From erik.osterlund at oracle.com Mon Apr 14 11:33:33 2025 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Mon, 14 Apr 2025 11:33:33 +0000 Subject: =?utf-8?B?UmU6IENGVjogTmV3IFpHQyBDb21taXR0ZXI6IEpvZWwgU2lrc3Ryw7Zt?= In-Reply-To: <0baaee2f-a5ea-4429-a2b8-d335b24a5403@oracle.com> References: <0baaee2f-a5ea-4429-a2b8-d335b24a5403@oracle.com> Message-ID: <802C4698-1425-4996-B7E2-34CADC60194F@oracle.com> Vote: yes /Erik > On 14 Apr 2025, at 10:39, Stefan Karlsson wrote: > > ?I hereby nominate Joel Siktr?m to ZGC Committer. > > Joel has been working on ZGC and created a number of patches to optimize and fix ZGC issues. See JBS issues below: > > 8350441: ZGC: Overhaul Page Allocation > 8353471: ZGC: Redundant generation id in ZGeneration > 8351167: ZGC: Lazily initialize livemap > 8351216: ZGC: Store NUMA node count > 8342102: ZGC: Optimize copy constructors in ZPhysicalMemory > 8339161: ZGC: Remove unused remembered sets > 8337674: ZGC: Consistent style for naming private static constants > 8339661: ZGC: Move some page resets and verification to callsites > 8339579: ZGC: Race results in only one of two remembered sets being cleared > 8339163: ZGC: Race in clearing of remembered sets > 8339399: ZGC: Remove unnecessary page reset when splitting pages > 8337939: ZGC: Make assertions and checks less convoluted and explicit > > Votes are due by 2025-04-28T10:36+02:00. > > Only current ZGC Committers [1] are eligible to vote > on this nomination. Votes must be cast in the open by replying > to this mailing list. > > For Lazy Consensus voting instructions, see [2]. > > Stefan Karlsson > > [1] https://openjdk.org/census > [2] https://openjdk.org/projects/#committer-vote From rehn at rivosinc.com Mon Apr 14 12:21:21 2025 From: rehn at rivosinc.com (Robbin Ehn) Date: Mon, 14 Apr 2025 14:21:21 +0200 Subject: =?UTF-8?Q?Re=3A_CFV=3A_New_ZGC_Committer=3A_Joel_Sikstr=C3=B6m?= In-Reply-To: <0baaee2f-a5ea-4429-a2b8-d335b24a5403@oracle.com> References: <0baaee2f-a5ea-4429-a2b8-d335b24a5403@oracle.com> Message-ID: Vote: yes /Robbin On Mon, Apr 14, 2025 at 10:40?AM Stefan Karlsson wrote: > > I hereby nominate Joel Siktr?m to ZGC Committer. > > Joel has been working on ZGC and created a number of patches to optimize > and fix ZGC issues. See JBS issues below: > > 8350441: ZGC: Overhaul Page Allocation > 8353471: ZGC: Redundant generation id in ZGeneration > 8351167: ZGC: Lazily initialize livemap > 8351216: ZGC: Store NUMA node count > 8342102: ZGC: Optimize copy constructors in ZPhysicalMemory > 8339161: ZGC: Remove unused remembered sets > 8337674: ZGC: Consistent style for naming private static constants > 8339661: ZGC: Move some page resets and verification to callsites > 8339579: ZGC: Race results in only one of two remembered sets being > cleared > 8339163: ZGC: Race in clearing of remembered sets > 8339399: ZGC: Remove unnecessary page reset when splitting pages > 8337939: ZGC: Make assertions and checks less convoluted and explicit > > Votes are due by 2025-04-28T10:36+02:00. > > Only current ZGC Committers [1] are eligible to vote > on this nomination. Votes must be cast in the open by replying > to this mailing list. > > For Lazy Consensus voting instructions, see [2]. > > Stefan Karlsson > > [1] https://openjdk.org/census > [2] https://openjdk.org/projects/#committer-vote From mdmitriev at linkedin.com Mon Apr 21 21:49:01 2025 From: mdmitriev at linkedin.com (Misha Dmitriev) Date: Mon, 21 Apr 2025 21:49:01 +0000 Subject: Any plans to emit Thread Stall events through MXBeans? Message-ID: Hi ZGC team, Are there any plans at this time to enable emission of Thread Stall events through MXBeans? We all know that ZGC stop-the-world GC pauses are normally very short. But we've noticed that for real apps at LinkedIn, thread stalls can sometimes be really long (up to a few sec), and apparently, they may affect latencies very noticeably at times. Thus, any system that monitors ZGC performance needs to track both types of GC events. We currently achieve that in some of our monitoring systems through gc.log parsing in real time, but MXBeans may have some advantages in some situations. The same question is valid for Shenandoah as well. Thanks, Misha -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan.karlsson at oracle.com Tue Apr 22 06:59:03 2025 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 22 Apr 2025 08:59:03 +0200 Subject: Any plans to emit Thread Stall events through MXBeans? In-Reply-To: References: Message-ID: <5ab33ee6-87dc-4804-88d0-da0bb71794c6@oracle.com> Hi Misha, On 2025-04-21 23:49, Misha Dmitriev wrote: > Hi ZGC team, > > Are there any plans at this time to enable emission of Thread Stall > events through MXBeans? We all know that ZGC stop-the-world GC pauses > are normally very short. But we've noticed that for real apps at > LinkedIn, thread stalls can sometimes be really long (up to a few > sec), and apparently, they may affect latencies very noticeably at > times. Thus, any system that monitors ZGC performance needs to track > both types of GC events. We currently achieve that in some of our > monitoring systems through gc.log parsing in real time, but MXBeans > may have some advantages in some situations. Just to be sure we are on the same page, when you are saying Thread Stall events are you referring to what we call Allocation Stalls, where Java threads are waiting for the GC to reclaim memory? Or are referring to all kinds of stall that could block the threads? We don't have any plans to add more MXBeans to? ZGC, but maybe we should? Nowadays we tend to send events through JFR instead, and I don't see much new development of features with MXBeans in the JVM. I'll bring question up with the Serviceability team. If JFR would work for you, we do have the ZAllocationStall event that gets sent every time we hit an allocation stall. Maybe this together with the newish JFR streaming API would be enough to monitor these stalls? See: https://openjdk.org/jeps/349 Cheers, StefanK > > The same question is valid for Shenandoah as well. > > Thanks, > > Misha > -------------- next part -------------- An HTML attachment was scrubbed... URL: From evaristojosec at yahoo.es Tue Apr 22 08:49:17 2025 From: evaristojosec at yahoo.es (=?UTF-8?Q?Evaristo_Jos=C3=A9_Camarero?=) Date: Tue, 22 Apr 2025 08:49:17 +0000 (UTC) Subject: Any plans to emit Thread Stall events through MXBeans? In-Reply-To: <5ab33ee6-87dc-4804-88d0-da0bb71794c6@oracle.com> References: <5ab33ee6-87dc-4804-88d0-da0bb71794c6@oracle.com> Message-ID: <1214974762.320958.1745311757039@mail.yahoo.com> Hi, As commented here Allocation Stalls is a MUST to proper monitor ZGC performance, because in practice is usually the main contributor to latency spikes. In our case we are monitoring JVM memory and Gen ZGC collections via MBeans, and I would say that is still quite common (e.g. Spring actuator). JFR events will provide the info, BUT at least has small performance penalty compared with MBeans. In my view having the info via JMX could be a nice addition. Probably having something like number of allocation stalls and the accumulated time for all of them could be a great addition in my view. Thanks a lot for all the good work with ZGC. Regards, Evaristo En martes, 22 de abril de 2025, 08:59:42 CEST, Stefan Karlsson escribi?: Hi Misha, On 2025-04-21 23:49, Misha Dmitriev wrote: #yiv1653332920 P {margin-top:0;margin-bottom:0;} Hi ZGC team, Are there any plans at this time to enable emission of Thread Stall events through MXBeans? We all know that ZGC stop-the-world GC pauses are normally very short. But we've noticed that for real apps at LinkedIn, thread stalls can sometimes be really long (up to a few sec), and apparently, they may affect latencies very noticeably at times. Thus, any system that monitors ZGC performance needs to track both types of GC events. We currently achieve that in some of our monitoring systems through gc.log parsing in real time, but MXBeans may have some advantages in some situations. Just to be sure we are on the same page, when you are saying Thread Stall events are you referring to what we call Allocation Stalls, where Java threads are waiting for the GC to reclaim memory? Or are referring to all kinds of stall that could block the threads? We don't have any plans to add more MXBeans to? ZGC, but maybe we should? Nowadays we tend to send events through JFR instead, and I don't see much new development of features with MXBeans in the JVM. I'll bring question up with the Serviceability team. If JFR would work for you, we do have the ZAllocationStall event that gets sent every time we hit an allocation stall. Maybe this together with the newish JFR streaming API would be enough to monitor these stalls? See: https://openjdk.org/jeps/349 Cheers, StefanK The same question is valid for Shenandoah as well. Thanks, Misha -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdmitriev at linkedin.com Wed Apr 23 01:34:46 2025 From: mdmitriev at linkedin.com (Misha Dmitriev) Date: Wed, 23 Apr 2025 01:34:46 +0000 Subject: Any plans to emit Thread Stall events through MXBeans? In-Reply-To: <5ab33ee6-87dc-4804-88d0-da0bb71794c6@oracle.com> References: <5ab33ee6-87dc-4804-88d0-da0bb71794c6@oracle.com> Message-ID: Hi Stefan, Thank you for the quick reply, and sorry about the terminology. Yes, I am talking about the Allocation Stall events. For the purposes of this conversation, we are interested specifically in the GC impact on app behavior. In my specific case, I was interested in getting Allocation Stall notifications via JMX, because one of our existing systems that monitors GC performance from inside each running app, and makes some decisions based on that, already uses JMX to get STW pause notifications from other GCs. But if for ZGC we will need to use something else anyway, then we will figure it out. Good to know that JFR now provides the streaming API, and looks like it has been implemented in Java 14? We should probably consider using it at some point. Misha ________________________________ From: Stefan Karlsson Sent: Monday, April 21, 2025 11:59 PM To: Misha Dmitriev ; zgc-dev at openjdk.org Subject: Re: Any plans to emit Thread Stall events through MXBeans? Hi Misha, On 2025-04-21 23:49, Misha Dmitriev wrote: Hi ZGC team, Are there any plans at this time to enable emission of Thread Stall events through MXBeans? We all know that ZGC stop-the-world GC pauses are normally very short. But we've noticed that for real apps at LinkedIn, thread stalls can sometimes be really long (up to a few sec), and apparently, they may affect latencies very noticeably at times. Thus, any system that monitors ZGC performance needs to track both types of GC events. We currently achieve that in some of our monitoring systems through gc.log parsing in real time, but MXBeans may have some advantages in some situations. Just to be sure we are on the same page, when you are saying Thread Stall events are you referring to what we call Allocation Stalls, where Java threads are waiting for the GC to reclaim memory? Or are referring to all kinds of stall that could block the threads? We don't have any plans to add more MXBeans to ZGC, but maybe we should? Nowadays we tend to send events through JFR instead, and I don't see much new development of features with MXBeans in the JVM. I'll bring question up with the Serviceability team. If JFR would work for you, we do have the ZAllocationStall event that gets sent every time we hit an allocation stall. Maybe this together with the newish JFR streaming API would be enough to monitor these stalls? See: https://openjdk.org/jeps/349 Cheers, StefanK The same question is valid for Shenandoah as well. Thanks, Misha -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter_booth at me.com Wed Apr 23 18:39:19 2025 From: peter_booth at me.com (Peter Booth) Date: Wed, 23 Apr 2025 14:39:19 -0400 Subject: Any plans to emit Thread Stall events through MXBeans? In-Reply-To: References: <5ab33ee6-87dc-4804-88d0-da0bb71794c6@oracle.com> Message-ID: <8D8804DF-B695-4BC5-9D85-16FE8223E526@me.com> I?m a little puzzled by the thread. Are we saying that, at present, there is a type of GC related stall that isn?t visible from log parsing or are all stalls visible from existing logs. I?m trying to understand whether this is an issue of *how* the JVM emits events about its state or *what* information is being emitted (regardless of mechanism). In the past, when managing JVMs that included Oracle, Azul, IBM I encountered difficulties interpreting MXBean memory data due to inconsistent terminology /concepts between collectors which, eight years ago, seemed insurmountable. Peter > On Apr 22, 2025, at 9:34?PM, Misha Dmitriev wrote: > > Hi Stefan, > > Thank you for the quick reply, and sorry about the terminology. Yes, I am talking about the Allocation Stall events. For the purposes of this conversation, we are interested specifically in the GC impact on app behavior. > > In my specific case, I was interested in getting Allocation Stall notifications via JMX, because one of our existing systems that monitors GC performance from inside each running app, and makes some decisions based on that, already uses JMX to get STW pause notifications from other GCs. But if for ZGC we will need to use something else anyway, then we will figure it out. Good to know that JFR now provides the streaming API, and looks like it has been implemented in Java 14? We should probably consider using it at some point. > > Misha > > From: Stefan Karlsson > Sent: Monday, April 21, 2025 11:59 PM > To: Misha Dmitriev ; zgc-dev at openjdk.org > Subject: Re: Any plans to emit Thread Stall events through MXBeans? > > Hi Misha, > > On 2025-04-21 23:49, Misha Dmitriev wrote: >> Hi ZGC team, >> >> Are there any plans at this time to enable emission of Thread Stall events through MXBeans? We all know that ZGC stop-the-world GC pauses are normally very short. But we've noticed that for real apps at LinkedIn, thread stalls can sometimes be really long (up to a few sec), and apparently, they may affect latencies very noticeably at times. Thus, any system that monitors ZGC performance needs to track both types of GC events. We currently achieve that in some of our monitoring systems through gc.log parsing in real time, but MXBeans may have some advantages in some situations. > > Just to be sure we are on the same page, when you are saying Thread Stall events are you referring to what we call Allocation Stalls, where Java threads are waiting for the GC to reclaim memory? Or are referring to all kinds of stall that could block the threads? > > We don't have any plans to add more MXBeans to ZGC, but maybe we should? Nowadays we tend to send events through JFR instead, and I don't see much new development of features with MXBeans in the JVM. I'll bring question up with the Serviceability team. > > If JFR would work for you, we do have the ZAllocationStall event that gets sent every time we hit an allocation stall. Maybe this together with the newish JFR streaming API would be enough to monitor these stalls? See: https://openjdk.org/jeps/349 > > Cheers, > StefanK > >> >> The same question is valid for Shenandoah as well. >> >> Thanks, >> >> Misha -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdmitriev at linkedin.com Wed Apr 23 19:06:30 2025 From: mdmitriev at linkedin.com (Misha Dmitriev) Date: Wed, 23 Apr 2025 19:06:30 +0000 Subject: Any plans to emit Thread Stall events through MXBeans? In-Reply-To: <8D8804DF-B695-4BC5-9D85-16FE8223E526@me.com> References: <5ab33ee6-87dc-4804-88d0-da0bb71794c6@oracle.com> <8D8804DF-B695-4BC5-9D85-16FE8223E526@me.com> Message-ID: My understanding is that gc.log always contains all the information about GC events, including Allocation Stalls. It's just that the other mechanisms for communication of JVM events, especially MXBeans, may or may not cover all possible events. Misha ________________________________ From: Peter Booth Sent: Wednesday, April 23, 2025 11:39 AM To: Misha Dmitriev Cc: Stefan Karlsson ; zgc-dev at openjdk.org Subject: Re: Any plans to emit Thread Stall events through MXBeans? I?m a little puzzled by the thread. Are we saying that, at present, there is a type of GC related stall that isn?t visible from log parsing or are all stalls visible from existing logs. I?m trying to understand whether this is an issue of *how* the JVM emits events about its state or *what* information is being emitted (regardless of mechanism). In the past, when managing JVMs that included Oracle, Azul, IBM I encountered difficulties interpreting MXBean memory data due to inconsistent terminology /concepts between collectors which, eight years ago, seemed insurmountable. Peter On Apr 22, 2025, at 9:34?PM, Misha Dmitriev wrote: Hi Stefan, Thank you for the quick reply, and sorry about the terminology. Yes, I am talking about the Allocation Stall events. For the purposes of this conversation, we are interested specifically in the GC impact on app behavior. In my specific case, I was interested in getting Allocation Stall notifications via JMX, because one of our existing systems that monitors GC performance from inside each running app, and makes some decisions based on that, already uses JMX to get STW pause notifications from other GCs. But if for ZGC we will need to use something else anyway, then we will figure it out. Good to know that JFR now provides the streaming API, and looks like it has been implemented in Java 14? We should probably consider using it at some point. Misha ________________________________ From: Stefan Karlsson Sent: Monday, April 21, 2025 11:59 PM To: Misha Dmitriev ; zgc-dev at openjdk.org Subject: Re: Any plans to emit Thread Stall events through MXBeans? Hi Misha, On 2025-04-21 23:49, Misha Dmitriev wrote: Hi ZGC team, Are there any plans at this time to enable emission of Thread Stall events through MXBeans? We all know that ZGC stop-the-world GC pauses are normally very short. But we've noticed that for real apps at LinkedIn, thread stalls can sometimes be really long (up to a few sec), and apparently, they may affect latencies very noticeably at times. Thus, any system that monitors ZGC performance needs to track both types of GC events. We currently achieve that in some of our monitoring systems through gc.log parsing in real time, but MXBeans may have some advantages in some situations. Just to be sure we are on the same page, when you are saying Thread Stall events are you referring to what we call Allocation Stalls, where Java threads are waiting for the GC to reclaim memory? Or are referring to all kinds of stall that could block the threads? We don't have any plans to add more MXBeans to ZGC, but maybe we should? Nowadays we tend to send events through JFR instead, and I don't see much new development of features with MXBeans in the JVM. I'll bring question up with the Serviceability team. If JFR would work for you, we do have the ZAllocationStall event that gets sent every time we hit an allocation stall. Maybe this together with the newish JFR streaming API would be enough to monitor these stalls? See: https://openjdk.org/jeps/349 Cheers, StefanK The same question is valid for Shenandoah as well. Thanks, Misha -------------- next part -------------- An HTML attachment was scrubbed... URL: