<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /></head><body style='font-size: 10pt; font-family: Verdana,Geneva,sans-serif'>
<p>Hi,</p>
<p>Yes turning off jdk.OldObjectSample event solved the issue : the real time execution of my sample with zgc and JFR recording with jdk.OldObjectSample turned off is now very close to that without JFR recording.</p>
<p>Thank you very much.</p>
<p>Best regards.</p>
<p>Fabrice</p>
<div id="signature"></div>
<p><br /></p>
<p id="reply-intro">Le 2026-01-14 19:30, Markus Gronlund a écrit :</p>
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0">
<style type="text/css">@font-face
{ font-family: "Cambria Math"; }
@font-face
{ font-family: Calibri; }
@font-face
{ font-family: Verdana; }
#replybody1 p.MsoNormal, #replybody1 li.MsoNormal, #replybody1 div.MsoNormal
{ margin: 0cm; font-size: 11.0pt; font-family: "Calibri",sans-serif; }
#replybody1 a:link, #replybody1 span.MsoHyperlink
{ mso-style-priority: 99; color: blue; text-decoration: underline; }
#replybody1 span.EmailStyle21
{ mso-style-type: personal-reply; font-family: "Calibri",sans-serif; color: windowtext; }
#replybody1 .MsoChpDefault
{ mso-style-type: export-only; font-size: 10.0pt; mso-ligatures: none; }
@page WordSection1
{ size: 612.0pt 792.0pt; margin: 72.0pt 72.0pt 72.0pt 72.0pt; }
#replybody1 div.WordSection1
{}</style>
<div id="replybody1" lang="en-SE" style="word-wrap: break-word;">
<div class="v1WordSection1">
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">Hi again,</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">I just remembered we have improved our ergonomics over the years.</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">Therefore, there is a much easier way for you to do this without configuring anything in the .jfc files: you can simply override event settings on the command line. [1]</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">-XX:StartFlightRecording:jdk.OldObjectSample#enabled=false</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">Way easier!</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">Cheers</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">Markus</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">[1] <a href="https://egahlin.github.io/2022/05/31/improved-ergonomics.html" target="_blank" rel="noopener noreferrer"> https://egahlin.github.io/2022/05/31/improved-ergonomics.html</a></span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<div><br />
<p style="font-family: Calibri; font-size: 10pt; color: #000000; margin: 5pt; font-style: normal; font-weight: normal; text-decoration: none;" align="Left">Confidential- Oracle Internal</p>
<div style="border: none; border-top: solid #E1E1E1 1.0pt; padding: 3.0pt 0cm 0cm 0cm;">
<p class="v1MsoNormal"><strong><span>From:</span></strong><span> hotspot-jfr-dev <hotspot-jfr-dev-retn@openjdk.org> <strong>On Behalf Of </strong>Markus Gronlund<br /><strong>Sent:</strong> Wednesday, 14 January 2026 19:15<br /><strong>To:</strong> Fabrice Bibonne <fabrice.bibonne@courriel.eco><br /><strong>Cc:</strong> hotspot-jfr-dev@openjdk.org<br /><strong>Subject:</strong> RE: Using JFR both with ZGC degrades application throughput</span></p>
</div>
</div>
<p class="v1MsoNormal"> </p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">Hi Fabrice,</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">Thank you very much for reporting this and also for providing a great reproducer.</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">We have made some progress towards understanding the problem space, at least.</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">To help you continue with your demonstrations, explanations, and comparisons, I only need you to do the following:</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">In the jdk/lib/jfr directory, there are two files that control the default and profile sets of JFR events: default.jfc and profile.jfc, respectively.</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> <event name="jdk.OldObjectSample"></span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> <setting name="enabled" control="old-objects-enabled"><span style="color: red;">false</span></setting></span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> <setting name="stackTrace" control="old-objects-stack-trace">false</setting></span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> <setting name="cutoff" control="old-objects-cutoff">0 ns</setting></span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </event></span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">Turn off the jdk.OldObjectSample event by setting enabled to false.</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">This effectively turns off JFRs capability to monitor memory leaks in the background.</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">With this small change, you should be back on track for proper comparisons, also when using JFR.</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">Let me know if you have any questions. We will be thinking about how to solve this properly.</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">Cheers for now</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">Regards</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;">Markus</span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<p class="v1MsoNormal"><span style="mso-fareast-language: EN-US;"> </span></p>
<div>
<p class="v1MsoNormal"><span> </span></p>
<p style="margin: 5.0pt;"><span style="font-size: 10.0pt; color: black;">Confidential- Oracle Internal</span></p>
<div style="border: none; border-top: solid #E1E1E1 1.0pt; padding: 3.0pt 0cm 0cm 0cm;">
<p class="v1MsoNormal"><strong><span>From:</span></strong><span> hotspot-jfr-dev <<a href="mailto:hotspot-jfr-dev-retn@openjdk.org" rel="noreferrer">hotspot-jfr-dev-retn@openjdk.org</a>> <strong>On Behalf Of </strong>Fabrice Bibonne<br /><strong>Sent:</strong> Monday, 12 January 2026 16:59<br /><strong>To:</strong> <a href="mailto:hotspot-jfr-dev@openjdk.org" rel="noreferrer">hotspot-jfr-dev@openjdk.org</a><br /><strong>Subject:</strong> Re: Using JFR both with ZGC degrades application throughput</span></p>
</div>
</div>
<p class="v1MsoNormal"><span> </span></p>
<p><span style="font-size: 10.0pt; font-family: 'Verdana',sans-serif;">Here is a unique source code file for the reproducer (the big String is generated when starting as you suggested). It changes a little the results but the run with zgc + jfr is still taking lot of time. </span></p>
<p><span style="font-size: 10.0pt; font-family: 'Verdana',sans-serif;">Thanks you for having a look.</span></p>
<p><span style="font-size: 10.0pt; font-family: 'Verdana',sans-serif;">Fabrice</span></p>
<p><span style="font-size: 10.0pt; font-family: 'Verdana',sans-serif;"> </span></p>
<p><span style="font-size: 10.0pt; font-family: 'Verdana',sans-serif;"> </span></p>
<p id="v1reply-intro"><span style="font-size: 10.0pt; font-family: 'Verdana',sans-serif;">Le 2026-01-12 10:56, Erik Gahlin a écrit :</span></p>
<blockquote style="border: none; border-left: solid #1010FF 1.5pt; padding: 0cm 0cm 0cm 5.0pt; margin: 5.0pt 0cm 5.0pt 0cm;">
<div>
<p class="v1MsoNormal"><span style="font-size: 10.0pt; font-family: 'Courier New';">Hi Fabrice,<br /><br />Thanks for reporting!<br /><br />Could you post the source code for the reproducer here? The 36 MB file could probably be replaced with a String::repeat expression.<br /><br />JFR does use some memory, which could impact available heap and performance, although the degradation you're seeing seems awfully high.<br /><br />Thanks<br />Erik<br /><br />________________________________________<br />From: hotspot-jfr-dev <<a href="mailto:hotspot-jfr-dev-retn@openjdk.org" rel="noreferrer">hotspot-jfr-dev-retn@openjdk.org</a>> on behalf of Fabrice Bibonne <<a href="mailto:fabrice.bibonne@courriel.eco" rel="noreferrer">fabrice.bibonne@courriel.eco</a>><br />Sent: Sunday, January 11, 2026 7:23 PM<br />To: <a href="mailto:hotspot-jfr-dev@openjdk.org" rel="noreferrer">hotspot-jfr-dev@openjdk.org</a><br />Subject: Using JFR both with ZGC degrades application throughput<br /><br />Hi all,<br /><br /> I would like to report a case where starting jfr for an application running with zgc causes a significant throughput degradation (compared to when JFR is not started).<br /><br /> My context : I was writing a little web app to illustrate a case where the use of ZGC gives a better throughput than with G1. I benchmarked with grafana k6 my application running with G1 and my application running with ZGC : the runs with ZGC gave better throughputs. I wanted to go a bit further in explanation so I began again my benchmarks with JFR to be able to illustrate GC gains in JMC. When I ran my web app with ZGC+JFR, I noticed a significant throughput degradation in my benchmark (which was not the case with G1+JFR).<br /><br /> Although I did not measure an increase in overhead as such, I still wanted to report this issue because the degradation in throughput with JFR is such that it would not be usable as is on a production service.<br /><br />I wrote a little application (not a web one) to reproduce the problem : the application calls a little conversion service 200 times with random numbers in parallel (to be like a web app in charge and to pressure GC). The conversion service (a method named `convertNumberToWords`) convert the number in a String looking for the String in a Map with the number as th key. In order to instantiate and destroy many objects at each call, the map is built parsing a huge String at each call. Application ends after 200 calls.<br /><br />Here are the step to reproduce :<br />1. Clone <a href="https://framagit.org/FBibonne/poc-java/-/tree/jfr+zgc_impact" target="_blank" rel="noopener noreferrer"> https://framagit.org/FBibonne/poc-java/-/tree/jfr+zgc_impact</a> (be aware to be on branch jfr+zgc_impact)<br />2. Compile it (you must include numbers200k.zip in resources : it contains a 36 Mo text files whose contents are used to create the huge String variable)<br />3. in the root of repository :<br />3a. Run `time java -Xmx4g -XX:+UseZGC -XX:+UseCompressedOops -classpath target/classes poc.java.perf.write.TestPerf #ZGC without JFR`<br />3b. Run `time java -Xmx4g -XX:+UseZGC -XX:+UseCompressedOops -XX:StartFlightRecording -classpath target/classes poc.java.perf.write.TestPerf #ZGC with JFR`<br />4. The real time of the second run (with JFR) will be considerably higher than that of the first<br /><br />I ran these tests on my laptop :<br />- Dell Inc. Latitude 5591<br />- openSUSE Tumbleweed 20260108<br />- Kernel : 6.18.3-1-default (64-bit)<br />- 12 × Intel® Core™ i7-8850H CPU @ 2.60GHz<br />- RAM 16 Gio<br />- openjdk version "25.0.1" 2025-10-21<br />- OpenJDK Runtime Environment (build 25.0.1+8-27)<br />- OpenJDK 64-Bit Server VM (build 25.0.1+8-27, mixed mode, sharing)<br />- many tabs opened in firefox !<br /><br />I also ran it in a container (eclipse-temurin:25) on my laptop and with a windows laptop and came to the same conclusions : here are the measurements from the container :<br /><br />| Run with | Real time (s) |<br />|-----------|---------------|<br />| ZGC alone | 7.473 |<br />| ZGC + jfr | 25.075 |<br />| G1 alone | 10.195 |<br />| G1 + jfr | 10.450 |<br /><br />After all these tests I tried to run the app with an other profiler tool in order to understand where is the issue. I join the flamegraph when running jfr+zgc : for the worker threads of the ForkJoinPool of Stream, stack traces of a majority of samples have the same top lines :<br />- PosixSemaphore::wait<br />- ZPageAllocator::alloc_page_stall<br />- ZPageAllocator::alloc_page_inner<br />- ZPageAllocator::alloc_page<br /><br />So many thread seem to spent their time waiting in the method ZPageAllocator::alloc_page_stall when the JFR is on. The JFR periodic tasks threads has also a few samples where it waits at ZPageAllocator::alloc_page_stall. I hope this will help you to find the issue.<br /><br />Thank you very much for reading this email until the end. I hope this is the good place for such a feedback. Let me know if I must report my problem elsewhere. Be free to ask me more questions if you need.<br /><br />Thank you all for this amazing tool !</span></p>
</div>
</blockquote>
</div>
</div>
</blockquote>
</body></html>