Using JFR both with ZGC degrades application throughput

[Fabrice Bibonne]

Hi all,

  I would like to report a case where starting jfr for an application 
running with zgc causes a significant throughput degradation (compared 
to when JFR is not started).

  My context : I was writing a little web app to illustrate a case where 
the use of ZGC gives a better throughput than with G1. I benchmarked 
with grafana k6 my application running with G1 and my application 
running with ZGC  : the runs with ZGC gave better throughputs. I wanted 
to go a bit further in explanation so I began again my benchmarks with 
JFR to be able to illustrate GC gains in JMC. When I ran my web app with 
ZGC+JFR, I noticed a significant throughput degradation in my benchmark 
(which was not the case with G1+JFR).

  Although I did not measure an increase in overhead as such, I still 
wanted to report this issue because the degradation in throughput with 
JFR is such that it would not be usable as is on a production service.

I wrote a little application (not a web one) to reproduce the problem : 
the application calls a little conversion service 200 times with random 
numbers in parallel (to be like a web app in charge and to pressure GC). 
The conversion service (a method named `convertNumberToWords`) convert 
the number in a String looking for the String in a Map with the number 
as th key. In order to instantiate and destroy many objects at each 
call, the map is built parsing a huge String at each call. Application 
ends after 200 calls.

Here are the step to reproduce :
1. Clone https://framagit.org/FBibonne/poc-java/-/tree/jfr+zgc_impact 
(be aware to be on branch jfr+zgc_impact)
2. Compile it (you must include numbers200k.zip in resources : it 
contains a 36 Mo text files whose contents are used to create the huge 
String variable)
3. in the root of repository :
3a. Run `time java -Xmx4g -XX:+UseZGC -XX:+UseCompressedOops -classpath 
target/classes poc.java.perf.write.TestPerf #ZGC without JFR`
3b. Run `time java -Xmx4g -XX:+UseZGC -XX:+UseCompressedOops 
-XX:StartFlightRecording -classpath target/classes 
poc.java.perf.write.TestPerf #ZGC with JFR`
4. The real time of the second run (with JFR) will be considerably 
higher than that of the first

I ran these tests on my laptop :
- Dell Inc. Latitude 5591
- openSUSE Tumbleweed 20260108
- Kernel : 6.18.3-1-default (64-bit)
- 12 × Intel(R) Core(tm) i7-8850H CPU @ 2.60GHz
- RAM 16 Gio
- openjdk version "25.0.1" 2025-10-21
- OpenJDK Runtime Environment (build 25.0.1+8-27)
- OpenJDK 64-Bit Server VM (build 25.0.1+8-27, mixed mode, sharing)
- many tabs opened in firefox !

I also ran it in a container (eclipse-temurin:25) on my laptop and with 
a windows laptop and came to the same conclusions : here are the 
measurements from the container :

| Run with  | Real time (s) |
|-----------|---------------|
| ZGC alone | 7.473         |
| ZGC + jfr | 25.075        |
| G1 alone  | 10.195        |
| G1 + jfr  | 10.450        |

After all these tests I tried to run the app with an other profiler tool 
in order to understand where is the issue. I join the flamegraph when 
running jfr+zgc : for the worker threads of the ForkJoinPool of Stream, 
stack traces of a majority of samples have the same top lines :
- PosixSemaphore::wait
- ZPageAllocator::alloc_page_stall
- ZPageAllocator::alloc_page_inner
- ZPageAllocator::alloc_page

So many thread seem to spent their time waiting in the method 
ZPageAllocator::alloc_page_stall when the JFR is on. The JFR periodic 
tasks threads has also a few samples where it waits at 
ZPageAllocator::alloc_page_stall. I hope this will help you to find the 
issue.

Thank you very much for reading this email until the end. I hope this is 
the good place for such a feedback. Let me know if I must report my 
problem elsewhere. Be free to ask me more questions if you need.

Thank you all for this amazing tool !
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-jfr-dev/attachments/20260111/b775f969/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: withJfr.html
Type: application/octet-stream
Size: 76123 bytes
Desc: not available
URL: <https://mail.openjdk.org/pipermail/hotspot-jfr-dev/attachments/20260111/b775f969/withJfr-0001.html>