Updated benchmark numbers in premain README
I've updated the benchmark numbers using the latest premain tip (as of 2025/09/30): https://github.com/openjdk/leyden/blob/premain/README.md#premain-aot-cache-s... There's quite a bit of improvement due to the new compiler tuning changes by Aleksey. I also added a "2 cores only" mode to emulate microservices that are allocated very few CPUs. Here are the results of {premain AOT} vs {JDK 25, no CDS, no AOT}. Note that premain is generally much more beneficial with the 2 Cores Only case, except for PetClinic: Benchmark Desktop/Server Class (28 Cores) 2 Cores Only *---------------------------------------------------------------------------- *Helidon Quick Start 3.59x 4.11x ++ JavacBenchApp 50 source files 2.21x 3.17x ++++ Micronaut First App Demo 2.91x 4.90x +++++++ Quarkus Getting Started Demo 2.97x 3.74x +++ Spring-boot Getting Started Demo 4.13x 4.70x ++ Spring PetClinic Demo 3.33x 3.03x -- ??? Changes from last time ( https://github.com/openjdk/leyden/blob/46e03d000efd1b2784ad4dcd4c83310ace498... ) The 2 Core Only case see a lot of improvements with the compiler tuning. PetClinic is again an outlier. Benchmark Desktop/Server Class (28 Cores) 2 Cores Only ---------------------------------------------------------------------------- Helidon Quick Start 3.60 -> 3.59x ~ 3.53 -> 4.11x ++ JavacBenchApp 50 source files 2.17 -> 2.21x ~ 2.74 -> 3.17x ++ Micronaut First App Demo 2.85 -> 2.91x + 4.39 -> 4.90x ++ Quarkus Getting Started Demo 2.73 -> 2.97x ++ 2.95 -> 3.74x +++ Spring-boot Getting Started Demo 3.96 -> 4.13x ++ 3.70 -> 4.70x +++ Spring PetClinic Demo 3.24 -> 3.33x + 3.04 -> 3.03x ~ (The +, -, ~ are calculated by my eyeballs so not very scientific :-)
For the Spring Petclinic demo, I am not sure yet about the reason for it to be an outlier, but I would suggest to: - Update to a more recent commit than the outdated one (see https://github.com/openjdk/leyden/blob/premain/test/hotspot/jtreg/premain/sp... ) - Leverage the CDS/AOT cache friendly unpacking feature (available as of Spring Boot 3.3) https://docs.spring.io/spring-boot/reference/packaging/efficient.html - Use a more production-like arrangement by avoiding creating the database schema at startup Spring-boot Getting Started Demo should be also updated to a more recent version and use the CDS/AOT cache friendly unpacking feature. I suggest we collaborate on that. On Tue, Sep 30, 2025 at 9:49 PM <ioi.lam@oracle.com> wrote:
I've updated the benchmark numbers using the latest premain tip (as of 2025/09/30):
https://github.com/openjdk/leyden/blob/premain/README.md#premain-aot-cache-s...
There's quite a bit of improvement due to the new compiler tuning changes by Aleksey. I also added a "2 cores only" mode to emulate microservices that are allocated very few CPUs.
Here are the results of {premain AOT} vs {JDK 25, no CDS, no AOT}. Note that premain is generally much more beneficial with the 2 Cores Only case, except for PetClinic:
Benchmark Desktop/Server Class (28 Cores) 2 Cores Only
*---------------------------------------------------------------------------- *Helidon Quick Start 3.59x 4.11x ++ JavacBenchApp 50 source files 2.21x 3.17x ++++ Micronaut First App Demo 2.91x 4.90x +++++++ Quarkus Getting Started Demo 2.97x 3.74x +++ Spring-boot Getting Started Demo 4.13x 4.70x ++ Spring PetClinic Demo 3.33x 3.03x -- ???
Changes from last time ( https://github.com/openjdk/leyden/blob/46e03d000efd1b2784ad4dcd4c83310ace498... )
The 2 Core Only case see a lot of improvements with the compiler tuning. PetClinic is again an outlier.
Benchmark Desktop/Server Class (28 Cores) 2 Cores Only
---------------------------------------------------------------------------- Helidon Quick Start 3.60 -> 3.59x ~ 3.53 -> 4.11x ++ JavacBenchApp 50 source files 2.17 -> 2.21x ~ 2.74 -> 3.17x ++ Micronaut First App Demo 2.85 -> 2.91x + 4.39 -> 4.90x ++ Quarkus Getting Started Demo 2.73 -> 2.97x ++ 2.95 -> 3.74x +++ Spring-boot Getting Started Demo 3.96 -> 4.13x ++ 3.70 -> 4.70x +++ Spring PetClinic Demo 3.24 -> 3.33x + 3.04 -> 3.03x ~
(The +, -, ~ are calculated by my eyeballs so not very scientific :-)
Hi Sebastien, if you could help us update the benchmark, that would be great! Thanks - Ioi On 10/2/25 2:52 AM, Sebastien Deleuze wrote:
For the Spring Petclinic demo, I am not sure yet about the reason for it to be an outlier, but I would suggest to: - Update to a more recent commit than the outdated one (see https://github.com/openjdk/leyden/blob/premain/test/hotspot/jtreg/premain/sp...) - Leverage the CDS/AOT cache friendly unpacking feature (available as of Spring Boot 3.3) https://docs.spring.io/spring-boot/reference/packaging/efficient.html - Use a more production-like arrangement by avoiding creating the database schema at startup
Spring-boot Getting Started Demo should be also updated to a more recent version and use the CDS/AOT cache friendly unpacking feature.
I suggest we collaborate on that.
On Tue, Sep 30, 2025 at 9:49 PM <ioi.lam@oracle.com> wrote:
I've updated the benchmark numbers using the latest premain tip (as of 2025/09/30):
https://github.com/openjdk/leyden/blob/premain/README.md#premain-aot-cache-s...
There's quite a bit of improvement due to the new compiler tuning changes by Aleksey. I also added a "2 cores only" mode to emulate microservices that are allocated very few CPUs.
Here are the results of {premain AOT} vs {JDK 25, no CDS, no AOT}. Note that premain is generally much more beneficial with the 2 Cores Only case, except for PetClinic:
Benchmark Desktop/Server Class (28 Cores) 2 Cores Only *---------------------------------------------------------------------------- *Helidon Quick Start 3.59x 4.11x ++ JavacBenchApp 50 source files 2.21x 3.17x ++++ Micronaut First App Demo 2.91x 4.90x +++++++ Quarkus Getting Started Demo 2.97x 3.74x +++ Spring-boot Getting Started Demo 4.13x 4.70x ++ Spring PetClinic Demo 3.33x 3.03x -- ???
Changes from last time ( https://github.com/openjdk/leyden/blob/46e03d000efd1b2784ad4dcd4c83310ace498... )
The 2 Core Only case see a lot of improvements with the compiler tuning. PetClinic is again an outlier.
Benchmark Desktop/Server Class (28 Cores) 2 Cores Only ---------------------------------------------------------------------------- Helidon Quick Start 3.60 -> 3.59x ~ 3.53 -> 4.11x ++ JavacBenchApp 50 source files 2.17 -> 2.21x ~ 2.74 -> 3.17x ++ Micronaut First App Demo 2.85 -> 2.91x + 4.39 -> 4.90x ++ Quarkus Getting Started Demo 2.73 -> 2.97x ++ 2.95 -> 3.74x +++ Spring-boot Getting Started Demo 3.96 -> 4.13x ++ 3.70 -> 4.70x +++ Spring PetClinic Demo 3.24 -> 3.33x + 3.04 -> 3.03x ~
(The +, -, ~ are calculated by my eyeballs so not very scientific :-)
participants (2)
-
ioi.lam@oracle.com
-
Sebastien Deleuze