RFR: 8369735: [Leyden] AOT compiled methods have lower peak performance

[Aleksey Shipilev]

On Thu, 16 Oct 2025 12:50:35 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> This should get us back on track with peak performance.
> 
> Recently, I fixed the recent regression in compilation times: https://github.com/openjdk/leyden/commit/b8cfee49c6cb60225b13cbe3dc57a37975125385 -- caused by https://github.com/openjdk/leyden/commit/7b7648a4c9f67be509c6fccbcbc0502648388fdc. In doing so, I dropped the `is_initialized()` check from `notice_jit_observation()`, because I noticed it filtered out too many dependencies. So the net result was that our dependencies are now _overly conservative_, which means our JIT compilation times are great, but we are stuck in AP4 code without switching to A4. I instantiated the check back.
> 
> _That_ gets us back to original regression I have been fixing (https://github.com/openjdk/leyden/commit/7b7648a4c9f67be509c6fccbcbc0502648388fdc), so I took another look what goes wrong there. And I think I figured it out: most of the uncommon traps are from `invokedynamic` call-sites that get uncommon-trapped (at `Parse::can_not_compile_call_site`) in A4 code, I believe because LF invokers (like `j.l.invoke.Invokers$Holder`) are _not fully initialized_ in assembly phase! Oops. So I added the code that reports core JLI classes as fully initialized for A4 code, which I believe they are in production run. This seems to stop premature AP4 -> A4 switch in all workloads I tried.
> 
> Plus, I took Vladimir's patch to replace AP4 -> A4 code even when there are no clinit barriers. We reasoned it is a right thing to do, because AP4 is compiled without uncommon traps and other speculations. This whole ordeal shows that AP4 is indeed significantly slower than A4.
> 
> Plus, in Spring Boot Petclinic I noticed that we still JIT compiling quite a bit of code, which meant `SkipTier2IfPossible` opt-in that we recently did (https://github.com/openjdk/leyden/commit/3be99e0087c3588693dc7ce0f8d0a0b860ecbbd4) -- got warmup significantly worse there, as we waited for C2 compilations to complete. I took that block out again. We can do this separately, but the improvements this PR gives is on par with winning due to more reasonable tiered compilation.
> 
> Additional testing:
>  - [ ] Benchmarks, see comments
>  - [x] Linux x86_64 server fastdebug, `runtime/cds`

Profile occupancies for `JavacBench2` shows the issue pretty directly: we are stuck at `AP4` before, and now we are able to switch to `A4`, plus there are no significant JIT compilations happening.

Before:

<img width="1800" height="1350" alt="plot-java-before" src="https://github.com/user-attachments/assets/c53f03cc-8e31-4e2e-8c83-1f3736b938d9" />

After:

<img width="1800" height="1350" alt="plot-javac-after" src="https://github.com/user-attachments/assets/c42bf202-5b40-4938-b3e0-ff2c3c1b27cd" />

Same thing with SpringBoot petclinic:

Before:
<img width="1800" height="1350" alt="plot-petclinic-before" src="https://github.com/user-attachments/assets/3473b71b-caff-44fc-bc2a-6c48dab17be8" />

After:
<img width="1800" height="1350" alt="plot-petclinic-after" src="https://github.com/user-attachments/assets/d08d7d26-1642-4df2-a8b9-296cbfe7616b" />

-------------

PR Comment: https://git.openjdk.org/leyden/pull/103#issuecomment-3411240921
PR Comment: https://git.openjdk.org/leyden/pull/103#issuecomment-3411251259