RFR: 8369735: [Leyden] AOT compiled methods have lower peak performance [v3]

[Aleksey Shipilev]

> This should get us back on track with peak performance.
> 
> Recently, I fixed the recent regression in compilation times: https://github.com/openjdk/leyden/commit/b8cfee49c6cb60225b13cbe3dc57a37975125385 -- caused by https://github.com/openjdk/leyden/commit/7b7648a4c9f67be509c6fccbcbc0502648388fdc. In doing so, I dropped the `is_initialized()` check from `notice_jit_observation()`, because I noticed it filtered out too many dependencies. So the net result was that our dependencies are now _overly conservative_, which means our JIT compilation times are great, but we are stuck in AP4 code without switching to A4. I instantiated the check back.
> 
> _That_ gets us back to original regression I have been fixing (https://github.com/openjdk/leyden/commit/7b7648a4c9f67be509c6fccbcbc0502648388fdc), so I took another look what goes wrong there. And I think I figured it out: most of the uncommon traps are from `invokedynamic` call-sites that get uncommon-trapped (at `Parse::can_not_compile_call_site`) in A4 code, I believe because LF invokers (like `j.l.invoke.Invokers$Holder`) are not in dependencies! Oops. So I added the code that reports core JLI classes as fully initialized for A4 code, which I believe they are in production run. This seems to stop premature AP4 -> A4 switch in all workloads I tried.
> 
> Plus, I took Vladimir's patch to replace AP4 -> A4 code even when there are no clinit barriers. We reasoned it is a right thing to do, because AP4 is compiled without uncommon traps and other speculations. This whole ordeal shows that AP4 is indeed significantly slower than A4.
> 
> Plus, in Spring Boot Petclinic I noticed that we still JIT compiling quite a bit of code, which meant `SkipTier2IfPossible` opt-in that we recently did (https://github.com/openjdk/leyden/commit/3be99e0087c3588693dc7ce0f8d0a0b860ecbbd4) -- got warmup significantly worse there, as we waited for C2 compilations to complete. I took that block out again. We can do this separately, but the improvements this PR gives is on par with winning due to more reasonable tiered compilation.
> 
> Additional testing:
>  - [x] Benchmarks, see comments
>  - [x] Linux x86_64 server fastdebug, `runtime/cds`

Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:

  Touchup comments

-------------

Changes:
  - all: https://git.openjdk.org/leyden/pull/103/files
  - new: https://git.openjdk.org/leyden/pull/103/files/33118207..29d6b243

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=leyden&pr=103&range=02
 - incr: https://webrevs.openjdk.org/?repo=leyden&pr=103&range=01-02

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/leyden/pull/103.diff
  Fetch: git fetch https://git.openjdk.org/leyden.git pull/103/head:pull/103

PR: https://git.openjdk.org/leyden/pull/103