RFR: 8326205: Grouping frequently called C2 nmethods in CodeCache [v3]

[Chad Rakoczy]

> ### Summary
> This PR implements [JDK-8326205](https://bugs.openjdk.org/browse/JDK-8326205), introducing experimental support for grouping hot code within the CodeCache.
> 
> ### Description
> The feature works by periodically sampling the execution of C2-compiled methods to identify hot code, then relocating those methods into a dedicated `HotCodeHeap` section of the CodeCache.
> 
> Sampling is performed by the `HotCodeSampler`, which runs on a new dedicated `HotCodeGrouper` thread. The thread wakes up every `HotCodeIntervalSeconds` (default 300s) and collects samples for a duration of `HotCodeSampleSeconds` (default 120s). During each sampling period, it iterates over all Java threads, inspects their last Java frame, obtains the current program counter (PC), and maps it to the corresponding nmethod. This allows the sampler to maintain a profile of the most frequently executed methods.
> 
> The `HotCodeGrouper` uses the sampling data to select methods for grouping. Methods are ranked by sample count to form the candidate set. The grouper then relocates these methods (along with their callees, which has been shown to improve performance on AArch64 due to better branch prediction) into the `HotCodeHeap` in descending order of hotness, continuing until the fraction of samples attributable to hot methods exceeds `HotCodeSampleRatio` (default 0.8). The process continues to ensure that the hot-method ratio remains above the threshold.
> 
> The `HotCodeHeap` is a new code heap segment with a default size of 20% of the non-profiled heap, though this can be overridden. This size was chosen based on the principle that roughly 20% of methods contribute to 80% of the work. Only C2-compiled nmethods are eligible for relocation, and the relocation process leverages existing infrastructure from [JDK-8316694](https://bugs.openjdk.org/browse/JDK-8316694).
> 
> Relocation occurs entirely on the grouper thread and runs concurrently with the application. To maintain correctness, the thread acquires the `CodeCache_lock` and `Compile_lock` during relocation but releases these locks between individual relocations to avoid blocking GC safepoints. Removal of nmethods from the `HotCodeHeap` is handled by the GC.
> 
> ### Performance
> Testing has shown up to a 20% latency reduction in an internal service with a large CodeCache (512 MB). Public benchmark results are forthcoming.
> 
> ### Testing
> * CodeCache tests have been updated to cover the new `HotCodeHeap`.  
> * Added dedicated tests for the `HotCodeGrouper`
>   ...

Chad Rakoczy has updated the pull request incrementally with one additional commit since the last revision:

  Add HotCodeGrouperMoveFunction test

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/27858/files
  - new: https://git.openjdk.org/jdk/pull/27858/files/ce1c685f..3697718f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=27858&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=27858&range=01-02

  Stats: 91 lines in 1 file changed: 91 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/27858.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/27858/head:pull/27858

PR: https://git.openjdk.org/jdk/pull/27858