RFR: 8367332: Replace BlockTree tree logic with an intrusive red-black tree

[Casper Norrbin]

On Thu, 11 Sep 2025 09:40:21 GMT, Casper Norrbin <cnorrbin at openjdk.org> wrote:

> Hi everyone,
> 
> Metaspace's `BlockTree` is currently implemented as a simple binary search tree, using the memory blocks it stores as the nodes. While this is straightforward and generally fine, it can become unbalanced depending on insert/remove order, leading to degraded performance.
> 
> This is a good fit for the utilities `IntrusiveRBTree`, instead of using internal tree logic. With it, we can replace all tree functions and instead rely on a few insert/remove calls. `BlockTree` still remains as the layer coordinating these blocks, and still handles the list of same-size nodes. The intrusive red-black tree only handles balancing and linking/unlinking nodes in the tree structure. 
> 
> Validation and printing are preserved by using the hooks provided by the red-black tree. We keep the same checks (and get a few more from the rb-tree), and the printed output is kept identical. The only real difference is that the red-black tree traverses the tree instead.
> 
> Testing:
> - Oracle tiers 1-8

src/hotspot/share/memory/metaspace/blockTree.hpp line 59:

> 57: //           +-----+         +-----+
> 58: //           | 80  |         | 120 |
> 59: //           +-----+         +-----+

Small note: This extra node was added to the ascii tree since without it the tree isn't a valid red-black tree.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/27212#discussion_r2339847969