RFR: 8340103: Add internal set_flag function to VMATree [v22]

[Johan Sjölen]

On Thu, 7 Nov 2024 10:58:45 GMT, Johan Sjölen <jsjolen at openjdk.org> wrote:

>> Johan Sjölen has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - Remove external API, use small helper function instead
>>  - Do not modify input args
>
> Mac OSX failure unrelated to PR.

> @jdksjolen " Last test didn't actually make any sense Since the VMATree will remove the released range when it recognises that it is a no-op"
> 
> not so hasty :) The test was fine. You said that set_tag just ignores Released sections in range. So, what should have happened is that in the test, the remaining reserved section should have remained and changed to mtGC. The tree then should consist of `[0, 50) Released, mtNone, nostack) [50, 75) mtGC, si) [75, 100) Released, mtNone, nostack)`.
> 
> In other words, the test would test that calling set_tag across a range that contains Released sections does not change said sections.

I actually am stuck thinking about this, and it's worse than that :-).

Consider a simpler test:

```c++
{
    VMATree tree;
    Tree::RegionData class_shared(si, mtClassShared);
    tree.reserve_mapping(10, 10, class_shared);
    tree.set_tag(0, 100, mtCompiler);
}


This will crash on an assert. After the `reserve_mapping` the state of the tree is `Res,10 - Res,20`. That is: Two reserved nodes, at position 10 and 20.

So, when we run `find_enclosing_range(0)`, we get a non-existent range back, because no range encloses `0`. I'm pretty sure that we can have better behavior by interpreting the piece-wise results of `find_enclosing_range`.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20994#issuecomment-2464255767