RFR: 8307958: Metaspace verification is slow causing extreme class unloading times [v2]

[Coleen Phillimore]

On Mon, 22 May 2023 15:59:04 GMT, Axel Boldt-Christmas <aboldtch at openjdk.org> wrote:

>> From JBS:
>>>We see very long class unloading times with Generational ZGC in fastdebug builds. MacOS spindumps show that it is caused by Metaspace verification code.
>>>
>>>I think the main problem is that many of the verification operations are performed at every operation (add, remove, split, merge) and the individual operation verifies much more than a constant number of chunks. This seems to give a quadratic complexity which blows up if you don't class unload often enough.
>> 
>> The approach here is to resolve this by putting the verification behind the SOMETIMES macro. It is then possible to turn it of completely with `-XX:VerifyMetaspaceInterval=0` while still benefiting from some stochastic verification. 
>> 
>> The goal here is to have the calls to Verify unchanged. Leaving all explicit gtest verify and Universe::verify calls unaffected.  
>> The verification was also left unchanged when committing the memory. 
>> 
>> All SOMETIMES uses the same frequency, `VerifyMetaspaceInterval`. I looked at the possibility of adding weight to some of the call site, but it did not seem to be worth the complexity. 
>> 
>> Testing: GHA and Oracle CI (tier1-3 ongoing), various long running stress tests.
>
> Axel Boldt-Christmas has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Always verify gtests -XX:VerifyMetaspaceInterval=1
>  - Revert "Fix gtest"
>    
>    This reverts commit 5991e03fde88d9262778ea460067139b38ed497a.

This seems fine.  To be clear, was most of the time spent in the ChunkManager::get_chunk() and return_chunk(), and maybe purge() calls, and not much in the split and merge chunks?  I agree with adding SOMETIMES to all for consistency, but I hope we're not doing more of these operations that I assume should be infrequent.

-------------

Marked as reviewed by coleenp (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/14084#pullrequestreview-1437211020