RFR: JDK-8273783: Simplify Metaspace arena guard handling [v3]

[Coleen Phillimore]

On Sat, 18 Sep 2021 05:36:28 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Hi,
>> 
>> may I please have reviews for this patch.
>> 
>> This simplifies metaspace arena guard handling and reduces the complexity guard mode brings to metaspace and its tests. It makes future modifications easier (esp. wrt Lilliput).
>> 
>> ----
>> 
>> Metaspace has a guard mode which I added as part of JEP 387. It means that metaspace allocations are bracketed by canaries which, if broken, indicate a buffer overrun in metaspace. It can be switched on and off using -XX:+MetaspaceGuardAllocations, is by default off, and a debug-only feature.
>> 
>> I was always a bit unhappy with the implementation though since it seemed too complex and one needed a bit of lateral thinking when looking at arenas. This is because guard prefixes change block size, cause problems with deallocations, require thinking about user pointer alignment etc.
>> 
>> When investigating how to compress compressed class pointers beyond 32-bit for Lilliput, working around all that special handling became too complicated and therefore I'd like a simpler solution.
>> 
>> The new mechanism uses the fact that Metaspace is an arena allocator and we therefore know that (almost) all subsequent allocations are adjacent to each other in memory. Therefore, instead of adding guards as prefixes/suffixes with each allocation, we can just allocate the guard block as a separate block after each user allocation, as if the guard were just another user allocation. 
>> 
>> The result is exactly the same as before: all real allocations are separated in memory by a guard. But to most of the metaspace, these guard blocks are just normal allocations. A lot of special handling can be removed. And we don't have to disable deallocation handling in guard mode.
>> 
>> The guard blocks are chained together via a simple pointer list and verifying them just means walking that list and examining the eyecatcher. Note that the guard has two eyecatchers, bracketing the next pointer between them - that way we don't have to do a sanity check on the pointer since an overwriter would likely have overwritten one of the eyecatchers before corrupting the pointer.
>> 
>> Small caveats:
>> - the first block in a chunk has no leading fence
>> - the last block in a chunk *may* have no trailing fence
>> - the guard now costs 24 bytes, up from 16 before.
>> 
>> Note that all these caveats could be easily solved with a bit more intricate bit fiddling. However, I opted for simplicity here. Guard mode is only activated on demand, if one suspects a problem, and I think it works good enough this way.
>> 
>> Tests:
>> - Nightlies at SAP
>> - manual tests on x64 and x86 with and without guards
>> - manual gtests
>> - manual metaspace jtreg tests
>> 
>> Thanks, Thomas
>
> Thomas Stuefe has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
> 
>  - Merge
>  - Reduce diff and improve comments
>  - JDK-8273783-Simplify-metaspace-arena-guard-handling

This looks good, and much simpler!

-------------

Marked as reviewed by coleenp (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/5518