RFR: 8330174: Protection zone for easier detection of accidental zero-nKlass use [v5]

[Ioi Lam]

On Mon, 27 Jan 2025 09:32:49 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> src/hotspot/share/cds/archiveBuilder.cpp line 370:
>> 
>>> 368:     // The region that will be located at the bottom of the encoding range at runtime shall have
>>> 369:     // space for a protection zone.
>>> 370:     MetaspaceShared::allocate_and_mark_protection_zone(rw_region());
>> 
>> This will add up 64KBs of space to the front of the RW region, increasing the file size. Could you try changing the `CDSFileMapRegion::_mapping_offset` of the RW region to `MetaspaceShared::core_region_alignment()` so we can avoid storing the data in the CDS file?
>
> @iklam Ugh, this is more invasive than I thought. I worked my way through the belly of `MetaspaceShared::reserve_address_space_for_archives`, fixing up things for an explicit protection zone at the beginning. 
> 
> This gets complicated because of Windows and its inability to replace memory mappings. So I need to reserve space for the protection zone as an individual ReservedSpace, including fixing `release_reserved_spaces` etc. The final splitup for the Posix variant is also getting more complex.
> 
> Could we not just deprecate the "Windows mapping and use_archive_base_addr=true path" ? Does anyone still benefit since we, by default, look for a randomized base anyway?
> 
> I will probably have to delay this work until mid-February. Unless you would be okay with me to push it in its current form and do the improvement for jsa file size later.

Since this is a diagnostic feature, I think there's no rush and it's better to do it the right way.

> This gets complicated because of Windows and its inability to replace memory mappings.

`os::protect_memory()` calls `VirtualProtect()` which can change the permission to `PAGE_NOACCESS`.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23190#discussion_r1931341559