RFR: 8318817: Could not reserve enough space in CodeHeap 'profiled nmethods' (0K) [v8]

[Roberto Castañeda Lozano]

On Mon, 13 Nov 2023 15:55:22 GMT, Daniel Lundén <duke at openjdk.org> wrote:

>> This changeset fixes an issue where certain large values of `-XX:NMethodSizeLimit` could cause code heap initialization to fail with the error message
>> 
>> Error occurred during initialization of VM
>> Could not reserve enough space in CodeHeap 'profiled nmethods' (0K)
>> 
>> 
>> Two components in `CodeCache::initialize_heaps` may cause the error:
>> 1. The minimum size (`min_size = os::vm_page_size();`) for profiled and non-profiled code heaps is sometimes not large enough. Specifically, when `profiled_size` and/or `non_profiled_size` are set to `min_size` and `alignment > os::vm_page_size()`, the `align_down` below (from `CodeCache::initialize_heaps`) sets the size to 0.
>>       ```
>>       const size_t alignment = MAX2(ps, os::vm_allocation_granularity());
>>       non_nmethod_size = align_up(non_nmethod_size, alignment);
>>       profiled_size    = align_down(profiled_size, alignment);
>>       non_profiled_size = align_down(non_profiled_size, alignment);
>>       ```
>> 2. The calculation for default code heap sizes does not consider the edge case when `cache_size > non_nmethod_size > cache_size - 2 * min_size`.
>> 
>> This changeset
>> 1. ensures that `min_size >= alignment`, and
>> 2. fixes the edge case.
>> 
>> The changeset includes new tests that exercise the above, also in combination with `-XX:+UseTransparentHugePages` on Linux  to test the case when `os::can_execute_large_page_memory()` is true (which can affect `alignment`).
>> 
>> ### Testing (on all Oracle-supported platforms)
>> `tier1`, `tier2`, `tier3`, `tier4`, `tier5`
>
> Daniel Lundén has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 10 additional commits since the last revision:
> 
>  - Merge remote-tracking branch 'upstream/master' into nmethodsizelimit-8318817
>  - Add comment for max_non_nmethod_size
>  - Realign comment
>  - Fix double space in comment
>  - Remove min_size = alignment alias
>  - Tidy up check for max non_nmethod_size
>  - Invert incorrect assert
>  - Change to MAX2 and split tests
>  - Remove unnecessary flag
>  - Fix issue

Marked as reviewed by rcastanedalo (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/16373#pullrequestreview-1729092291