RFR: 8319795: Static huge pages are not used for CodeCache

[Evgeny Astigeevich]

On Fri, 17 Nov 2023 06:57:12 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>>> Hm... I looked into `CodeCache::page_size`, its history and uses. The function smells. It has an interesting hack for the large page case:
>>> 
>>> ```
>>>   if (os::can_execute_large_page_memory()) {
>>>     if (InitialCodeCacheSize < ReservedCodeCacheSize) {
>>>       // Make sure that the page size allows for an incremental commit of the reserved space
>>>       min_pages = MAX2(min_pages, (size_t)8);
>>>     }
>>> ```
>>> 
>>> The uses are:
>>> 
>>>     * 2 of `page_size(false, 8)`
>>> 
>>>     * 1 of `page_size(false, 1)`
>>> 
>>>     * 1 of `page_size(true, 1)`
>>> 
>>> 
>>> This looks strange to me. I need to if everything is correct.
>> 
>> I'm not the original author, but my guess is this tries to prevent that if the large page size is very large, e.g. 1G, it should use smaller page sizes, e.g. 2M or 4K, because otherwise a CodeCacheSize of 1G would allocate 1x1GB page, and that would be fully committed right from the start and increase memory footprint.
>> 
>> Seems like an odd optimization though. It feels a bit arbitrary since we don't do this for the java heap, for example. Also, it prevents users from ever using e.g. 1G pages for code cache if they really want that.
>> 
>> @TobiHartmann ?
>> 
>> As for the `if (os::can_execute_large_page_memory()) {`, I believe that was slightly wrong. It feels like it wants to guard the large-page case, but on Windows "can_execute_large_page_memory" is always true, regardless of UseLargePages. And now, on Linux, it is also always true.
>> 
>> I would just swap that with `if (UseLargePages)`.
>
>> @tstuefe, @TobiHartmann, `InitialCodeCacheSize` is useless when static huge pages are used. When static huge pages are reserved, they are committed: https://github.com/openjdk/jdk/blob/master/src/hotspot/share/memory/virtualspace.cpp#L248
>> 
>> ```
>>   if (use_explicit_large_pages(page_size)) {
>>     // System can't commit large pages i.e. use transparent huge pages and
>>     // the caller requested large pages. To satisfy this request we use
>>     // explicit large pages and these have to be committed up front to ensure
>>     // no reservations are lost.
>>     do {
>> ```
> 
> Yes. For that case, the "lets at least do 8 pages so that we can commit as we go" logic makes no sense, and we may just as well live with one page, e.g. if pagesize = 1GB. I think that was what @TobiHartmann alluded to.
> 
> There is still the question of THPs. There, this logic still makes theoretical sense since THPs can be committed and uncommitted. But THPs (for now) only come in 2MB size. And as we established earlier, said logic contradicts what the java heap does.
> 
> My vote goes for just removing that logic.

Hi @tstuefe, @TobiHartmann,
I'd like to discuss the changes I've made in `CodeCache::initialize` to support different pages sizes.
A problem with large pages is that the final page size is unknown till a successful reservation. A page size before a successful reservation is a preferred page size. 
Based on this I changed `CodeCache::initialize`:
1. Calculate a preferred page size. By default, use the one which allows magical 8 pages at least. If large pages are requested, the largest  page is used only if a user has specified `ReservedCodeCacheSize` in cmd. Most users use large pages for the Java heap not for CodeCache. So for large pages more than 2M, e.g. 1G, users will need to specify  `ReservedCodeCacheSize` .
2. Try to enable the segmented CodeCache. I moved the code from `CompilerConfig`. I think this is CodeCache responsibility to be or not to be segmented.
3. Initialise code heaps sizes.
4. As the preferred page size calculated in Step 1 might not be suitable for code heaps, we need to recalculate it with respect of code heaps sizes.
5. We reserve memory.
6. We create code heaps.  As the special reserved  space for CodeCache is fully committed (true for explicit huge pages), the initial sizes of code heaps is set to reserved sizes.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16582#issuecomment-1830398625