RFR: 8271195: Use largest available large page size smaller than LargePageSizeInBytes when available [v3]

[Thomas Stuefe]

On Fri, 4 Feb 2022 12:50:49 GMT, Swati Sharma <duke at openjdk.java.net> wrote:

>> Hi Team,
>> 
>> In this patch I have fixed two issues related to large pages, following is the summary of changes :-
>> 
>> 1. Patch fixes existing large page allocation functionality where if a commit over 1GB pages fails allocation should happen over next small page size i.e. 2M where as currently its happening over 4kb pages resulting into significant TLB miss penalty.
>> Patch includes new JTREG Test case covering various scenarios for checking the correct explicit page allocation according ​to the 1G, 2M, 4K priority.
>> 2. While attempting commit over larger pages we first try to reserve requested bytes over the virtual address space, in case commit to large page fails we should be un reserving entire reservation to avoid leaving any leaks in virtual address space.
>> 
>> ​
>> Please find below the performance data with and without patch for the JMH benchmark included with the patch.
>> 
>> ![image](https://user-images.githubusercontent.com/96874289/152189587-4822a4ca-f5e2-4621-b405-0da941485143.png)
>> 
>> 
>> Please review and provide your valuable comments.
>> 
>> 
>> 
>> Thanks,
>> Swati Sharma
>> Runtime Software Development Engineer 
>> Intel
>
> Swati Sharma has updated the pull request incrementally with one additional commit since the last revision:
> 
>   8271195: Review comments resolved.

Hi @swati-sha,

What tests have you run, and on what hardware?

It would be nice if @kstefanj could take a look at this.

> 1. Patch fixes existing large page allocation functionality where if a commit over 1GB pages fails allocation should happen over next small page size i.e. 2M where as currently its happening over 4kb pages resulting into significant TLB miss penalty.

This text (and the description in the associated JBS issue) are a bit confusing. What should be the desired behavior of LargePageSizeInBytes wrt to heap allocation?

Currently, the behavior is different between code cache and heap: 
1) For code cache `LargePageSizeInBytes` means the largest possible page size to use, with the explicit permission to use smaller ones where available. 
2) For the heap, `LargePageSizeInBytes` means "use that or no large pages at all".
Both behaviors could make sense. (2) could make sense if the administrator set up a 2M pool for some other program and now it gets gnawed at by the JVM. Though admittedly that seems contrived. "Use that or fail" would make more sense.

Unfortunately, the java man page is no help:


LargePageSizeInBytes
Sets the maximum large page size (in bytes) used by the JVM.


... could mean either 1 or 2.

The code cache side we had extensively discussed when reviewing https://bugs.openjdk.java.net/browse/JDK-8256155. 

> 2. While attempting commit over larger pages we first try to reserve requested bytes over the virtual address space, in case commit to large page fails we should be un reserving entire reservation to avoid leaving any leaks in virtual address space.

This part of the PR text is confusing. I do not find this in the patch. Does the text match the code? Note that it would make sense in a separate bug fix.

Cheers, Thomas

-------------

PR: https://git.openjdk.java.net/jdk/pull/7326