RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v27]

[Stefan Johansson]

On Fri, 2 Apr 2021 16:38:02 GMT, Marcus G K Williams <github.com+168222+mgkwill at openjdk.org> wrote:

>> When using LargePageSizeInBytes=1G, os::Linux::reserve_memory_special_huge_tlbfs* cannot select large pages smaller than 1G. Code heap usually uses less than 1G, so currently the code precludes code heap from using
>> Large pages in this circumstance and when os::Linux::reserve_memory_special_huge_tlbfs* is called page sizes fall back to Linux::page_size() (usually 4k).
>> 
>> This change allows the above use case by populating all large_page_sizes present in /sys/kernel/mm/hugepages in _page_sizes upon calling os::Linux::setup_large_page_size().
>> 
>> In os::Linux::reserve_memory_special_huge_tlbfs* we then select the largest large page size available in _page_sizes that is smaller than bytes being reserved.
>
> Marcus G K Williams has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 44 commits:
> 
>  - Merge branch 'master' into update_hlp
>  - Rebase on pull/3073
>    
>    Signed-off-by: Marcus G K Williams <marcus.williams at intel.com>
>  - Merge branch 'pull/3073' into update_hlp
>  - Thomas review.
>    
>    Changed commit_memory_special to return bool to signal if the request succeeded or not.
>  - Self review.
>    
>    Update helper name to better match commit_memory_special().
>  - Marcus review.
>    
>    Updated comments.
>  - Ivan review
>    
>    Renamed helper to commit_memory_special and updated the comments.
>  - 8262291: Refactor reserve_memory_special_huge_tlbfs
>  - Merge branch 'master' into update_hlp
>  - Addressed kstefanj review suggestions
>    
>    Signed-off-by: Marcus G K Williams <marcus.williams at intel.com>
>  - ... and 34 more: https://git.openjdk.java.net/jdk/compare/f60e81bf...f5bfe224

So now #3073 has been integrated, so you could merge with master to get this change.

I also discussed the meaning of `LargePageSizeInBytes` with some folks internally and @stefank pointed me to:
https://docs.oracle.com/en/java/javase/16/docs/specs/man/java.html

There the flag is defined as:

-XX:LargePageSizeInBytes=size
    Sets the maximum size (in bytes) for large pages used for the Java heap. The size argument must be
    a power of 2 (2, 4, 8, 16, and so on). Append the letter k or K to indicate kilobytes, m or M to 
    indicate megabytes, or g or G to indicate gigabytes. By default, the size is set to 0, meaning that 
    the JVM chooses the size for large pages automatically. ...


So with this and what's in the code, the default behavior for the VM should be to try to use the best suitable large page size available for a mapping. If `LargePageSizeInBytes` is set this will limit the page sizes used to only include the ones less than or equal to `LargePageSizeInBytes`.

Does that make sense to everyone? 

This will change the behavior compared to the what we have today: 
* Default (`LargePageSizeInBytes=0`)
  - old behavior - use the systems default large page size 
  - new behavior - use all configure large page sizes
* Value set (`LargePageSizeInBytes=X`)
  - old behavior - use only large pages of size X
  - new behavior - use large pages of size X or smaller

So the new behavior better fits what's in the docs, but I still suspect we might need a CSR.

-------------

PR: https://git.openjdk.java.net/jdk/pull/1153