RFR: 8271195: Use largest available large page size smaller than LargePageSizeInBytes when available [v3]

Stefan Johansson sjohanss at openjdk.java.net
Tue Feb 15 16:20:14 UTC 2022 

On Thu, 10 Feb 2022 10:09:24 GMT, Swati Sharma <duke at openjdk.java.net> wrote:

>>> > I think my example, that a 128m heap is aligned up to 512m if the large page size is 512m, is a case that could be considered a bug, but it is not crystal clear. Cause the user have specified both that it wants large pages but also a heap size of 128m.
>>> 
>>> I remember us discussing this recently: https://bugs.openjdk.java.net/browse/JDK-8267475
>>> 
>> Thanks for digging out the JBS issue for this.
>> 
>>> > Which of these are more important? On the other, if we could satisfy the heap of 128m using 2m pages we would be closer to what I would see as the correct solution. This would be achieved today by setting `LargePageSizeInBytes=2m`.
>>> 
>>> I would actually like the following behavior:
>>> 
>>>     * LargePageSizeInBytes is the largest page size usable by the VM. It is free to choose whatever it likes but should give preference to larger page sizes if possible
>>> 
>>>     * when reserving a region and UseLargePages=true, use the largest page size possible which fulfills the size requirement (and possibly the alignment requirement if a wish address was specified).
>>> 
>>> 
>>> I think this is close to what we do now.
>>> 
>>> So, `-XX:+UseLargePages -XX:LargePageSizeInBytes=1G -Xmx1536m -XX:ReservedCodeCacheSize=256m` would use
>>> 
>>>     * a single 1G page and 256 2m pages for the heap
>>> 
>> 
>> Currently this would round the heap up to 2G and use 2 1G pages, right? But we've discussed doing something like this in the past and I think it would be nice from a perf perspective. But there are some issue, say that the 2m commit above fail, then the whole reservation need to be restarted (because we can't guarantee that we still have the range reserved), and should we then try to just use fewer 2m page or directly revert down to 4k pages. There is also other things that well be affected, for example we have some code in G1 that is tracking the page size of the underlying mapping to know when regions can be truly uncommited (multiple regions sharing one underlying OS page), having the heap consist of multiple page sizes would make this more complicated. 
>> 
>>>     * 128 2m pages for the code cache
>>>       ... and if we ever re-introduce large pages for metaspace, those smallish segments would probably use 2m pages too.
>>> 
>>> 
>>> Open question would be whether we even need LargePageSizeInBytes. Why not simplify and always use the largest possible page size we find available? If there are 1G pages and they would fit into the to-be-reserved address range, we use them. Why would an administrator allow large pages in general, create a 1G page pool, but disallow them?
>> 
>> This would be the simplest and cleanest approach in the code now when we support multiple page sizes. One argument I've heard against this is that an administrator might want to setup a 1G page pool for some other application, like a database, while letting the JVM only use 2M pages. So there might be usecases.
>> 
>> If we would go down this route I also think we should stop caring about what the "default" large page size is as well, and always just use the ones configured. But then there is this question about what is "configured", must a page size pass the sanity test to be considered configured (that is basically what this change proposes). 
>> 
>> Counter question, if we go down that route, would we still have `os::large_page_size()` or should all users always ask for a page size given the size of thier mapping?
>
>> > > I think my example, that a 128m heap is aligned up to 512m if the large page size is 512m, is a case that could be considered a bug, but it is not crystal clear. Cause the user have specified both that it wants large pages but also a heap size of 128m.
>> > 
>> > 
>> > I remember us discussing this recently: https://bugs.openjdk.java.net/browse/JDK-8267475
>> 
>> Thanks for digging out the JBS issue for this.
>> 
>> > > Which of these are more important? On the other, if we could satisfy the heap of 128m using 2m pages we would be closer to what I would see as the correct solution. This would be achieved today by setting `LargePageSizeInBytes=2m`.
>> > 
>> > 
>> > I would actually like the following behavior:
>> > ```
>> > * LargePageSizeInBytes is the largest page size usable by the VM. It is free to choose whatever it likes but should give preference to larger page sizes if possible
>> > 
>> > * when reserving a region and UseLargePages=true, use the largest page size possible which fulfills the size requirement (and possibly the alignment requirement if a wish address was specified).
>> > ```
>> > 
>> > 
>> >     
>> >       
>> >     
>> > 
>> >       
>> >     
>> > 
>> >     
>> >   
>> > I think this is close to what we do now.
>> > So, `-XX:+UseLargePages -XX:LargePageSizeInBytes=1G -Xmx1536m -XX:ReservedCodeCacheSize=256m` would use
>> > ```
>> > * a single 1G page and 256 2m pages for the heap
>> > ```
>> 
>> Currently this would round the heap up to 2G and use 2 1G pages, right? But we've discussed doing something like this in the past and I think it would be nice from a perf perspective. But there are some issue, say that the 2m commit above fail, then the whole reservation need to be restarted (because we can't guarantee that we still have the range reserved), and should we then try to just use fewer 2m page or directly revert down to 4k pages. There is also other things that well be affected, for example we have some code in G1 that is tracking the page size of the underlying mapping to know when regions can be truly uncommited (multiple regions sharing one underlying OS page), having the heap consist of multiple page sizes would make this more complicated.
>> 
>> > ```
>> > * 128 2m pages for the code cache
>> >   ... and if we ever re-introduce large pages for metaspace, those smallish segments would probably use 2m pages too.
>> > ```
>> > 
>> > 
>> >     
>> >       
>> >     
>> > 
>> >       
>> >     
>> > 
>> >     
>> >   
>> > Open question would be whether we even need LargePageSizeInBytes. Why not simplify and always use the largest possible page size we find available? If there are 1G pages and they would fit into the to-be-reserved address range, we use them. Why would an administrator allow large pages in general, create a 1G page pool, but disallow them?
>> 
>> This would be the simplest and cleanest approach in the code now when we support multiple page sizes. One argument I've heard against this is that an administrator might want to setup a 1G page pool for some other application, like a database, while letting the JVM only use 2M pages. So there might be usecases.
>> 
>> If we would go down this route I also think we should stop caring about what the "default" large page size is as well, and always just use the ones configured. But then there is this question about what is "configured", must a page size pass the sanity test to be considered configured (that is basically what this change proposes).
>> 
>> Counter question, if we go down that route, would we still have `os::large_page_size()` or should all users always ask for a page size given the size of thier mapping?
> 
> Hi @kstefanj , @tstuefe 
> 
> Thanks for sharing your views and comments. 
> Please suggest what specific change you want us to do in the patch since it is fixing the already existing functionality.
> 
> Best Regards,
> Swati

@swati-sha, I took a closer look at the test-case and right now it won't work when run on aarch64? On those systems in our test environment the base page size is 64k and the large page sizes are 2m and 512m. I took it for a spin to see what failures we would get and here it is:

[0.001s][info][pagesize] LargePageSizeInBytes is not a valid large page size (1G) using the default large page size: 512M
[0.001s][info][pagesize] Usable page sizes: 64k, 2M, 512M
[0.001s][info][pagesize] Large page size (512M) failed sanity check, checking if smaller large page sizes are usable
[0.001s][warning][pagesize] UseLargePages disabled, no large pages configured and available on the system.
[0.002s][info   ][pagesize] CodeHeap 'non-nmethods':  min=2496K max=5760K base=0x0000fffec55d0000 page_size=64K size=5760K
[0.002s][info   ][pagesize] CodeHeap 'profiled nmethods':  min=2496K max=120000K base=0x0000fffec5b70000 page_size=64K size=120000K
[0.002s][info   ][pagesize] CodeHeap 'non-profiled nmethods':  min=2496K max=120000K base=0x0000fffecd0a0000 page_size=64K size=120000K
[0.002s][info   ][pagesize] Heap:  min=1G max=2G base=0x0000000080000000 page_size=64K size=2G
[0.002s][info   ][pagesize] Card Table:  min=4194305B max=4194305B base=0x0000fffee0db0000 page_size=64K size=4160K
[0.003s][info   ][pagesize] Mark Bitmap:  min=64M max=64M base=0x0000fffebc000000 page_size=64K size=64M
[0.003s][info   ][pagesize] Parallel Compact Data:  min=160K max=160K base=0x0000fffee08b0000 page_size=64K size=192K
[0.003s][info   ][pagesize] Parallel Compact Data:  min=4M max=4M base=0x0000fffee04b0000 page_size=64K size=4M
System does not support 1G pages
Number of 2M pages = 0

System does not support 1G pages
Number of reserved 1G pages = -1

Number of reserved 2M pages = 0

TestCase1 skipped

TestCase2 skipped

Exceptionjava.lang.RuntimeException: System property 'test.jdk' not set. This property is normally set by jtreg. When running test separately, set this property using '-Dtest.jdk=/path/to/jdk'.
TestCase4 skipped

----------System.err:(11/560)----------
java.lang.AssertionError: Failed 4K page allocation

	at runtime.os.TestExplicitPageAllocation.main(TestExplicitPageAllocation.java:81)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:577)
	at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:312)
	at java.base/java.lang.Thread.run(Thread.java:828)


So either we need to make the test more robust to also work without having hard coded page sizes but rather figure them out during the "setup" stage or we need to add a requires to avoid running on aarch64. I would prefer if the test was fixed to get more coverage and testing with different page sizes. Doing the requires could look something like: 

@requires os.arch=="amd64" | os.arch=="x86_64"

-------------

PR: https://git.openjdk.java.net/jdk/pull/7326

More information about the hotspot-runtime-dev mailing list