RFR: JDK-8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v19]
Marcus G K Williams
github.com+168222+mgkwill at openjdk.java.net
Mon Mar 15 22:21:13 UTC 2021
On Mon, 15 Mar 2021 15:42:15 GMT, Marcus G K Williams <github.com+168222+mgkwill at openjdk.org> wrote:
>> Hi Markus,
>>
>> first off, starting to look better and better.
>>
>> About the assert, I'm quite sure this is the result of using `os::page_size_for_region_aligned()` - as Stefan mentioned above you should use the unaligned version of this function since the input size 21098496 is not aligned to 2M, which causes the function to return 4K even though we have space enough to fit 9-10 2M pages here:
>>
>> [0.825s][info][pagesize] Large page size returned from os::page_size_for_region_aligned: 4096, for bytes: 21098496
>>
>> Sorry, I don't have time to dive deeper. I know @kstefanj had some more ideas and this ties in with his work, but he may be occupied with other things right now and may not be quick to reply.
>>
>> All in all this starts to look real good. I try to resist the urge to refactor everything on the back of this PR. There will be work left for future RFEs.
>>
>> Cheers, Thomas
>
>> Hi Markus,
>>
>> first off, starting to look better and better.
>>
>> About the assert, I'm quite sure this is the result of using `os::page_size_for_region_aligned()` - as Stefan mentioned above you should use the unaligned version of this function since the input size 21098496 is not aligned to 2M, which causes the function to return 4K even though we have space enough to fit 9-10 2M pages here:
>>
>> ```
>> [0.825s][info][pagesize] Large page size returned from os::page_size_for_region_aligned: 4096, for bytes: 21098496
>> ```
>>
>> Sorry, I don't have time to dive deeper. I know @kstefanj had some more ideas and this ties in with his work, but he may be occupied with other things right now and may not be quick to reply.
>>
>> All in all this starts to look real good. I try to resist the urge to refactor everything on the back of this PR. There will be work left for future RFEs.
>>
>> Cheers, Thomas
>
> Thanks @tstuefe Thomas.
>
> Changed to os::page_size_for_region_unaligned at os::Linux::reserve_memory_special_huge_tlbfs_mixed. This helped. Working on new patch now.
>
> I'd really like to resist too much refactoring here also. I believe @kstefanj is looking at more in depth refactor.
>
> Thanks,
> Marcus
2 Issues:
- Page sizes reported in higher level functions can be differ from page_size actually present. In my current setup 1G large pages are present and os::page_size_for_region_unaligned picks them and uses them even thought 2M pages are set in large_page_init. Should we force large_pages to the largest page configured instead of using largest available? Or should we update how we calculate the large page used?
Even though `Large page size returned from os::page_size_for_region_unaligned: 1073741824, for bytes: 16206790656` and 1G large pages padded with 4k pages is used, trace_page_sizes reports `Heap: min=8M max=15456M base=0x000000043a000000 page_size=2M size=15456M`
[0.022s][info][pagesize] os::Linux::reserve_memory_special_huge_tlbfs_mixed
[0.022s][info][pagesize] Large page size returned from os::page_size_for_region_unaligned: 1073741824, for bytes: 16206790656
[0.022s][info][pagesize] alignment: 8388608
[0.022s][info][pagesize] Page size returned from (size_t)os::vm_page_size(): 4096
[0.022s][info][pagesize] req_addr: 0x000000043a000000
[0.022s][info][pagesize] Memory: 4k page, physical 131844416k(50816280k free), swap 0k(0k free)
[0.022s][info][pagesize] 2048k default large page
[0.022s][info][pagesize] Page Sizes: 4k, 2M, 1G
[0.022s][info][pagesize] start addr: 0x000000043a000000
[0.022s][info][pagesize] LP start addr: 0x0000000440000000
[0.022s][info][pagesize] LP end addr: 0x0000000800000000
[0.022s][info][pagesize] end addr: 0x0000000800000000
[0.022s][info][pagesize] Heap: min=8M max=15456M base=0x000000043a000000 page_size=2M size=15456M
src/hotspot/share/memory/virtualspace.cpp at 286
size_t ReservedSpace::actual_reserved_page_size(const ReservedSpace& rs) {
size_t page_size = os::vm_page_size();
if (UseLargePages) {
// There are two ways to manage large page memory.
// 1. OS supports committing large page memory.
// 2. OS doesn't support committing large page memory so ReservedSpace manages it.
// And ReservedSpace calls it 'special'. If we failed to set 'special',
// we reserved memory without large page.
if (os::can_commit_large_page_memory() || rs.special()) {
// An alignment at ReservedSpace comes from preferred page size or
// heap alignment, and if the alignment came from heap alignment, it could be
// larger than large pages size. So need to cap with the large page size.
page_size = MIN2(rs.alignment(), os::large_page_size());
}
}
- I don't think that test/hotspot/jtreg/runtime/os/TestTracePageSizes.java takes into account mixed large page reservation using os::Linux::reserve_memory_special_huge_tlbfs_mixed where large pages are padded at the start and end with small pages. It seems to me that the test needs to be updated.
Test Log Output with Debug
Added range: [43a400000, 440000000) pageSize=4KB isTHP=false isHUGETLB=false
Added range: [440000000, 800000000) pageSize=1048576KB isTHP=false isHUGETLB=true
...
[0.019s][info][pagesize] Heap: min=8M max=15452M base=0x000000043a400000 page_size=2M size=15452M
>From smaps: [43a400000, 440000000) pageSize=4KB isTHP=false isHUGETLB=false
Failure: 4 != 2048
Extra Debug Logging:
[0.022s][info][pagesize] os::Linux::reserve_memory_special_huge_tlbfs_mixed
[0.022s][info][pagesize] Large page size returned from os::page_size_for_region_unaligned: 1073741824, for bytes: 16206790656
[0.022s][info][pagesize] start addr: 0x000000043a000000
[0.022s][info][pagesize] LP start addr: 0x0000000440000000
[0.022s][info][pagesize] LP end addr: 0x0000000800000000
[0.022s][info][pagesize] end addr: 0x0000000800000000
[0.022s][info][pagesize] Heap: min=8M max=15456M base=0x000000043a000000 page_size=2M size=15456M
Function where the compare of address and page size allocation:
test/hotspot/jtreg/runtime/os/TestTracePageSizes.java at 195
if (pageSizeFromSmaps != pageSizeFromTrace) {
if (pageSizeFromTrace > pageSizeFromSmaps && range.isTransparentHuge()) {
// Page sizes mismatch because we can't know what underlying page size will
// be used when THP is enabled. So this is not a failure.
debug("Success: " + pageSizeFromTrace + " > " + pageSizeFromSmaps + " and THP enabled");
} else {
debug("Failure: " + pageSizeFromSmaps + " != " + pageSizeFromTrace);
throw new AssertionError("Page sizes mismatch: " + pageSizeFromSmaps + " != " + pageSizeFromTrace);
}
} else {
debug("Success: " + pageSizeFromSmaps + " == " + pageSizeFromTrace);
}
-------------
PR: https://git.openjdk.java.net/jdk/pull/1153
More information about the hotspot-gc-dev
mailing list