RFR: 8234930: Use MAP_JIT when allocating pages for code cache on macOS [v3]

[Thomas Stuefe]

On Tue, 6 Oct 2020 21:03:34 GMT, Anton Kozlov <akozlov at openjdk.org> wrote:

> 
> Could you explain how the choice between SysV and mmap is made on AIX? It looks like
> 
> ```
> develop(uintx, Use64KPagesThreshold, 0,                                           \
>           "4K/64K page allocation threshold.")                                      \
> ...
>   if (os::vm_page_size() == 4*K) {
>     return reserve_mmaped_memory(bytes, NULL /* requested_addr */);
>   } else {
>     if (bytes >= Use64KPagesThreshold) {
>       return reserve_shmated_memory(bytes, NULL /* requested_addr */);
>     } else {
>       return reserve_mmaped_memory(bytes, NULL /* requested_addr */);
>     }
>   }
> ```
> 
> (there only two calls to reserve_shmated_memory and both of them are like above. Is SysV SHM used in product builds?)
> For now, the AIX case looks a bit different. The choice is made by the platform and the shared code cannot control
> this. So yes, I cannot see how to avoid handle_t or similar.

On AIX we have 4K and 64K pages (actually more but those are interesting). 64K pages are desireable for larger areas
like heap. 64K pages can only be allocated with SystemV shared memory. mmap'ed memory is always 4K paged. But SystemV
shared mem has a number of disadvantages, like inability to protect the memory, and a large attach alignment (256M). So
it is cumbersome.

os::vm_page_size() on AIX is a fake. The hotspot code assumes that the underlying Operating System has some sort of
"base page size" (usually what is returned by sysconf(_SC_PAGESIZE)), and then optionally some sort of huge page size
which follows different rules (e.g. pinned). On Aix things are more fluid.

When investigating 64K page support on AIX I decided eventually to fool hotspot into thinking that the base page size
is 64k. Long story, this was way before the OpenJDK existed and this was a propietary code base with no possibilty of
changing things upstream. Therefore os::vm_page_size returns 64K ("64K fake mode"). This can be disabled.

So above code fragment uses mmaped memory if 64K fake mode is disabled, and if it is enabeld, it uses mmap for smaller
regions and shmget for larger ones.

> 
> In contrast, THP and MAP_JIT are the way to implement a request from the shared code. Even for THP, shared code seems
> to know why it should "realign" (not sure why commit has an alignment_hint parameter, while it is possible to realign
> after a regular commit). I assume there is enough context in the shared code that can be provided for platform
> functions, without a handle_t. And the same context should anyway be provided to reserve function, so handle_t can be
> filled with all necessary information.

I believe the alignment hint and the TPH code had their roots in Solaris code. So its current form (I guess) is heavily
warped by history. A new implementation would maybe just have a "os::set_tph(start, size)" function and leave it at
that. And yes, I do not think it is necessary for os::commit to do this.

In fact, Linux could probably set TPH unconditionally always when UseTransparentHugePages is active. That would
alleviate the need for the alignment_hint parameter and the realign function.

I opened https://bugs.openjdk.java.net/browse/JDK-8253890 to follow up on this.

-------------

PR: https://git.openjdk.java.net/jdk/pull/294