RFR: 8234930: Use MAP_JIT when allocating pages for code cache on macOS [v6]

[Thomas Stuefe]

On Fri, 4 Dec 2020 15:30:09 GMT, Anton Kozlov <akozlov at openjdk.org> wrote:

> > 1. mmap(MAP_FIXED|MAP_NORESERVE, PROT_NONE)
> > 2. madvise(MADV_FREE) + mprotect(PROT_NONE)
> > 
> > Or (2) does not work, as you claim. Then why bother at all?
> 
> Right, (2) is an actual state. It does not work like we'd want to (at least not immediately accounted in RSS). But the OS provides this interface and claims to react in some way.
> 
> > ```
> >  MADV_FREE        Indicates that the application will not need the information contained in this address range, so the pages may be reused right away.  The address range will remain valid.  This is used with madvise() system call.
> > ```
> 
> It enables the OS release the memory sometime later, for example. In contrast, doing nothing will keep the memory that is garbage.
>
> > I wonder if [MAP_NORESERVE] gets even honored on MacOS
> 
> Right, it's defined and not used, the only occurrence is https://github.com/apple/darwin-xnu/blob/0a798f6738bc1db01281fc08ae024145e84df927/bsd/sys/mman.h#L116
> 
> The use of MAP_NORESRVE slipped in from the original BSD pd_commit_memory code. I can delete it, if it bothers.
> 
> > If MAP_NORESERVE has no meaning, we do not need to call mmap() for committing and uncommitting; mprotect, maybe combined with madvise(MADV_FREE), should suffice.
> 
> I could not follow, sorry. Why it's so?
> 

Your original chain of thought, if I understand you correctly, was like this:
- We want to provide MAP_JIT on a mapping. Apple tells us to.
- On reserve, we add it to the initial mmap call. Easy.
- But hotspot later - commit/uncommit - replaces that mapping again. With subsequent mmap calls. On those, MAP_FIXED is specified since the original mapping gets replaced.
- But on those secondary mmap calls we cannot add MAP_JIT. Because it cannot be combined with MAP_FIXED.
- Therefore we switch to: on commit, do mprotect(RW). On uncommit, madvise(MADV_FREE) + mprotect(NONE).

Am I right so far?

My thought is this: 

In existing code, the technique used to commit and uncommit memory is to switch the MAP_NORESERVE flag on the mapping, combined with a change in protection.

E.g. mmap(MAP_NORESERVE, PROT_NONE) to uncommit. No documentation says this releases memory. But from experience we see it work on Linux - the kernel takes the hint that the pages are not needed anymore.

Changing the MAP_NORESERVE flag is the only reason we have those subsequent mmap calls. Those that need MAP_FIXED. And conflict with MAP_JIT.

If MAP_NORESERVE is a noop on Mac, the only other effect those mmap calls have is the protection change: in uncommit, the mmap(MAP_NORESERVE|MAP_FIXED, PROT_NONE) would be equivalent to a mprotect(PROT_NONE). Since todays os::uncommit_memory() works on MacOS (at least I assume it does??), then that PROT_NONE protection change must be the reason for it working, since the MAP_NORESERVE is a noop.

So, if MAP_NORESERVE is a noop, there is no reason to have secondary calls to mmap(MAP_FIXED), which conflict with MAP_JIT. Then, the code you proposed for the exec=true path should work equally well for the standard path. That would make the coding a lot easier.

Cheers, Thomas

-------------

PR: https://git.openjdk.java.net/jdk/pull/294