RFR: 8234930: Use MAP_JIT when allocating pages for code cache on macOS [v6]

[Anton Kozlov]

On Fri, 4 Dec 2020 16:12:24 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> > > 1. mmap(MAP_FIXED|MAP_NORESERVE, PROT_NONE)
> > > 2. madvise(MADV_FREE) + mprotect(PROT_NONE)
> > > 
> > > Or (2) does not work, as you claim. Then why bother at all?
> > 
> > 
> > Right, (2) is an actual state. It does not work like we'd want to (at least not immediately accounted in RSS). But the OS provides this interface and claims to react in some way.
> > > ```
> > >  MADV_FREE        Indicates that the application will not need the information contained in this address range, so the pages may be reused right away.  The address range will remain valid.  This is used with madvise() system call.
> > > ```
> > 
> > 
> > It enables the OS release the memory sometime later, for example. In contrast, doing nothing will keep the memory that is garbage.
> > > I wonder if [MAP_NORESERVE] gets even honored on MacOS
> > 
> > 
> > Right, it's defined and not used, the only occurrence is https://github.com/apple/darwin-xnu/blob/0a798f6738bc1db01281fc08ae024145e84df927/bsd/sys/mman.h#L116
> > The use of MAP_NORESRVE slipped in from the original BSD pd_commit_memory code. I can delete it, if it bothers.
> > > If MAP_NORESERVE has no meaning, we do not need to call mmap() for committing and uncommitting; mprotect, maybe combined with madvise(MADV_FREE), should suffice.
> > 
> > 
> > I could not follow, sorry. Why it's so?
> 
> Your original chain of thought, if I understand you correctly, was like this:
> 
> * We want to provide MAP_JIT on a mapping. Apple tells us to.
> * On reserve, we add it to the initial mmap call. Easy.
> * But hotspot later - commit/uncommit - replaces that mapping again. With subsequent mmap calls. On those, MAP_FIXED is specified since the original mapping gets replaced.
> * But on those secondary mmap calls we cannot add MAP_JIT. Because it cannot be combined with MAP_FIXED.
> * Therefore we switch to: on commit, do mprotect(RW). On uncommit, madvise(MADV_FREE) + mprotect(NONE).
> 
> Am I right so far?

That's correct.

> 
> My thought is this:
> 
> In existing code, the technique used to commit and uncommit memory is to switch the MAP_NORESERVE flag on the mapping, combined with a change in protection.

I read this as MAP_NORESERVE is supposed to be an attribute of the mapping. I don't think this is true. On Linux it specifies, can the system overcommit in statisfying this mapping (https://man7.org/linux/man-pages/man2/mmap.2.html) By default overcommit is enabled and MAP_NORESERVE is also a "noop" on Linux, i.e. it specifies the mode that is enabled for all mappings anyway. This also mean that you can drop MAP_NORESERVE flag from uncommit and it will work anyway, on Linux and macOS.

What definition of MAP_NORESERVE do you use? 

> E.g. mmap(MAP_NORESERVE, PROT_NONE) to uncommit. No documentation says this releases memory. But from experience we see it work on Linux - the kernel takes the hint that the pages are not needed anymore.

It's not a hint. From https://man7.org/linux/man-pages/man2/mmap.2.html:

>  If the memory region specified by addr and len
              overlaps pages of any existing mapping(s), then the overlapped
              part of the existing mapping(s) *will be discarded.*

So one of uncommit's tasks is to release the memory.

> If MAP_NORESERVE is a noop on Mac, the only other effect those mmap calls have is the protection change: in uncommit, the mmap(MAP_NORESERVE|MAP_FIXED, PROT_NONE) would be equivalent to a mprotect(PROT_NONE).

mmap(MAP_NORESERVE, ...) is equal to mmap(...), but it does not mean that mmap(MAP_NORESERVE) is noop.

Also, after you do mprotect(NONE), you can do mprotect(RWX) and get back the original content. But after you do mmap(FIXED) the memory content is discarded, you cannot reverse this operation.

-------------

PR: https://git.openjdk.java.net/jdk/pull/294