RFR: 8234930: Use MAP_JIT when allocating pages for code cache on macOS

[Stefan Karlsson]

On Thu, 24 Sep 2020 16:27:29 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Please review an updated RFR from https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-August/041463.html
>> 
>> On macOS, MAP_JIT cannot be used with MAP_FIXED[1]. So pd_reserve_memory have to provide MAP_JIT for mmap(NULL,
>> PROT_NONE), the function was made aware of exec permissions.
>> For executable and data regions, pd_commit_memory only unlocks the memory with mprotect, this should make no difference
>> compared with old code.
>> For data regions, pd_uncommit_memory still uses a new overlapping anonymous mmap which returns pages to the OS and
>> immediately reflects this in diagnostic tools like ps.  For executable regions it would require MAP_FIXED|MAP_JIT, so
>> instead madvise(MADV_FREE)+mprotect(PROT_NONE) are used. They should also allow OS to reclaim pages, but apparently
>> this does not happen immediately. In practice, it should not be a problem for executable regions, as codecache does not
>> shrink (if I haven't missed anything, by the implementation and in principle).  Tested:
>> * local tier1
>> * jdk-submit
>> * codesign[2] with hardened runtime and allow-jit but without
>> allow-unsigned-executable-memory entitlements[3] produce a working bundle.
>> 
>> (adding GC group as suggested by @dholmes-ora)
>> 
>> 
>> [1] https://github.com/apple/darwin-xnu/blob/master/bsd/kern/kern_mman.c#L227
>> [2]
>>  
>>   codesign \
>>     --sign - \
>>     --options runtime \
>>     --entitlements ents.plist \
>>     --timestamp \
>>     $J/bin/* $J/lib/server/*.dylib $J/lib/*.dylib
>> [3]
>>   <?xml version="1.0" encoding="UTF-8"?>
>>   <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
>>   <plist version="1.0">
>>     <dict>
>>       <key>com.apple.security.cs.allow-jit</key>
>>       <true/>
>>       <key>com.apple.security.cs.disable-library-validation</key>
>>       <true/>
>>       <key>com.apple.security.cs.allow-dyld-environment-variables</key>
>>       <true/>
>>     </dict>
>>   </plist>
>
> I started to look at some of these changes and I have a hunch that we don't ever use MAP_FIXED when reserving memory. I
> have a prototype that cleans that out, but I need more time to complete it and pull it through our testing.

I've now pushed the change to get rid of MAP_FIXED from the reserve memory code:
https://github.com/openjdk/jdk/commit/625a9352bfdc017f426c3bd2a5c24b0f7598ccce

-------------

PR: https://git.openjdk.java.net/jdk/pull/294