RFR: 8253638: Cleanup os::reserve_memory and remove MAP_FIXED

[Stefan Karlsson]

On Fri, 25 Sep 2020 13:13:27 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> There's a code path through os::reserve_memory that uses MAP_FIXED. This path is used if a requested address is given.
>> It's very dangerous to use MAP_FIXED, since it causes pre-existing mappings to be replaced. Alternative, move safe,
>> functions exists; named os_attempt_reserve_memory_at.  It's not only dangerous to use that API, the Mac AARCH64 port
>> also can't use it in conjunction with its MAP_JIT.
>> I started to split os::reserve_memory into two functions; one that doesn't care about were the memory is mapped
>> (os::reserve_memory), and another that uses the dangerous MAP_FIXED flag (os::reserve_memory_at). However, I noticed
>> that it's only windows code that actually use os::reserve_memory with a requested address. All other usages have been
>> cleaned out and replaced with os::attempt_reserve_memory. And on windows os::attempt_reserve_memory uses
>> os::reserve_memory with comments that it's safe because it only "attempts" to place the mapping.  So, this patch: 1)
>> Removes the forcefully reserving of memory and its MAP_FIXED usages! 2) Changes the windows code to call the
>> attempt_reserve_memory_at version, that it actually did anyway. 3) Flips the call order between pd_reserve_memory and
>> pd_attempt_reserve_memory_at, on windows. 4) There's also some unification and split done to handle MEMFLAGS and fds.
>> This part needs followup cleanups (IMHO). 5) The AIX implementation already refused to use MAP_FIXED, and now it
>> doesn't need special code to handle that. 6) Small cleanups around anon_mmap and anon_mmap_aligned
>
> Hi Stefan,
> 
> very nice cleanup! I had a similar change in work for some time now, but your work made that obsolete. See also
> https://bugs.openjdk.java.net/browse/JDK-8079351, which now can be closed as duplicate of your bug.
> Most of my remarks are idle nits; I leave it up to you what you take from them.

Thanks @tstuefe for reviewing this. You need to click the "Files changed" and then "Review changes" to formally review
this. However, since you seem to indicate that you have review I'll see if I can add you.

-------------

PR: https://git.openjdk.java.net/jdk/pull/357