RFR: JDK-8266222: [aix] In mmap-mode, partial releases with os::release_memory may trash internal bookkeeping

[Thomas Stuefe]

On Wed, 28 Apr 2021 15:35:47 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> In os_aix.cpp we keep book about for reserved address ranges since we need to keep information about which API (mmap or shmat) had been used for this range. (code hint: see vmembk_(add|remove|find)).
> 
> When releasing memory via os::release_memory, we remove those ranges from the internal bookkeeping.
> 
> However, for mmap() the release may be partial. In that case the internal bookkeeping is not updated correctly: we just remove the record for the whole original address range. 
> 
> After that operation, the remainder address range is still mapped from the point of the VM, but the bookkeeping is lost and subsequent calls to os::release_memory or os::commit/uncommit_memory for this range will run into a guarantee.
> 
> Partial releases can happen e.g. when, in the process of running os::reserve_memory_aligned(), the extra-aligned pages are unmapped. This is of more concern now with the new Metaspace, since we now reserve with larger alignments (4G+).
> 
> This problem was hidden by the fact that we typically run in 64K paged mode where we use SysV shared memory; but if AME is enabled on AIX, 64K pages are disabled, and the VM falls back to 4K pages and using mmap, and runs into this bug.
> 
> Edit: Tested on AIX machines with and without AME activated, solves the problems. All SAP nightlies ran, no problems caused by this patch.

Danke, Martin!

-------------

PR: https://git.openjdk.java.net/jdk/pull/3765