RFR: JDK-8312018: Improve zero-base-optimized reservation of class space

[Thomas Stuefe]

TL;DR This patch introduces a new reservation API to reserve memory in low address space; depending on the OS, it may use optimized placement techniques. That new API is used to optimize the placement of class space and CDS for zero-based encoding. 

A future RFE will use the same API to optimize the zero-based heap reservation and thereby consolidate a lot of coding. We also plan to use this API in other places, e.g. for Shenandoah CollectionSet reservation.

-------

With CDS off or at dump time, we currently attempt to optimize class space location by reserving in low address ranges.

We do this by examining the *java heap* end (which has been allocated at that point) and, if that had been allocated in lower address regions, attempt to allocate adjacent to it. Essentially, we piggyback on what we hope for is an optimized heap placement. If that fails, we attempt to map at HeapBaseMinAddress.

This approach has many disadvantages:

- it depends on the VM either using CompressedOops and getting a zero-based heap or the region around HeapBaseMinAddress being free. 
 
- HeapBaseMinAddress is an odd choice: it is 2G on all platforms, for reasons unknown to me, but that denies us half of the valuable low address range below 4G right away.

- We only get 1 shot. It's either one of these two addresses.

- And we only use this strategy for CDS=off or CDS=dumptime; we don't use it for the CDS-runtime-fallback-case when attaching to the primary attach point failed.

- It assumes narrow Klass encoding uses the same geometry (bit size, shift) as narrow Oops, which is not guaranteed with future developments (lilliput).

- It actually reduces the chance of getting a zero-based *java heap*. This is because when attempting to place the heap, we leave a gap for what we assume will be the later class space. That gap is `CompressedClassSpaceSize` bytes, which is often grossly over-dimensioned. A zero-based heap is more valuable than a zero-based class space. Therefore the heap should get the best chance of low-address heap reservation.

- It introduces an unnecessary dependency between heap reservation, narrow Oop encoding, and class space reservation. That makes the code base brittle.

- Getting the heap region to place class space adjacent to it is actually tricky. We lack a common get-heaprange-API because ZGC. This code misuses the CompressedOops interface. But CompressedOops is the encoding range and thus only loosely correlated to the heap range (the latter must contain the former). The fact that `CompressedOops::end()` returns the heap range end can be seen as aberration since the actual encoding range end usually far outreaches the heap range end. But as long this code relies on it returning the heap range end, we cannot fix that.

-------------

This RFE instead proposes a different approach:

- Let us have an API, `os::attempt_reserve_memory_below(address max, ...)`. This API will do its best to reserve memory with a given size and alignment below a given max address. It will, on supporting OSes, attempt to use OS-specific means to find a suitable address space hole to place the reservation in. Otherwise, it will do the typical ladder-reservation approach with an adjustable maximum number of tries.

- Let's use this API to reserve zero-base-friendly class space. Let's remove all knowledge of heap and CompressedOops. Now we are independent of what the heap does. It may or may not be located in lower address ranges. If it is, the new API will work around it and find a gap to place the class space if it is not, even better.

- Let's remove the "leave a gap for class space" logic from Heap reservation. We don't need it. It is harmful: Heap should have the best chance for zero-based - if I only can have one, I rather have a zero-based heap than a zero-based class space.

The end result will be a JVM with a much better chance to get zero-based class space *and* zero-based heap; we will have removed dependencies between heap and class space; we will have an API that can be used for similar problems (e.g. an obvious future enhancement would be to use this new reservation API for zero-based heap reservation as well, and other places could use it too, eg. Shenandoah CollectionSet reservation).

-------------

Commit messages:
 - better zero-based reservation strategy

Changes: https://git.openjdk.org/jdk/pull/14867/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14867&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8312018
  Stats: 490 lines in 11 files changed: 458 ins; 30 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/14867.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/14867/head:pull/14867

PR: https://git.openjdk.org/jdk/pull/14867