RFR: JDK-8310111: Shenandoah wastes memory when running with very large page sizes

[Thomas Stuefe]

This proposal changes the reservation of bitmaps and region storage to reduce the wastage associated with running with very large page sizes (e.g. 1GB on x64, 512M on arm64) for both non-THP- and THP-mode.

This patch does:
- introducing the notion of "allowed overdraft factor" - allocations for a given page size are rejected if they would cause more wastage than the factor allows
- if it makes sense, it places mark- and aux-bitmap into a contiguous region to let them share a ginourmous page. E.g. for a heap of 16G, both bitmaps would now share a single GB page.

Examples:

Note: annoyingly, huge page usage does not show up in RSS. I therefore use a script that parses /proc/pid/smaps and counts hugepage usage to count cost for the following examples:

Example 1:

A machine configured with 1G pages (and nothing else). Heap is allocated with 1G pages, the bitmaps fall back to 4K pages because JVM figures 1GB would be wasted:


thomas at starfish$ ./images/jdk/bin/java -Xmx4600m -Xlog:pagesize -XX:+UseShenandoahGC -XX:+UseLargePages
...
[0.028s][info][pagesize] Mark Bitmap: req_size=160M req_page_size=4K base=0x00007f8149fff000 size=160M page_size=4K
[0.028s][info][pagesize] Aux Bitmap: req_size=160M req_page_size=4K base=0x00007f813fffe000 size=160M page_size=4K
[0.028s][info][pagesize] Region Storage: req_size=320K req_page_size=4K base=0x00007f817c06f000 size=320K page_size=4K


Cost before: 8GB. Cost now: 5GB + (2*160M)

Example 2: JVM with 14GB heap: mark and aux bitmap together are large enough to justify another 1G page, so they share it. Notice how we also place the region storage on this page:


thomas at starfish:/shared/projects/openjdk/jdk-jdk/output-release$ ./images/jdk/bin/java -Xmx14g -Xlog:pagesize 
-XX:+UseShenandoahGC -XX:+UseLargePages -cp $REPROS_JAR de.stuefe.repros.Simple
[0.003s][info][pagesize] Heap: req_size=14G req_page_size=1G base=0x0000000480000000 size=14G page_size=1G
[0.003s][info][pagesize] Mark+Aux Bitmap: req_size=896M req_page_size=1G base=0x00007fee00000000 size=1G page_size=1G
[0.003s][info][pagesize] Region Storage: piggy-backing on Mark Bitmap: base=0x00007fee38000000 size=1792
<press key>


Cost before: 17GB. Cost now: 15GB.

>From a bang-for-hugepages-buck multiples of 16GB are a sweet spot here since (on x64 with 1GB pages) since this allows us to put both 512m bitmaps onto a single huge page.

-----------

No test yet, since I wanted to see if people reject this proposal. It does add a bit of complexity. If this is well-received, I'll do some tests.

-------------

Commit messages:
 - Cowabunga
 - wipwopwip
 - wipwip
 - wip
 - wip
 - start
 - JDK-8310110-Trace-page-sizes

Changes: https://git.openjdk.org/jdk/pull/14559/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14559&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8310111
  Stats: 386 lines in 5 files changed: 327 ins; 27 del; 32 mod
  Patch: https://git.openjdk.org/jdk/pull/14559.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/14559/head:pull/14559

PR: https://git.openjdk.org/jdk/pull/14559