RFR: JDK-8262074: Investigate defaults for MetaspaceSize

[Thomas Stuefe]

I was looking at whether the default values for MetaspaceSize (the initial threshold to start off a metaspace-motivated GC) still make sense after JEP-387.

The default is dependent on compiler tier and bitness. It is also spread across all platforms.

In addition to that, it also may get modified after Metaspace::ergo_initialize() in client-compiler-emulation-mode:

https://github.com/openjdk/jdk/blob/2b00367e1154feb2c05b84a11d62fb5750e46acf/src/hotspot/share/compiler/compilerDefinitions.cpp#L194-L196

which is unexpected and causes confusion (eg JDK-8261907, JDK-8261907).

The reasons for this seem to originate from PermGen times:
https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2021-February/045536.html

----

Today, MetaspaceSize defaults to:

- no compiler (eg Zero):      **4M** (32bit)   **5.19M** (64bit)
- C1-only build:  	   **12M**
- C1+C2 build (standard):    **16M** 	(32bit) **20,75M** (64bit)

I was surprised to see that they do not depend on any compiler *runtime* switches. It only depends on build time decisions.

---

How much do we use? I analyzed a simple java app to see the difference VM settings make on initial metaspace consumption. Committed space, used in brackets:


(Note:                  (used)          committed
CDS on:

64bit:       		(181,58 KB)     384 KB  (a)
64bit tier1 only:       (170,04 KB)     384 KB
64bit Xint:		(16,62 KB)      256 KB

32bit    		(178 KB)        256 KB
32bit tier1 only: 	(144 KB)        256 KB
32bit Xint: 		(11 KB) (b)     128 KB  

CDS off:

64bit:			(5,06 MB)       5.62 MB
64bit tier1 only:       (5,00 MB)       5,56 MB
64bit Xint:		(4,84 MB)       5.44 MB

32bit    		(3,69 MB)       3.75 MB
32bit tier1 only: 	(3,65 MB)       3.75 MB
32bit Xint: 		(3,52 MB)       3.62 MB

Class space on/off

CDS off, 64bit, +CompressedClassPointers: 5.44M
CDS off, 64bit, -CompressedClassPointers: 5.38M


_Notes:
(a) Since JEP-387, with CDS=on, we pay very little committed footprint upfront (384K). For comparison, JDK 15 commits here 5.75M.
(b) The seemingly high difference between Xint and C1+C2 - 11K vs 178K - is misleading: All initial classes get compiled, but since most of their metadata live in CDS, not in Metaspace, all we allocate at the start are MethodCounters. Hence, with -Xint, we almost allocate nothing. That changes as soon as we start loading application classes._

Conclusions:
- CDS=off increases metaspace footprint by a flat amount, in my case ~5MB, which makes sense.
- Running with (any) compiler has not much influence once we start using Metaspace for real. The difference between C1-only and C1+C2 is neglectible, the difference between Xint and C1+C2 amounts to about 2% wrt to initial metaspace consumption.
- Running with or without compressed Klass pointers makes not much difference. With class space, we pay for certain overhead twice, but at this early stage this is not noticeable.
- The difference between 64bit and 32bit is more like 1.4-1.5, not the 1.3 factor we currently assume

-----

Proposal:

1) I propose to make MetaspaceSize independent from compiler. For one, if the intention was to have a lower threshold with compilers deactivated, that has never worked. E.g. on 64bit we always had a threshold of 20.75MB regardless of Xint/TieredStopAtLevel. Even if it worked, the compiler does not make that much difference in metaspace footprint.

2) I propose to slightly lower MetaspaceSize - on 32bit from 16M to 14M, on 64bit from 20.75M to 20M. This takes the slightly lower metaspace footprint since JEP 387 into account (less waste) and the scale I found to be higher than 1.3. 

This is all very cautious. For the standard VM, very little changes, so this is mainly a cleanup patch. We could probably tune MetaspaceSize down to much lower levels. And/or make size it differently depending on UseSharedSpaces. However, atm I don't have time to hunt regressions due to too early GCs.

-----

Tests: GA, nightlies at SAP

-------------

Commit messages:
 - Start

Changes: https://git.openjdk.java.net/jdk/pull/2675/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2675&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8262074
  Stats: 28 lines in 13 files changed: 0 ins; 27 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2675.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2675/head:pull/2675

PR: https://git.openjdk.java.net/jdk/pull/2675