RFR: 8278020: ~13% variation in Renaissance-Scrabble

[Vladimir Kozlov]

On Tue, 14 Dec 2021 18:45:55 GMT, Ioi Lam <iklam at openjdk.org> wrote:

> We found that when CDS is enabled, there is a ~13% variation in the Renaissance-Scrabble benchmark between different builds of the JDK. In one example, only two core-lib classes, unrelated to the benchmark, changed between two builds, but one build is consistently faster than the other.
> 
> When CDS is disabled, we do not see such variations.
> 
> In the slow case, there seems to be frequent dcache misses when loading the `Klass::_vtable_len` field, which is at offset 24 from the beginning of the Klass (see [bug report](https://bugs.openjdk.java.net/browse/JDK-8278020) for details). 
> 
> We suspect that the problem is with the layout of the CDS archive. Specifically, in CDS, Klass objects are inter-mixed with other metadata objects (such as Methods). In contrast, when CDS is disabled, (on 64-bit platforms with compressed klass pointers), Klass objects are allocated in their own space, separated from other metadata objects.
> 
> My theory is: when CDS is enabled, perhaps the modification of an object that sits immediately above the Klass invalidates the cacheline that holds `Klass::_vtable_len`. In a different JDK build, the exact addresses of the metadata objects in the CDS archive may be slightly nudged so we don't see the cacheline effect anymore.
> 
> As an experiment, I swapped `Klass::_vtable_len` with `Klass::_modifier_flags` (which was at offset 164 before this patch), and the variation stopped. Both fields are 32 bits in size.
> 
> I have no concrete proof that my theory is correct, but this change seems to be harmless. @ericcaspole has run all the benchmarks in Oracle's CI and found consistent improvement with Renaissance-Scrabble, and no degradation in other benchmarks.

Yes, this small change is good for JDK 18.
Is it possible for JDK 19 (or later) to make CDS segmented to separate different types of data (Klass, Method, etc)?

An other experiment could be done is to add padding (space) before Klass data to make sure it is in different cache line.

-------------

PR: https://git.openjdk.java.net/jdk/pull/6838