Integrated: 8278020: ~13% variation in Renaissance-Scrabble

[Ioi Lam]

On Tue, 14 Dec 2021 18:45:55 GMT, Ioi Lam <iklam at openjdk.org> wrote:

> We found that when CDS is enabled, there is a ~13% variation in the Renaissance-Scrabble benchmark between different builds of the JDK. In one example, only two core-lib classes, unrelated to the benchmark, changed between two builds, but one build is consistently faster than the other.
> 
> When CDS is disabled, we do not see such variations.
> 
> In the slow case, there seems to be frequent dcache misses when loading the `Klass::_vtable_len` field, which is at offset 24 from the beginning of the Klass (see [bug report](https://bugs.openjdk.java.net/browse/JDK-8278020) for details). 
> 
> We suspect that the problem is with the layout of the CDS archive. Specifically, in CDS, Klass objects are inter-mixed with other metadata objects (such as Methods). In contrast, when CDS is disabled, (on 64-bit platforms with compressed klass pointers), Klass objects are allocated in their own space, separated from other metadata objects.
> 
> My theory is: when CDS is enabled, perhaps the modification of an object that sits immediately above the Klass invalidates the cacheline that holds `Klass::_vtable_len`. In a different JDK build, the exact addresses of the metadata objects in the CDS archive may be slightly nudged so we don't see the cacheline effect anymore.
> 
> As an experiment, I swapped `Klass::_vtable_len` with `Klass::_modifier_flags` (which was at offset 164 before this patch), and the variation stopped. Both fields are 32 bits in size.
> 
> I have no concrete proof that my theory is correct, but this change seems to be harmless. @ericcaspole has run all the benchmarks in Oracle's CI and found consistent improvement with Renaissance-Scrabble, and no degradation in other benchmarks.

This pull request has now been integrated.

Changeset: 4ba980ba
Author:    Ioi Lam <iklam at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/4ba980ba439f94a6b5015e64382a6c308476d63f
Stats:     6 lines in 1 file changed: 3 ins; 0 del; 3 mod

8278020: ~13% variation in Renaissance-Scrabble

Reviewed-by: dholmes, stuefe, kvn

-------------

PR: https://git.openjdk.java.net/jdk/pull/6838