RFR: Change affiliation representation [v2]

[Y. Srinivas Ramakrishna]

On Wed, 16 Nov 2022 14:43:08 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> With generational mode of Shenandoah, each region is associated with OLD, YOUNG, or FREE.  During certain marking and update-refs activities, the region affiliation is frequently consulted.  In the original implementation, the region affiliation was stored in a field of the ShenandoahHeapRegion object.  In this code, we maintain a separate array, indexed by region number, to represent the affiliation of each region.  This saves one level of indirection and improves cache locality for looking up the affiliation of each region.
>> 
>> Measurements show significant improvement in throughput.  One workload that was configured to perform back-to-back old-gen collections was able to increase the frequency of old-gen collections by almost 5 fold.  With a 20-minute Extremem workload using a 48G heap, 20G old-gen, the P95 latency improvement was 0.54% (2.395 ms) and the P99.999 latency improvement was 58.21% (29.195 ms) in comparison to the implementation before this patch.
>
> Kelvin Nilsen has updated the pull request incrementally with six additional commits since the last revision:
> 
>  - Add comment to clarify that _affiliation required for non-generational
>  - Add assertions to clarify locks required for setting affiliation
>  - Adjust asserts for heap lock
>  - Undo change to _affiliations add assertion to clarify heap lock
>  - Only initialize _affiliations if mode is generational
>  - Only initialize _affiiations in generational mode

src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 619:

> 617:   //                       YOUNG      L       X
> 618:   //                         OLD      L       X      X
> 619:   //  X means state transition won't happen (so don't care)

Nits:
1. Instead of the blank entry in the cell for Young->Old, may be use `N` and replace `Blank` in the legend below?
2. If we don't expect some transitions (and believe those are incorrect), it might make sense to assert that in `set_affiliation()`?
3. Further below, it looks like all current (and expected) uses of `assert_lock_for_affiliation` will be from assertion checking code in non-release builds. In that case, you can avoid the `#ifdef ASSERT` verbage at call sites by declaring the method as `PRODUCT_RETURN` which should elide the empty method in release builds.

-------------

PR: https://git.openjdk.org/shenandoah/pull/170