RFR: 8276201: Shenandoah: Race results degenerated GC to enter wrong entry point

[Aleksey Shipilev]

On Sat, 30 Oct 2021 12:56:38 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> There is subtle race when concurrent GC comes out of final mark safepoint: an allocation failure occurred before control thread checks OOM conditional, that triggers degenerated GC enters "mark" degenerated point.
> 
> Degenerated GC re-executes final mark, then switching off SATB, but it is already off, because concurrent GC already completed final mark, that triggers the assertion.
> 
> The solution is to consult concurrent_mark_in_progress flag when selects degen-point.
> 
> Test:
> - [x] hotspot_gc_shenandoah

Looks fine.

-------------

Marked as reviewed by shade (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/6179