RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded

[William Kemper]

On Thu, 21 Sep 2023 07:56:17 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 940:
> 
>> 938:   *gc_overhead_limit_was_exceeded = false;
>> 939:   ShenandoahAllocRequest req = ShenandoahAllocRequest::for_shared(size);
>> 940:   if (get_gc_no_progress_count() > ShenandoahNoProgressThreshold) {
> 
> Move this block to `ShenandoahHeap::allocate_memory`? This would require adding a (default) argument to that method, but it is a good tradeoff for putting all mechanics around this into a single method.

I put it here to keep it on the slow(er) path so that allocations for LABs won't have additional checks on the progress counter. Also, a failed LAB allocation will come back to `mem_allocate` to try for a shared allocation.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1333308520