RFR: 8373468: Shenandoah: Mutator may block at _gc_waiters_lock after allocation failure even block parameter is false [v5]

[Xiaolong Peng]

On Thu, 11 Dec 2025 18:56:15 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> There is behavior discrepancy between Shenandoah generational mode and non-generational when it handles mutator allocation failure, as stated in the description of [JDK-8373468](https://bugs.openjdk.org/browse/JDK-8373468), in non-generational, the `block` parameter may not always work.
>> 
>> Further looking into ShenandoahGenerationalControlThread and ShenandoahControlThread, they handle the _requested_gc_cause and ShenandoahHeap::cancelled_cause differently in other places as well, the change in this PR will minimize the gap to unify the behavior, and also fix potentially missed allocation failure notifications in some rare cases for both control threads.
>> 
>> ### Test
>> - [x] hotspot_gc_shenandoah
>> - [x] GHA (includes tier1 but no unit unit with gtest) 
>> - [x] gtest
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   format issue

Closing this PR as we may not want to change the behavior of classic Shenandoah as long as it is not broken.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/28758#issuecomment-3842535770