RFR: 8377011: Shenandoah: assert_bounds should be only called when boundaries have changed [v6]

[Kelvin Nilsen]

On Tue, 24 Feb 2026 00:58:44 GMT, Xiaolong Peng <xpeng at openjdk.org> wrote:

>> This fix the problem causing following 3 jtreg tests with tlab disabled to fail:
>> 
>> gc/shenandoah/TestRetainObjects.java#no-tlab 8361099 generic-all
>> gc/shenandoah/TestSieveObjects.java#no-tlab 8361099 generic-all
>> gc/shenandoah/TestSieveObjects.java#no-tlab-genshen 8361099 generic-all
>> 
>> 
>> 
>> The problem is assert_bounds is always called from function ShenandoahFreeSet::try_allocate_in, no matter the boundaries have changed or not after the allocation,  this is new behavior introduced by https://bugs.openjdk.org/browse/JDK-8365880, before which assert_bounds was only called after retiring a region.
>> 
>> The fix is to call assert_bounds only when boundaries have changed:
>> 1. When retire a region;
>> 2. When a the object is allocated in a new region - one region became non-empty.
>> 
>> There is also minor improvement to the if-else code in the PR.
>> 
>> Overall the change should be safe and only affect Shenandoah related tests, I have run Shenandoah test suite on MacOS.
>> 
>> Profiler data from the problematic jtreg test: [jtreg-gc_shenandoah_TestRetainObjects_no_tlab.html](https://github.com/user-attachments/files/25027324/jtreg-gc_shenandoah_TestRetainObjects_no_tlab.html) 
>> 
>> ### Tests
>> - [x] hotspot_gc_shenandoah
>> - [x] GHA
>
> Xiaolong Peng has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Add leftmost and rightmost asserts back

Thanks.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/jdk/pull/29537#pullrequestreview-3849665641