RFR: 8372513: Shenandoah: ShenandoahMaxRegionSize can produce an unaligned heap alignment

[Stefan Karlsson]

On Tue, 25 Nov 2025 14:19:53 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

> While rewriting some of the heap size initialization code we hit a corner-case where the setting of `ShenandoahMaxRegionSize` to something that isn't a power-of-2 will hit an assert in `max_heap_for_compressed_oops`.
> 
> When running with:
> 
> java -XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:ShenandoahMaxRegionSize=33m -version
> 
> 
>  The following code:
> 
>   size_t displacement_due_to_null_page = align_up(os::vm_page_size(),
>                                                   _conservative_max_heap_alignment)
> 
> triggers:
> 
> # assert(is_power_of_2(alignment)) failed: must be a power of 2: 34603008
> 
> because `_conservative_max_heap_alignment` is not a power-of-2.
> 
> This happens because Shenandoah's `conservative_max_heap_alignment()`
>  returns a potentially unaligned `ShenandoahMaxRegionSize` value.
> 
> 
> size_t ShenandoahArguments::conservative_max_heap_alignment() {
>   size_t align = ShenandoahMaxRegionSize;
>   if (UseLargePages) {
>     align = MAX2(align, os::large_page_size());
>   }
>   return align;
> }
> 
> 
> I propose a small fix to adjust `align` to be a power-of-2. I've also added an earlier assert about this in `set_conservative_max_heap_alignment` and added an additional test-case in TestRegionSizeArgs.java
> 
> WDYT, is this an OK fix for this corner-case?

I realized that having an unconstrained flag like this can cause overflows and asserts because of that. If we want to fix that we could do something like the following:

https://github.com/stefank/jdk/compare/8372513_shenandoah_max_region_size...stefank:jdk:8372513_shenandoah_max_region_size_constraints

Then running

 java -XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:ShenandoahMaxRegionSize=9223372036854775808 -version


Will give this error message instead of an assert:

$ java -XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:ShenandoahMaxRegionSize=9223372036854775808 -version
ShenandoahMaxRegionSize 8589934592G should be lower than (8589934592G).
Improperly specified VM option 'ShenandoahMaxRegionSize=9223372036854775808'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/28492#issuecomment-3576326172