RFR: 8356865: C2: Unreasonable values for debug flag FastAllocateSizeLimit can lead to left-shift-overflow, which is UB

[Dean Long]

On Tue, 17 Jun 2025 16:39:56 GMT, Evgeny Astigeevich <eastigeevich at openjdk.org> wrote:

>> This PR adds a range constraint for the `-XX:FastAllocateSizeLimit` debug flag. This prevents undefined behavior caused by left-shift overflow of the flag value in `GraphKit::new_array`.
>> 
>> ### Testing
>> - [x] [GitHub Actions](https://github.com/benoitmaillard/jdk/actions?query=branch%3AJDK-8356865)
>> - [x] tier1-3, plus some internal testing
>> - [x] Manual testing with values known to previously cause undefined behavior
>> 
>> Thanks!
>
> src/hotspot/share/opto/graphKit.cpp line 3807:
> 
>> 3805:     int log2_esize = Klass::layout_helper_log2_element_size(layout_con);
>> 3806:     fast_size_limit <<= (LogBytesPerLong - log2_esize);
>> 3807:     assert (fast_size_limit > 0, "increasing the size limit should not produce negative values");
> 
> Prior C++14 left shit producing a negative value is undefined behavior: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2161.pdf
> 
> Do we compile c++ source specifying the C++ standard?

Yes we use -std=c++14, but creating a negative value in this way still feels like a kind of overflow to me.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/25834#discussion_r2152762023