RFR: 8346888: [ubsan] block.cpp:1617:30: runtime error: 9.97582e+36 is outside the range of representable values of type 'int'

[Dean Long]

On Tue, 11 Mar 2025 09:03:45 GMT, Matthias Baesken <mbaesken at openjdk.org> wrote:

>> src/hotspot/share/opto/block.cpp line 1617:
>> 
>>> 1615:           float f_from_pct = (100 * freq) / b->_freq;
>>> 1616:           float f_to_pct = (100 * freq) / target->_freq;
>>> 1617:           int from_pct = (f_from_pct < (float)INT_MAX) ? (int)f_from_pct : INT_MAX;
>> 
>> I think (float)INT_MAX is problematic.  Due to rounding, isn't the result actually greater than INT_MAX?
>> Does it even make sense to have a "pct" that is greater than 100 here?
>> Do we want `int from_pct = MIN2((double)INT_MAX, (double)f_from_pct);` or maybe
>> `int from_pct = MIN2((100.0, (double)f_from_pct);`?
>
>> Adding to @dean-long's questions, I was wondering how we can get to a 9.97582e+36 value (since it is a runtime ubsan issue): is this a result of successive rounding-ups or there is perhaps an upstream issue?
> 
> That's a good question. I could add a bit of tracing or asserts to find out more about this.
> It seems this is an aarch64 related thing because on (Linux) x86_64/ppc64le  I never observed this.
> Do you think those high values are not expected ?

Also, to compute `from_pct`, we end up multiplying and then dividing by the same value `b->_freq`, which cancel out and simplify to `100 * b->succ_prob(j)`.  Furthermore, succ_prob() should always return a value between 0.0 and 1.0, so the real problem is probably only `to_pct` and very small values of `target->_freq`.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23962#discussion_r1990235113