RFR: 8346888: [ubsan] block.cpp:1617:30: runtime error: 9.97582e+36 is outside the range of representable values of type 'int'

[Matthias Baesken]

On Tue, 11 Mar 2025 00:02:14 GMT, Dean Long <dlong at openjdk.org> wrote:

>> When running jtreg tests on macOS aarch64 with ubsan - enabled binaries, in the test
>> java/foreign/TestHandshake
>> this error/warning is reported :
>>  
>> jdk/src/hotspot/share/opto/block.cpp:1617:30: runtime error: 9.97582e+36 is outside the range of representable values of type 'int'
>> UndefinedBehaviorSanitizer:DEADLYSIGNAL
>> UndefinedBehaviorSanitizer: nested bug in the same thread, aborting.
>>  
>> Seems it happens in this calculation (float value does not fit into an int) : int to_pct = (int) ((100 * freq) / target->_freq);
>
> src/hotspot/share/opto/block.cpp line 1617:
> 
>> 1615:           float f_from_pct = (100 * freq) / b->_freq;
>> 1616:           float f_to_pct = (100 * freq) / target->_freq;
>> 1617:           int from_pct = (f_from_pct < (float)INT_MAX) ? (int)f_from_pct : INT_MAX;
> 
> I think (float)INT_MAX is problematic.  Due to rounding, isn't the result actually greater than INT_MAX?
> Does it even make sense to have a "pct" that is greater than 100 here?
> Do we want `int from_pct = MIN2((double)INT_MAX, (double)f_from_pct);` or maybe
> `int from_pct = MIN2((100.0, (double)f_from_pct);`?

Hi Dean, I added the (float) cast because AIX failed to compile the file without it , for the other platforms/compilers it was okay without the cast.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23962#discussion_r1988646594