RFR: 8346888: [ubsan] block.cpp:1617:30: runtime error: 9.97582e+36 is outside the range of representable values of type 'int'

[Dean Long]

On Thu, 20 Mar 2025 19:25:14 GMT, Dean Long <dlong at openjdk.org> wrote:

>>> There may be a bug in frequency propagation.
>> 
>> I haven't looked at these failures but could it have to do with irreducible loops? This is a known reason of frequency propagation inaccuracy in C2, see [JDK-8258895](https://bugs.openjdk.org/browse/JDK-8258895).
>
> @robcasloz , that could be it, but I'm not enough of an expert.  It seems to be this loop:
> https://github.com/openjdk/jdk/blob/56038fb5a156568cce2e80f5db18b10ad61c06e4/test/jdk/java/foreign/TestHandshake.java#L104
> and the block gets detected as empty and moved to the end by PhaseCFG::remove_empty_blocks().  Roberto, do you have time to help look at this?

What I'm seeing is a block with 2 NeverBranchNode successors, which get a _freq of 0 (later changed to MIN_BLOCK_FREQUENCY). 

B64: #	out( B66 B65 ) <- in( N298 N301 )  Freq: 0
 148  Loop  === 148 297 296  [[ 148 173 174 147 ]] inner  !jvms: TestHandshake$AbstractSegmentAccessor::run @ bci:12 (line 104)
 147  NeverBranch  === 148  [[ 172 146 ]] 
 172  CProj  === 147  [[ 301 ]] #0
 146  CProj  === 147  [[ 295 ]] #1

Initially these successor blocks are mostly empty:

B66: #	out( B64 ) <- in( N148 )  Freq: 0
 301  Region  === 301 172  [[ 301 170 ]] 
 296  branch  === 166  [[ 148 ]]  !orig=230


B65: #	out( B1 ) <- in( N148 )  Freq: 0
 295  Region  === 295 146  [[ 295 145 ]] 
 145  ShouldNotReachHere  === 295 0 0 19 0  [[ 1 ]] 

but later we add nodes to B66 for some reason.  Also B64 is detected as empty are moved to the end.  At some point the NeverBranch is changed into a branchNode MachGotoNode, and that causes succ_prob() to return 1 instead of 0 for these edges.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23962#discussion_r2006542741