RFR: 8338197: ubsan: ad_x86.hpp:6417:11: runtime error: shift exponent 100 is too large for 32-bit type 'unsigned int' [v3]

[Dean Long]

On Tue, 26 Aug 2025 10:51:43 GMT, Andrew Dinn <adinn at openjdk.org> wrote:

> If we are to change anything here then I think we need a review of the accuracy of pipeline models and their current or potential value before doing so.

That's a good point.  While looking into this, I discovered that the initial masks generated by pipeline_res_mask_initializer() appear wrong.  For example, the mask for stage 0 with 1 cycle is computed as 0x80000001, not the 0x1 that I would expect.  Stage 2 with 1 cycle is 0x2, not 0x4, etc.  I guess if all the masks are wrong in the same way, the problems might mostly cancel out, but it does shed doubt on the usefulness of this code.

We could preserve the large latencies for now, and let them trigger the _maxcycleused > 32 code for more platforms.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26890#issuecomment-3225410594