Integrated: 8338197: [ubsan] ad_x86.hpp:6417:11: runtime error: shift exponent 100 is too large for 32-bit type 'unsigned int'

[Boris Ulasevich]

On Fri, 22 Aug 2025 00:47:48 GMT, Boris Ulasevich <bulasevich at openjdk.org> wrote:

> This reworks the recent update https://github.com/openjdk/jdk/pull/24696 to fix a UBSan issue on aarch64. The problem now reproduces on x86_64 as well, which suggests the previous update was not optimal.
> 
> The issue reproduces with a HeapByteBufferTest jtreg test on a UBSan-enabled build. Actually the trigger is `XX:+OptoScheduling` option used by test (by default OptoScheduling is disabled on most x86 CPUs). With the option enabled, the failure can be reproduced with a simple `java -version` run.
> 
> This fix is in ADLC-generated code. For simplicity, the examples below show the generated fragments.
> 
> The problems is that shift count `n` may be too large here:
> 
> class Pipeline_Use_Cycle_Mask {
> protected:
>   uint _mask;
>   ..
>   Pipeline_Use_Cycle_Mask& operator<<=(int n) {
>     _mask <<= n;
>     return *this;
>   }
> };
> 
> The recent change attempted to cap the shift amount at one call site:
> 
> class Pipeline_Use_Element {
> protected:
>   ..
>   // Mask of specific used cycles
>   Pipeline_Use_Cycle_Mask _mask;
>   ..
>   void step(uint cycles) {
>     _used = 0;
>     uint max_shift = 8 * sizeof(_mask) - 1;
>     _mask <<= (cycles < max_shift) ? cycles : max_shift;
>   }
> }
> 
> However, there is another site where `Pipeline_Use_Cycle_Mask::operator<<=` can be called with a too-large shift count:
> 
> // The following two routines assume that the root Pipeline_Use entity
> // consists of exactly 1 element for each functional unit
> // start is relative to the current cycle; used for latency-based info
> uint Pipeline_Use::full_latency(uint delay, const Pipeline_Use &pred) const {
>   for (uint i = 0; i < pred._count; i++) {
>     const Pipeline_Use_Element *predUse = pred.element(i);
>     if (predUse->_multiple) {
>       uint min_delay = 7;
>       // Multiple possible functional units, choose first unused one
>       for (uint j = predUse->_lb; j <= predUse->_ub; j++) {
>         const Pipeline_Use_Element *currUse = element(j);
>         uint curr_delay = delay;
>         if (predUse->_used & currUse->_used) {
>           Pipeline_Use_Cycle_Mask x = predUse->_mask;
>           Pipeline_Use_Cycle_Mask y = currUse->_mask;
> 
>           for ( y <<= curr_delay; x.overlaps(y); curr_delay++ )
>             y <<= 1;
>         }
>         if (min_delay > curr_delay)
>           min_delay = curr_delay;
>       }
>       if (delay < min_delay)
>       delay = min_delay;
>     }
>     else {
>       for (uint j = predUse->_lb; j <= predUse->_ub; j++) {
>         const Pipeline_Use_Element *currUse = element(j);
>         if (predUse->_used & currUse->_used) {
>  ...

This pull request has now been integrated.

Changeset: fa3af820
Author:    Boris Ulasevich <bulasevich at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/fa3af820ad310704e8d25cf496f676e09d60797d
Stats:     17 lines in 1 file changed: 0 ins; 6 del; 11 mod

8338197: [ubsan] ad_x86.hpp:6417:11: runtime error: shift exponent 100 is too large for 32-bit type 'unsigned int'

Reviewed-by: kvn, dlong

-------------

PR: https://git.openjdk.org/jdk/pull/26890