RFR: 8352140: UBSAN: fix the left shift of negative value in klass.hpp, array_layout_helper()

[Dean Long]

On Mon, 24 Mar 2025 09:52:43 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

> The `array_layout_helper()` with `jint tag` as its first arg, is called with a `tag` whose sign-bit is always set and considered as negative. This negative value is UB in left-shift operation.  Changing the type to `juint`  fixes this.
> 
> Tests:
> linux-x64-debug tier1 with UBSAN enabled.

src/hotspot/share/oops/klass.hpp line 527:

> 525:   }
> 526:   static jint array_layout_helper(jint tag, int hsize, BasicType etype, int log2_esize) {
> 527:     return ((juint)tag << _lh_array_tag_shift)

Doesn't this turn the type of the return expression to unsigned, causing an implicit conversion back to signed, which is implementation-defined?  I think that's the reason for the weird-looking reinterpret_cast<> in JAVA_INTEGER_OP.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/24184#discussion_r2011376367