RFR: 8346714: [ASAN] compressedKlass.cpp reported applying non-zero offset to null pointer [v4]

[Thomas Stuefe]

On Sat, 21 Dec 2024 15:56:15 GMT, SendaoYan <syan at openjdk.org> wrote:

>> Hi all,
>> CompressedKlassPointers::sanity_check_after_initialization() src/hotspot/share/oops/compressedKlass.cpp:104:38 reported runtime error: applying non-zero offset 4294967296 to null pointer by clang17 UndefinedBehaviorSanitizer.
>> 
>> The _base initial as nullptr in function CompressedKlassPointers::initialize(address addr, size_t len) shows as below. In C/C++, offsetting a null pointer is undefined behavior. This PR do not change the original logic but eliminate the undefined behaviour in code, the risk is low.
>> 
>> ```c++
>>     address const end = addr + len;
>>     if (end <= (address)unscaled_max) {
>>       _base = nullptr;
>>       _shift = 0;
>
> SendaoYan has updated the pull request incrementally with one additional commit since the last revision:
> 
>   add function p2u and use function p2u instead of cast manually

src/hotspot/share/oops/compressedKlass.cpp line 99:

> 97:   // Check that Klass range is fully engulfed in the encoding range
> 98:   const address encoding_start = _base;
> 99:   const address encoding_end = (address)(p2u(_base) + (uintptr_t)nth_bit(narrow_klass_pointer_bits() + _shift));

nth_bit should already give us a 64-bit value, why the second cast?
I see that nth_bit returns an intptr_t - is the sign the problem? We may want to change that to uintptr_t...

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/22848#discussion_r1895392988