RFR: 8333177: Invalid value used for enum Cell in ciTypeFlow::get_start_state [v2]

[Christian Hagedorn]

On Tue, 4 Jun 2024 10:17:14 GMT, Tobias Hartmann <thartmann at openjdk.org> wrote:

>> Ubsan detected undefined behavior in `ciTypeFlow::get_start_state` because an invalid value of `4294967295` is assigned to enum `Cell`:
>> https://github.com/openjdk/jdk/blob/ac7119f0d5319a3fb44dc67a938c3e1eb21b9202/src/hotspot/share/ci/ciTypeFlow.hpp#L150-L152
>> 
>> The problem is that if the C++ compiler decides to encode `Cell` with an unsigned int, casting a negative integer value will lead to an underflow and therefore a value > `Cell_max = INT_MAX`. Here, `state->tos()` returns a value < 0:
>> https://github.com/openjdk/jdk/blob/ac7119f0d5319a3fb44dc67a938c3e1eb21b9202/src/hotspot/share/ci/ciTypeFlow.cpp#L407
>> 
>> which is casted to a `Cell`:
>> https://github.com/openjdk/jdk/blob/ac7119f0d5319a3fb44dc67a938c3e1eb21b9202/src/hotspot/share/ci/ciTypeFlow.hpp#L211
>> 
>> I simply re-wrote the code to not require a negative `Cell` value to iterate over the locals and setting them to bottom type.
>> 
>> Thanks,
>> Tobias
>
> Tobias Hartmann has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update src/hotspot/share/ci/ciTypeFlow.cpp
>   
>   Co-authored-by: Christian Hagedorn <christian.hagedorn at oracle.com>

Looks good, thanks for the update!

-------------

Marked as reviewed by chagedorn (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/19520#pullrequestreview-2095970919