[lworld] RFR: 8332406: [lworld] C2: Turn flat in array property into proper lattice in type system to address remaining issues [v3]

[Christian Hagedorn]

On Mon, 8 Dec 2025 09:45:59 GMT, Christian Hagedorn <chagedorn at openjdk.org> wrote:

>> This patch turns the boolean flag `_flat_in_array` into a new `FlatInArray` enum to properly define a (Boolean) lattice. This mainly allows us to cleanly represent "maybe flat in array" and "not flat in array". The dedicated top element is the dual of "maybe flat in array".
>> 
>> To simplify the review and to better understand what the changes are, I added code comments directly in the PR.
>> 
>> Here is a high-level overview of the changes:
>> - `FlatInArray` enum to define a lattice for `flat_in_array`:
>> https://github.com/openjdk/valhalla/blob/6b1930c7b9a359223a998d0086a1326a9d7905e7/src/hotspot/share/opto/type.hpp#L1182-L1199
>> - Updated `meet` operations accordingly. Needed to be careful when doing the meet above the centerline.
>> - Added single `compute_flat_in_array()` method to properly compute the `flat_in_array` property from the information we have when we need to set flat in array newly (this was no properly done before).
>> - In `flatten_alias_type()`, we flatten to "maybe flat in array".
>> 
>> #### Testing:
>> - t1-4 + valhalla-comp-stress
>> 
>> Thanks for your feedback,
>> Christian
>
> Christian Hagedorn has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Apply suggestions from code review
>   
>   Co-authored-by: Tobias Hartmann <tobias.hartmann at oracle.com>

Thanks Tobias for your review! I pushed updates to address your comments.

-------------

PR Comment: https://git.openjdk.org/valhalla/pull/1774#issuecomment-3626020256