[lworld] RFR: 8332406: [lworld] C2: Turn flat in array property into proper lattice in type system to address remaining issues [v5]

[Tobias Hartmann]

On Tue, 9 Dec 2025 07:27:08 GMT, Christian Hagedorn <chagedorn at openjdk.org> wrote:

>> This patch turns the boolean flag `_flat_in_array` into a new `FlatInArray` enum to properly define a (Boolean) lattice. This mainly allows us to cleanly represent "maybe flat in array" and "not flat in array". The dedicated top element is the dual of "maybe flat in array".
>> 
>> To simplify the review and to better understand what the changes are, I added code comments directly in the PR.
>> 
>> Here is a high-level overview of the changes:
>> - `FlatInArray` enum to define a lattice for `flat_in_array`:
>> https://github.com/openjdk/valhalla/blob/6b1930c7b9a359223a998d0086a1326a9d7905e7/src/hotspot/share/opto/type.hpp#L1182-L1199
>> - Updated `meet` operations accordingly. Needed to be careful when doing the meet above the centerline.
>> - Added single `compute_flat_in_array()` method to properly compute the `flat_in_array` property from the information we have when we need to set flat in array newly (this was no properly done before).
>> - In `flatten_alias_type()`, we flatten to "maybe flat in array".
>> 
>> #### Testing:
>> - t1-4 + valhalla-comp-stress
>> 
>> Thanks for your feedback,
>> Christian
>
> Christian Hagedorn has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision:
> 
>  - review Frederic
>  - Merge branch 'lworld' into flat-in-array-rework
>  - review Tobias
>  - Apply suggestions from code review
>    
>    Co-authored-by: Tobias Hartmann <tobias.hartmann at oracle.com>
>  - update description
>  - add comment
>  - 8332406: [lworld] C2: Turn flat in array property into proper lattice in type system to address remaining issues

Marked as reviewed by thartmann (Committer).

-------------

PR Review: https://git.openjdk.org/valhalla/pull/1774#pullrequestreview-3555862194