RFR: 8371964: C2 compilation asserts with "Unexpected load/store size"

[Christian Hagedorn]

On Thu, 20 Nov 2025 08:42:46 GMT, Quan Anh Mai <qamai at openjdk.org> wrote:

> Hi,
> 
> This fixes the crash in `Load/StoreVectorMaskedNode::Ideal`. The issue here is that the graph is not canonical during idealization, which leads to us processing a dead node. The fix I propose is to bail-out when that happens.
> 
> To be more specific, for this issue, we have the graph that looks like:
> 
>     ConI -> ConvI2L -> CastLL(0..32) -> VectorMaskGen
> 
> with `ConI` being 45 and `MaxVectorSize` being 32. In this instance, `CastLL` is processed before `ConvI2L`, and when it is processed, it sees that the type of `ConvI2L` being its bottom type. As a result, it does not know that it is top, and since we are after macro expansion, which is after loop opts, the `CastLL` goes away, leaving us with:
> 
>     ConI -> ConvI2L -> VectorMaskGen
> 
> After `ConvI2L` is processed, we know that the input of `VectorMaskGen` is a constant 45, which is larger than `MaxVectorSize`, leading to the assert failure.
> 
> Please take a look and leave your thoughts, thanks a lot.

src/hotspot/share/opto/vectornode.cpp line 1152:

> 1150:         // Dead node, should go away
> 1151:         return nullptr;
> 1152:       }

Do I understand correctly that this widening/removal of the `CastLL` node is happening on an actual dead path that is going to be removed anyway?

It sounds like this problem is specific to post loop opts IGVN phases where we are allowed to widen `CastII/LL` nodes. Could we assert that this bailout only happens after post loop opts? 

Apart from that, I think your fix is reasonable. Were you able to also extract a reproducer?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/28410#discussion_r2545153008