RFR: 8374043: C2: assert(_base >= VectorMask && _base <= VectorZ) failed: Not a Vector [v2]

[Vladimir Ivanov]

On Mon, 19 Jan 2026 03:05:24 GMT, Xiaohong Gong <xgong at openjdk.org> wrote:

>> src/hotspot/share/opto/vectorIntrinsics.cpp line 625:
>> 
>>> 623:   }
>>> 624: 
>>> 625:   const TypeVect* mask_vt = TypeVect::makemask(elem_bt, num_elem);
>> 
>> Doesn't the same reasoning apply to vector intrinsics? If `mask_vec` and `opd` aren't TOP, they should produce vector values. So, additional input validation should rule out the problematic scenario.
>
> Vector intrinsics looks safer to me now. The APIs are inlined in an even earlier optimization stage, and the nodes are almost new created ones. Regarding to `unbox_vector()`, it either returns a new created `VectorUnboxNode` or a GVN transformed node of `VectorUnbox`. Currently there is not `TOP` input check for `VectorUnboxNode` itself during GVN.
> It might be an issue that we need to revisit once we add such checks for vector nodes. 
> 
> I agree with that additional input validation should be better. We can abort the API inlining as early as possible. Code may look like:
> 
> Node* mask_vec = unbox_vector(mask, mask_box_type, elem_bt, num_elem);
> if (mask_vec == nullptr || gvn().type(mask_vec) == Type::TOP) {
>     log_if_needed("  ** unbox failed mask=%s",
>                       NodeClassNames[argument(4)->Opcode()]);
>     return false;
> }
> 
> That looks a common issue for all APIs that we'd better fix for all code after `unbox_vector` ? I'm unsure whether I have to do this regarding to the issue this PR reported. Or maybe we could revisit the whole file in future. Any suggestions?

I'd suggest to shape it as follows:

Node* GraphKit::unbox_vector(Node* v, const TypeInstPtr* vbox_type, BasicType elem_bt, int num_elem) {
  ...
  Node* unbox = gvn().transform(new VectorUnboxNode(C, vt, v, merged_memory()));
  if (!gvn().type(unbox)->isa_vect()) {
    assert(gvn().type(unbox) == Type::TOP, "sanity");
    return nullptr; // not a vector
  }
  return unbox;
}

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/29057#discussion_r2709783849