RFR: 8342772: Assert in LateInlineMHCallGenerator::do_late_inline_check

[Damon Fenacci]

On Tue, 6 Jan 2026 22:15:42 GMT, Vladimir Ivanov <vlivanov at openjdk.org> wrote:

>> # Issue
>> The assertion
>> https://github.com/openjdk/jdk/blob/1c16d8a900928ebfa5e2343cf33312539509e815/src/hotspot/share/opto/callGenerator.cpp#L421
>> in `LateInlineMHCallGenerator::do_late_inline_check` fails while running a 24h-long Renaissance benchmark test.
>> 
>> # Cause
>> The assert failure is due to both `!cg->is_late_inline()` and `cg->is_mh_late_inline()` being false because the CallGenerator `cg` is of type `LateInlineVirtualCallGenerator`. `cg` is created just above by calling `for_method_handle_inline`. The only way for `cg` to be of type `LateInlineVirtualCallGenerator` is that `for_method_handle_inline` (which finds out that the intrinsic id is `vmIntrinsics::_linkToInterface`) calls `optimize_virtual_call`, which apparently cannot devirtualize the call, and then calls `call_generator` and this creates and returns a `LateInlineVirtualCallGenerator` (at the end of the method).
>> 
>> # Fix
>> There seem to be no apparent reason why the CallGenerator returned by `for_method_handle_inline` couldn't be of type `LateInlineVirtualCallGenerator`. So the sensible fix is to relax the assert to accommodate this type of call generator.
>> 
>> # Testing
>> Tier 1-3+
>> Failing test before and after.
>> 
>> Unfortunately it has proven impractical to create a specific test that consistently (or even only intermittently) reproduces the  issue.
>
> Looks good.

Thanks @iwanowww @TobiHartmann @chhagedorn for reviewing!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/28793#issuecomment-3717648402