Integrated: 8358751: C2: Recursive inlining check for compiled lambda forms is broken

[Vladimir Ivanov]

On Fri, 22 Aug 2025 01:24:52 GMT, Vladimir Ivanov <vlivanov at openjdk.org> wrote:

> Recursive inlining checks are relaxed for compiled LambdaForms. Since LambdaForms are heavily reused, the check is performed on `MethodHandle` receivers instead.
> 
> Unfortunately, the current implementation is broken. JVMState doesn't guarantee presence of receivers for caller frames.
> An attempt to fetch pruned receiver reports unrelated info, but, in the worst case, it ends up as an out-of-bounds access into node's input array and crashes the JVM.
>   
> Proposed fix captures receiver information as part of inlining and preserves it on `JVMState` for every compiled LambdaForm frame, so it can be reliably recovered during subsequent inlining attempts.  
> 
> Testing: hs-tier1 - hs-tier8
> 
> (Special thanks to @mroth23 who prepared a reproducer of the bug.)

This pull request has now been integrated.

Changeset: 9cca4f7c
Author:    Vladimir Ivanov <vlivanov at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/9cca4f7c760bea9bf79f7c03f37a70449acad51e
Stats:     76 lines in 4 files changed: 42 ins; 1 del; 33 mod

8358751: C2: Recursive inlining check for compiled lambda forms is broken

Reviewed-by: dlong, roland

-------------

PR: https://git.openjdk.org/jdk/pull/26891